Analysis of design research data begins before we think it does

Designers have a rationale that they use for up-front research in a human-centered design context: Problems are more efficiently solved if understood correctly and addressed early.

You want to be prepared, set yourself up for success, emphasize agile personal skills, and distance the obligations of employers and work environments. This is familiar territory for many researchers who rarely get to dictate the terms for proactive groundwork, and who usually need to manage reactive studies.

Preparing well includes considering things early while also remaining flexible to change. The aim is to develop a big-picture view of the research work while also tempering over-thinking and under-resourcing with lightly-held opinions about the outcomes. Embodying the values of ‘Semper Gumby’ and adapting to change while planning the research and approaching a state of readiness for the analysis itself.

What that preparation and readiness will look like in practice is determined by the data, the methods used to collect it, and the purposes of your research efforts. The secret to effectively navigating analysis lies in recognizing the connections between these elements.

A holistic view supports flexibility

Often, in an attempt to provide rigor and depth to the work, the research process is deconstructed into discrete stages and substages related to planning, collection, or analysis. This kind of partitioning might align with high-level decisions or allow focus. Still, it can also impact your opportunity to be mindfully adaptive to the inevitable challenges.

It is important to monitor how assumptions and hypotheses are forming and how the data’s relevance and quality appear in light of your research questions. This kind of analysis can begin as soon as the raw information can be accessed. There is no need to wait until after collection to start prodding and poking to see what the investigations are about.

The advantages of early analysis

The practice of taking small, analytical bites into research as early as reasonably possible is called ‘periodic analysis’. Most experienced qualitative researchers consider it good practice.

Doing analysis concurrently with data collection can be very rewarding for several reasons:

• You get to see what the data looks like as you're collecting it.

• You can collect new data to balance any critical gaps.

• You can identify if you're asking the wrong questions and fix them.

• You are afforded the opportunity to question routine assumptions and form rivaling hypotheses.

• You are able to use research to test new hypotheses formed during the analysis.

• You can move back and forth between thinking about the data and ways to collect new and better data.

• You can energize your fieldwork and help avoid pressure and boredom that can come from unrelenting interview schedules.

The most significant advantage of periodic analysis is that it saves you from dealing with an insurmountable amount of research data. Having familiarized with the data and drafted some ways of thinking about it, means you’ll already be on your journey to insight.

Research health checks

Analysis during collection helps assess the health of your research. Just as you may go to the dentist or mechanic for proactive care, you can incorporate checkpoints to evaluate the quality of your research projects or continuous collection activities. You can do this by jumping into your research data and starting to analyze it in small chunks while still continuing to collect data.

If high-quality data is the ultimate purpose of research, how might you look across your people, processes, research methods, documentation, and project alignment with the organization and your target audience?

For example:

• Is the protocol prepared for interviews with research participants experiencing any issues in producing high-quality data? This might help identify collection errors relating to methods or highlight that your line of questioning could be improved.

• Is the process for tagging transcripts in a digital tool working in the way you hoped it might? Change is inevitable and a part of progress, so keeping an eye on the way it impacts your work will help you make better decisions.

• Is the research team communicating and debriefing well together? Are there things missing that might cause misalignments in how to collect or interpret data?

• Are affinity mapping activities or data displays resulting in useful data? This helps highlight assumptions and challenges hypotheses to help adapt research and find the most valuable information.

Capturing post-session impressions

Both the researcher's interpretations and the evidence in the data guides the analysis process.

Keeping notes during sessions helps tame some of your mind’s acuity and maintain focus while moderating. These moments with participants and interviewees can provide a valuable set of impressions about the individuals, understanding of context, follow up questions, needs for better research focus, and other ideas and feelings that build empathy.

These rich impressions fade quickly and can be difficult to re-conjure if not adequately articulated. This is where allowing enough time between sessions is critical. Interviewing four or five participants in a single day can blur our memory. Some solo researchers allow 15 minutes to fill out and tidy up their scribbled field notes while refocusing for the next session. Collaborative teams of researchers might allow 45 minutes for similar reasons, but spend this time collectively discussing and documenting their observations.

They become an initial framework for later analysis to help fill in with other evidence from transcriptions and other artifacts. Importantly, notes often reflect interpretations of what you’ve seen, so keep your journey accountable as your understanding increases over time.

Tagging data as we go

It’s common to begin tagging data when you gather notes and conduct post-session reviews. Tags or shorthand labels that can be jotted in the margins or as part of session comments.

Some observers or note-takers use a pre-prepared taxonomy to record specific or expected responses during sessions. More commonly, tagging is done afterward by highlighting chunks in a transcription record of the session or slicing and labeling various forms of qualitative data.

The codes used in tagging might be known upfront, but you’ll likely find that you need to add new tags as you go, or that the tags you already have are skewed with assumptions. The power of tagging as you go lies in indexing the raw data.

There is no need to wait until data collection is complete before checking how it is shaping up. You see the beginnings of possible patterns emerging, and note the effect of particular variables on the themes you’re exploring. If you’re concerned about making judgments without a full picture, assess continually and confirm your findings at the end.

The benefits of going back to reconsider previous decisions and adjust or iterate on analysis as you gain a better understanding of a topic are well-known. But qualitative research, where people are at the focus of the study, is well suited to a reciprocal process of data collection and data analysis.

This kind of analysis is such a mentally intensive process that you probably welcome opportunities to spread the load over time. It will give you thinking space for ideas to develop and mature.