How to run a user research diary study with participants

A diary study is a qualitative research method used to learn about people’s behaviors, thoughts, and feelings. The researcher starts with a general topic they would like to learn about, for example, ‘young people’s eating habits in France.’ They will recruit participants from various backgrounds (in this case, young people in France). Diary studies usually have at least ten participants, but can be conducted at larger scales with over 100. Participants are typically incentivized with money or vouchers to take part in the study.

The study itself is sort of like a sandwich. The beginning has an introductory brief that explains the research topic and provides instructions to the participants. While the study is running (the ‘middle’ of the sandwich), participants receive questions or ‘prompts’ to help them stay on topic. At the end of the diary study, the researcher asks closing questions before analyzing the responses. Diary studies generally run from a few days to a couple of months.

What are the differences between a diary study and a survey?

Surveys collect self-reported information about a person’s habits and experiences outside of the context of the scenarios being studied. Diary studies are different in that they are more similar to ethnographic research — they’re very contextual and aim to get participants’ responses as they’re ‘living their life.’

Surveys can often be laborious in length, and the questions can be designed to favor a particular outcome. The danger of surveys is that they are incredibly simple to create, but very difficult to get right. Erika Hall writes about this in her fantastic article called On Surveys. On top of this, surveys are filled out in retrospect, which can lead to recall bias.

Lastly, diary studies are instrumental when you’re interested in measuring changes over time. A survey only offers a snapshot, whereas a researcher can ask the same questions multiple times over the course of a few weeks and observe changes when the participants’ environment is modified. For example, a diary study about a mentoring program could follow a ‘mentee’ before and after they have received mentorship from someone and see how their life changes due to the mentor’s influence.

When should you use a diary study?

Diary studies are useful for understanding long-term behaviors or changes in behavior, attitude, or feelings over time.

If you’re looking for a contextual understanding of user behaviors and experiences over time, it can be tough to appropriately create scenarios in a lab setting to gather these insights. Diary studies are useful for more in-depth research, like:

• Getting rich customer feedback on products or services. What are the users’ primary tasks? What pain points are they experiencing during key flows? When completing a specific task, what workflow do they use?

• Finding new opportunities in the market. What solution are people using currently? Do they have problems we can solve in a better way? Is there an opportunity for a product or service?

• Mapping customer journeys. What is the typical customer journey and user experience as customers interact with your organization? What do they do when they onboard? What makes them stick around? Tell their friends?

• Studying different cultures. What will we need to change about our brand / product / service to enter this market? How do people around the world perceive our brand? What sort of marketing resonates?

• Understanding deeper emotions. What motivates people to perform specific tasks? How are users feeling and thinking? Where are the opportunities for delighting them? What makes them frustrated or angry? Why do they contact support?

• Building persona libraries. What are some common, holistic behaviors across groups of our users?

How do you run a diary study?

Diary studies are typically composed of five stages: designing the study, recruiting participants, collecting data, analyzing the responses, and sharing what you learned.

Design the study

In this stage, you define the focus of the study and what you want to learn. You may have a hypothesis here. Define a timeline, your ideal participants, and prepare instructions or support materials.

Recruit participants

Since diary studies require more involvement compared to other research methods, be extra prudent in the recruiting process. Let participants know what is involved and expected of them upfront. Ask screening questions that will help you gauge the level of commitment you will get from them during the study, and confirm their availability for the entire study period.

Before you start the study itself, take some time to brief participants. Ideally, schedule a face-to-face meeting or phone call with each participant and discuss the details of the study. Walkthrough the schedule for the reporting period, and explain what your expectations are for their answers.

Collect data

There are many different tools to collect your diary responses. Some researchers use a combination of survey software and IFTTT or email scheduling tools to send participant prompts. You can also get creative and try creating Tumblr blogs for each participant then have them write their entries there.

Analyze responses

As soon as data starts to arrive from participants, start analyzing! You’re essentially looking to identify high-level patterns across the full data set. See if you can find insights across multiple people, and compare those to demographic data. For example, you might find that people who live in the countryside are more likely to eat at home than those who live in the city.

Share what you learned

Naturally, you will have people you want to share your insights with – teammates, other researchers, your manager, or friends. Often researchers will create a lengthy report or presentation to walk everyone through the design, method, and outcome of the study.

In summary

While diary studies do require more time and effort than other research methods, they yield invaluable information about real-life behaviors and experiences.

If you’re looking for organic insights and you can’t create a valid scenario in the lab, or you can’t get the data you need from a single survey, don’t force-fit the research into these methodologies. Try a diary study instead.