Field guide for the time-poor: How to conduct lean and unbiased research

As humans, we are hardwired to see patterns that we want to see. We look up in the clouds and recognize faces and animals. We always look at the clock at a particular time and start to attribute meaning to that number. We love to make meaning, even where no meaning exists. Keeping this in mind, how do we ensure that we aren't chasing specific answers when conducting our research sessions?

You can attend training courses to help negate biases and learn about unbiased experimental research design. However, this can be incredibly time-consuming and might not be doable in a pinch, especially if you are not a trained user researcher. When you lack time, expertise, and resources, it can be challenging to create an unbiased study.

Challenging, but not impossible.

Minimum viable research

A lot of us are aware of the MVP: the minimum viable product. We build this version to lean out the development process and include only the essential features to test out the idea. We can apply this same idea to research while keeping our studies sound and unbiased. This concept is beneficial if you are not a trained user researcher and are conducting research yourself. It leaves you with a simple research design that helps you avoid the pitfall of confirmation bias.

Creating an unbiased MVR plan

A plan is the best place to start to ensure your following research study is lean and unbiased. This MVR plan is a stripped-down version of my standard research plan, so check out the template if you want to include more detail. There are several steps to take and components to create a minimum viable research plan. Following this guide will help you build a research plan that is both lean and neutral. 

List out hypotheses/assumptions

We hold a lot of biases in our heads, whether we like it or not. Since we are human, there is no way we can make our prejudices disappear to conduct one hundred percent pure research. We can, however, take steps to make ourselves more aware of those assumptions we hold so that they are less likely to lead us. 

Before you get started on research problems and questions, sit down with whomever is involved in the research project and list all of your hypotheses, assumptions, and biases about the project, idea, and participants. With this exercise, you take the thoughts in your head and put them down on paper, making them more widely known and forcing you to be aware of them. Keep this list with you when writing your research plan and during analysis, as it can help keep the group in check.  

Reframe a business question to a user research problem

As a user researcher, colleagues come to me to request user research with a plethora of questions or problems. Unfortunately, often these requests revolve around the business rather than user needs or problems. Starting a research project based on a business question or concern could lead to some issues. 

A typical example of this happens when a colleague comes to me with a problem like, "people are purchasing once from our platform and then never coming back, or deleting the app. Can we find how to make people purchase more?"

Yes, this is a big problem! Retention, customer lifetime value, app download rate, revenue are all crucial metrics impacted by this issue. However, does the user care about these metrics? No. But how do you pivot these business problems into user research problems? Ask yourself: "How does this problem impact the user?"

In the example above, you could hypothesize that the user is:

• Not satisfied with the initial purchase

• Doesn't need to purchase more than once 

• The app is not solving a problem for them

• The purchase flow was too difficult

In this case, you could turn the business issue into the following research study: understand the process people go through when purchasing from our app and find any problems or pain points they may be encountering.

Now, instead of simply asking participants how they would purchase more, we can dive into the reasons why they are using the app in the first place and why they are not coming back. This approach will give you much more robust and informative qualitative data.

Create research goals

Research goals are a step further than the research question or problem you are exploring. These goals should address HOW we are going to study the problem statement. We do this by breaking the research problem down into several objectives. Here is the two-step process you can use:

1. Ask yourself: "What am I trying to learn?" and "what must the research achieve?"

2. Identify the stage of the product development phase, such as a general idea or topic (generative research) or a prototype or concept (evaluative research)

What I am trying to learn:

I want to understand the process people go through when purchasing from our app and find any problems or pain points they may be encountering.

This statement is broad, and we need to break it down to what we want to learn about that problem statement:

• How do people currently decide to purchase with us?

• Why do they choose to buy from us; who else is involved; when do they buy?

• What tools do they use to make a purchase?

• How do they feel about the current purchase process?

• What problems do they encounter when trying to make a purchase?

• What improvements would they make to the current purchasing process?

What are you researching?

Knowing what exactly we are researching will also help you hone your research objectives. There are three main buckets:

• A topic or idea: we need to understand the process people are currently going through (generative research)

• A concept or prototype: we need to uncover what people think about the prototype and how they expect to use it (generative + usability testing)

• Live code: we need to evaluate the performance of the product and what people think about it (usability testing)

These will change your objectives slightly to know whether you are focused on understanding an overall topic/idea, uncovering what people expect from a product, or evaluating product performance. A usability test is far different from generative research, so it is essential to know what type of research you are doing, as it helps us understand what we can expect to learn.

Once you have brainstormed the above, you can start writing goals! Each project generally has three goals that narrow in on exactly what you want to be researching. Follow these models for the most common types of objectives:

1. Discover people's current processes/decision-making about [research subject], and how they feel about the overall experience

2. Learn about people's current pain points, frustrations, and barriers about [existing process/current tools] and how they would improve it

3. Uncover the current tools people use to [achieve goal] and their experience with those tools. Uncover how they would improve those tools

4. Understand what [research subject] means to people (how they define it) and why it is important to them

5. Evaluate how people are using a [product/website/app/ service ] OR Evaluate how people are currently interacting with a [product/ website/app/service 

Write neutral interview questions

Now that you have your research goals sorted, writing questions becomes a bit easier. Ideally, you write research questions that align with and help you answer each goal. Your questions should be able to give you insights that answer your objectives. So when you ask a participant a question, it is ultimately answering one of the objectives. You can start by turning each objective into 3–5 questions.

However, before you get started writing any old question, there are ways to keep your phrasing as neutral and unbiased as possible. One of the best ways to write fair questions is to keep them open-ended and not use subjective words. If you are getting stuck with writing questions, you can use the TEDW framework:

• T = "Tell me more..." or "Take me through..."

• E = "Explain why..." or "Explain how..."

• D = "Describe why..." or "Describe how..."

• W = "Walk me through..."

So, instead of: "Were you upset when you couldn't purchase X or Y?"

Instead, you can ask: "Explain how you felt when you couldn't purchase X or Y."

Instead of: "When did you last purchase on our app?"

You can ask: "Tell me about the last time you purchased something on our app."

Instead of: "How happy/satisfied are you when using our app?"

You can ask: "Describe how you feel while using our app."

By using this method, you ask open-ended conversation-starting questions that give you rich, qualitative data and avoid leading or biasing the user to answer in a specific way.

Analyze via research goals rather than outcomes

A rut I see many colleagues get stuck in is during analysis. During that stage, people are looking for answers to what they are hoping to find. I've done it too! 

Rather than analyzing the qualitative data by expected (or hoped for) outcomes, use the goals you defined above. By looking at the research through the lens of the goals, you are less likely to pick out what you want to hear. You can use affinity diagrams to analyze in this way. Also, don't forget to keep that list of assumptions and biases next to you while analyzing the data. 

Following these lean steps can help lead to rich and helpful data, leading your team in a better direction—and without breaking the bank or timeline!

Written by Nikki Anderson, User Research Lead & Instructor. Nikki is a User Research Lead and Instructor with over eight years of experience. She has worked in all different sizes of companies, ranging from a tiny start-up called ALICE to large corporation Zalando, and also as a freelancer. During this time, she has led a diverse range of end-to-end research projects across the world, specializing in generative user research. Nikki also owns her own company, User Research Academy, a community and education platform designed to help people get into the field of user research, or learn more about how user research impacts their current role. User Research Academy hosts online classes, content, as well as personalized mentorship opportunities with Nikki. She is extremely passionate about teaching and supporting others throughout their journey in user research. To spread the word of research and help others transition and grow in the field, she writes as a writer at dscout and Dovetail. Outside of the world of user research, you can find Nikki (happily) surrounded by animals, including her dog and two cats, reading on her Kindle, playing old-school video games like Pokemon and World of Warcraft, and writing fiction novels.

Art by Ben Jomo, Illustrator.