Participant recruitment 101 with User Interviews’ VP of Product

There are no two ways to put it. Recruiting the right participants for your user research groups is hard. Whether it’s wading through screeners of unsuitable participants, dealing with monosyllabic non-responses, dodging the inevitable professional research participant, picking the right incentives, or simply getting people to turn up at all; finding the right users for your study is rife with problems that can derail otherwise fruitful customer research efforts.

Since we love research at Dovetail, we’ve invested quite a bit of time trying to work out how to make this process simple. In the past, we’ve written about how to find the right number of research participants, the dangers (and benefits) of leveraging your existing fanbase, different ways to find suitable candidates, good screening measures, and general guidelines to help you find the right person for the job. In the Dovetail product, we recently introduced “People,” a new feature designed to help researchers track and manage research participants.

You could say we think about it a lot. But, to be fair, probably not as much as our friends over at User Interviews, who have made it their core mission to help teams discover user insight with fast, easy access to high-quality participants for any kind of research.

So we moseyed on over to User Interviews (remotely moseyed, via video call) to speak with their VP of Product, John-Henry Forster, to get his hot-tips and takes on some of the thornier aspects of participant recruitment. Namely, the importance of diversity, how to vet your participants, and how much to spend on participant recruitment.

The importance of diversity in participant recruitment

Whether a product team is looking to expand on a particular problem area or conduct exploratory research on a new product or idea, finding the right person to interview is crucial to the outcome. But what does good look like?

According to Forster, an ideal research panel is “a diverse cross-section of users who have the problem that you’re doing discovery on or solving.”

And while countless pots of digital ink have been spilled dissecting the various considerations of finding the right participant, it’s Forster’s initial point we focus on. Diversity is important. And as we’ve written before, it’s also good for business.

“You don’t want to be talking to the five people with identical backgrounds that have your problem. You want to be talking to people that make up different vectors of backgrounds, so you’re not getting just a single perspective on what you’re doing,” explains Forster.

“In some of the smaller user research studies, it’s hard to get rich diversity across every vector when you’re only gonna talk to five or six people,” he concedes. “But I think you want to, at least at a minimum, avoid accidentally opting into a mono-perspective because you were so focused on the problem piece, you lost sight of the other makeup of that person.”

Often the value of diversity in participant recruitment is dismissed as focusing on edge cases, says Forster, but it’s important to remain aware of these experiences, even if you choose not to solve for them right then and there. “What you decide to do with that knowledge is always in your control, but it’s better to know it than not,” he asserts.

How to vet your participants

Even in mature organizations, the vetting process for finding the right research candidate can be a struggle. “You have to figure out if people are who they say they are, are articulate, and whether they’re worth the time to talk to,” explains Forster. “When you come to User Interviews or any other solution, we’ve done all that legwork up front, and in doing so, we’ve also collected a ton of data about the people. So we’re able to do the matchmaking in a faster way as well.”

If you’re setting out to conduct user research without the aid of a tool like User Interviews, Forster has some wisdom to share on screening for the right candidate. “The easiest one is throwing in an open-ended prompt in your screener survey where you ask somebody to reflect on a recent experience,” he says.

An open-ended question can help identify users who not only understand the problem you’re investigating but who can articulate it in a meaningful way. “Say you’re asking a question about pizza. A person who writes in a one-word answer versus somebody who says, ‘My kid was home sick from school, and we weren’t sure what to do, we didn’t have any food. So we decided, you know what, we’ll order pizza.’ That second person is going to be much richer when you go to probe them,” says Forster. “The person who can tell you a whole story is far more valuable than the person who just uses one word.”

How much should we be spending on recruiting participants?

At Dovetail, we often hear that researchers struggle to get buy-in from senior management. There are various reasons for this, and we’ve covered how to pitch research in a way that resonates with executive decision-makers in other articles. Long story short, a lot of the time, it comes down to cost and perceived ROI. With that in mind, we ask Forster about the importance of investing in research.

“It’s important to have a research budget in place,” approaches Forster. “It’s putting a stake in the ground as a business that there’s going to be resources set aside for research.”

On getting buy-in, the User Interviews’ product leader leans on well-known thought-leader John Cutler for the answer. “Cutler frames this problem as a hypothetical. If you thought about a problem in your business, and it’s vital to get the answer right, if you could magically buy that data from a consulting firm, how much would you spend to get that answer?” In Cutler’s example, over the many years he’s posed this question during workshops and seminars, participants throw out huge numbers. “People say 50 grand, 60 grand—big numbers,” says Forster. “Because when you do the back of the envelope math, answers to the big questions drive conversion impact where it’s needed most.”

“For some reason, people get hung up on spending 250 bucks on incentives every other week or so. But And when you add it up, it might cost $3000 for the answer, but if you apply Cutler’s framework, $3000 is not a lot of money for a business of any size to get an answer to the big questions,” says Forster. “For whatever reason, people don’t zoom out, and research feels like an expense rather than something valuable.”

This isn’t just in terms of participant recruitment. Overall, teams are still underinvesting in headcount and budget for research in general, says Forster. “Especially if you looked at the books and saw what they’re spending money on paying for. You know, there might be some random design tool that’s costing hundreds a month, and they barely use it,” he posits.

“If you think about where that money could be deployed instead, it’s kind of striking in that regard.”

What do you think? Are businesses spending enough on user research? Join other Dovetailers and continue this conversation over at our Slack Channel!

Written by John-Henry Forster, VP of Product, User Interviews. John-Henry (aka JH) Forster is a product leader with over a decade of experience helping product teams ship solutions for real user needs. He is currently the VP of Product at User Interviews, where he oversees product management and design—and geeks out about user research as the co-host of the Awkward Silences podcast. Before User Interviews, JH helped create and scale the recommender systems at Catalant Technologies and led the global pricing overhaul project at Vistaprint. When he is not thinking about the intersection of user research and product development, he is likely either going for a run, picking up a new hobby, or spending time with his family.