Subscribe
The four deadly sins of continuous discovery
Escaping the mouse race.
Published
19 May 2024
Creative
The things I’ve seen most teams do wrong when running continuous discovery—and why it can actually kill your product or business.
You’ve heard that 90 percent of startups fail, right? After eight years working with startups, I’d bet those same 90 percent failed at customer discovery first. 
But don’t assume that all those failing startups aren’t doing customer discovery. Many of them are. The saddest thing I see working with startups and scale-ups is that so many actually do research—and still fail. 
Over the past few years, I’ve seen a trend make this problem even more common: teams doing discovery work continuously often think that the regularity is enough to do it well. 
The truth is, continuous discovery often creates new problems you didn’t expect.
You have to do continuous discovery right to make sure it’s worth the time spent every week. 
Don’t believe me?
A few years ago, I was hired by a European startup developing their second product. The founders had been pretty lucky with their first product: they’d identified a personal problem, loosely validated it within their group of friends, and launched a solution. It gained traction fast, without intentional discovery work. 
But with a second product in the works, they had a more established business with employees and investors now—and a lot more risk. So the team read all the books and articles on customer discovery, and started applying the lessons.
That team ran regular, continuous discovery work for months to prepare for the new product. They followed the guidelines from others’ processes and talked to a specific number of customers every week. 
But at the six month mark, almost none of their discovery had helped them make meaningful progress. They weren’t any closer to finding the right audience and finding a problem worth solving that they could monetize. 
They launched a new product as a beta that didn’t deliver any of the results investors needed to see.
Why does this happen?
There can be many reasons behind a failure to run successful continuous discovery. But I’ve seen most teams consistently make a few critical mistakes when they’re new (and not so new) to running continuous discovery. 
When a team makes these “deadly” mistakes, they’re likely to fail at getting impactful insights, efficiently. They waste time, money, or effort on continuous discovery habits that lead to a failed product, or even a failed business.
The four deadly sins of continuous discovery
1. You’re not specifically defining what each round of discovery needs to achieve
This year, I worked with a product team where every team member had run at least 40 user sessions. That’s a lot! They were running session after session with users every week. But after so many sessions, they still struggled to design the right new product for their users. 
When I joined them, no one in the team could tell me what “success” should look like for the discovery they were running that week. As a result, none of their sessions focused on the right things for where they were as a business. Unsurprisingly, they weren't achieving their goals.
Like them, you might think you can just talk to customers every week, and magic will happen. 
In reality, you need to define your desired outcomes for any and all discovery, continuous or otherwise. If you aren’t clear about the decision or action that your discovery needs to help you make, you’re unlikely to get value from it.
In addition, each week of discovery should build on, add depth to, or explore a different track than the previous week. But most teams I’ve worked with use roughly the same script in round after round of discovery, losing the opportunity to gain the more focused insights they really need. 
In my experience, this is where the true value of running continuous discovery happens.
Knowing what you need to do with the results of discovery before you start will set you up to get the right information every time.
Get this right, and you’ll find yourself in the top one percent of teams running continuous discovery. 
Ask yourself: What decision or action were we trying to take this month? What two to three things did we need to understand? Did the last few weeks of discovery gather the information to make that decision clear and easy? 
2. Your standards are too low when selecting insights to work with. Be strict and ignore weak signals
I’m always surprised at how many teams accept lukewarm feedback from users as “good enough” for building expensive products.
In continuous discovery, you’re likely getting more feedback, more frequently, but you need to evaluate the input, too. 
Eventually, you find yourself with a huge pile that someone (probably you, sorry to say) has to filter and prioritize. Running discovery continuously means you need to be more selective than ever. 
When building new products, new methods of monetizing your product, or new features, signals should be very strong to pass the test.
I’m a big Warren Buffett fan. I take a cue from Buffett when doing research.  Prioritizing which insights to run with is a lot like choosing which investments to make. 
Buffett doesn’t buy businesses or shares unless the value is “obvious.”As he famously said, “I don't look to jump over seven-foot bars: I look around for one-foot bars that I can step over.”
Obviously good deals make decisions much easier, and investments much safer.
While it sounds as if “obvious” business deals would be obvious to others, Buffett has made billions digging for the insights that others miss. When he finds very strong signals, he acts.
I’ve rarely met founders and product managers who think this way about customer insights. But the truth is, great products and businesses are most often built on an insight with an obvious, very strong signal. 
The signals you prioritize should exist far enough to one end of the emotional spectrum to be an easy bet. You shouldn’t invest in solving a problem that is not excruciating for your customer, or a deeply rooted desire they’ll be ecstatic to fulfill.
When our standards aren’t high enough, we carry too many insights into consideration for what to build. It’s too easy to get lost in the weeds when collecting insights every single week.
3. You’re too focused on someone else’s routine, and not enough on whether it’s working for you
Many teams think continuous discovery is a one-size-fits-all process. It’s easier to believe you can follow someone else’s framework for perfect, repeatable results. Plug and play. No thinking required!
I wish that were true, but I've never seen that work.
Your team needs to account for your own environment and requirements to find a continuous discovery process that works for you.
I tell product teams to iterate their discovery processes the way they iterate their product. You need the same willingness to measure, evaluate, discuss and change whatever didn’t work.
The missing part of continuous discovery for most teams is: continuously challenge the discovery process.
Less important: fixating on the number of participants you recruit every week. 
More important: asking yourself whether you learned what you needed to learn from them, or if you need to change something next time.
Focus less on what someone else prescribes, and more on whether it’s working for you. 
4. You’re running a lot of user sessions, without tracking the path through decisions and outcomes
I’m often asked how to make sure the results of continuous discovery actually make an impact on the product.
The answer to this question has multiple parts, but the biggest piece is accountability.
If you’re not keeping a record of the path from insights to decisions, product changes and outcomes, then your team lacks accountability. 
If you don't hold your team accountable for ignoring insights and making opinion-based decisions, your product is unlikely to improve.
Of course, this is a bit easier said than done. I’ll be honest: not all the clients I’ve worked with have listened to me. 
I once worked with a major European bank building an internal fintech startup. They’d wasted a fortune developing a product that no one wanted. They brought me in to teach them how to run better continuous discovery and develop a product customers actually needed.
The results of discovery showed, however, that their target group didn’t have problems that the bank team wanted to solve (and monetize). So I challenged their definition of their target audience.
The bank’s team heard the insights, then promptly ignored them. They continued running research most weeks, but also building whatever they wanted to build. 
They continued not converting any landing page visitors to users.
The problem wasn’t just that they ignored the insights. They also didn’t document how they made product decisions with (or without) insights. 
They didn’t hold themselves accountable for the bad outcomes of those decisions. Sad to say, but this team is unlikely to learn and improve their product before their funding runs out.
Teams that track the following sequence of events learn faster about what works and doesn’t, and build better products:
Get an insight > Feed it into a project > Make and document the product decision > Measure what happens > Evaluate the outcome of the product change based on the insight and decision.
In theory, continuous discovery creates an opportunity for this learning cycle to happen faster—as long as accountability is involved.
If any of the four deadly sins of continuous discovery feel painfully familiar, I promise there's hope. 
I’ve seen most teams improve their continuous discovery outcomes considerably by taking less than an hour and targeting these fixes first:
  1. Figure out specifically what you need to learn in your next discovery. 
    1. What is the one decision you need to make? 
    2. What are two pieces of information you need from customers to make that decision
  2. Level up your standards and be more strict:
    1. Look for really strong emotional signals in your feedback and data (“I love that” or “I get so pissed every time I have to do [activity].”). 
    2. If there isn’t a strong emotional signal around your topic or feature, don’t prioritize it. You need to hunt for something stronger. 
You may even need to talk to another audience or sub-audience to find those stronger signals if they aren’t coming up with your existing target group.
On its own, a process of continuously talking to users won’t solve all your challenges with understanding customers. But taking the steps above will help ensure you get more value from your discovery continuously, and build better products continuously, too.

Subscribe to Outlier

Juicy, inspiring content for product-obsessed people. Brought to you by Dovetail.