Research repository: solving your organization’s research problems

At Dovetail, we constantly talk with researchers in organizations like Atlassian, Square, VMware, Shopify, Deliveroo, Thoughtworks, and more. We repeatedly hear the same problems regarding how user research data and insights are created and shared:

1. User research is siloed within departments

2. The same research is repeated over and over

3. Research reports go unread by stakeholders

4. Cumulatively scaling datasets don't lead to cumulatively scaling knowledge

Throughout this blog post, we’ll dive into these problems and discuss how a research repository can help and the principles that make for a great research repository solution.

User research is siloed within departments

Along with the user research team, almost all teams within an organization conduct some form of customer research, even if they don’t explicitly call it that. Product managers interview users, designers test prototypes, salespeople talk to leads, support tracks topics, marketing facilitates focus groups, HR run feedback surveys, and customer success run Net Promoter Score surveys.

These functions and teams all do user research differently, analyzing their data with different tools, and publish mountains of data, reports, and updates across a variety of formats and products. Stakeholders and consumers struggle to find what’s relevant to them—the signal in the noise; the needle in the haystack.

The same research is repeated over and over

Often, when a new researcher joins a product team, they go off, do some research, and come back with the same insights and findings that are already known. It takes months (years, even!) of experience to build the necessary context and to get to tangibly meaningful insights that have the power to effect change and be force multipliers in an organization.

Tenured employees already know this and grow skeptical of a research team that tells them things they already know. Over time, thanks to the same research being repeated, and the same conclusions echoed, the effectiveness and credibility of the research discipline within your organization is slowly eroded.

Sherif Mansour is a distinguished product manager at Atlassian with over 10 years of experience working on products like Jira and Confluence. He reflects that:

Because research is stored in a variety of formats, tools, and silos throughout the organization, there’s no way for anyone to quickly get up to speed on what is already known. Searching through years of unorganized archives of reports, documents, and presentations to build context is an impossible task.

Lastly, when that researcher leaves the company, they take a lot of undocumented knowledge of your customers with them—hopefully not to a competitor.

Reports go unread by stakeholders

Written reports and presentations are the ‘go-to’ format for sharing user research insights, but unfortunately, there’s no standard for how they’re produced or what content they should contain, and every researcher structures them differently.

These kinds of reports are usually inaccessible to everyone in the organization: they’re shared spontaneously over email and chat or lost in Google Drive or Dropbox. There isn’t a clear place for people to go to learn about your customers. You’re lucky if these reports are read, and you’re even luckier if any action is actually taken as a consequence of them.

Back in 2005, Jakob Nielsen from Nielsen Norman Group highlighted the pain of finding usability testing reports in his article Archiving Usability Reports:

Another detriment of traditional research reports is that they’re usually disconnected from the raw data and evidence that led to the formulation of the report in the first place. It’s difficult for stakeholders to understand how a researcher came to a particular insight, because there’s often no way to dive into the details of a specific usability test, interview, or survey response. This lack of traceability leaves stakeholders spending time poking holes in the research or methodology, instead of taking action on them.

Standard research reports also often lack any features that allow you to track engagement. PDFs don’t have read-receipts, and there are no easy ways to measure the impact research is having in your organization.

Worst of all, of the researchers we spoke to, none of them actually enjoy creating these reports. They find themselves spending more time designing and formatting internal reports for stakeholders, rather than actually talking to customers and users. The best workaround we’ve seen is reusable templates for research reports.

Cumulatively scaling datasets don't lead to cumulatively scaling knowledge

Research data and customer feedback comes in many shapes and sizes, including surveys, quotes, interview notes, video recordings, industry trend reports, social media posts, app reviews, and more. Researchers rely on all this information to synthesize research findings, usually in the form of a research report. Typically, this research lives in a file storage system like Google Drive or Dropbox or a wiki like Confluence or Notion.

Organizing and curating this ever-growing collection of data is relatively simple for a small organization. However as companies grow, the systems used to store customer research don’t scale and end up requiring significant administration to maintain. Someone has to ensure all the files are searchable, appropriately restricted, and tidy.

Inevitably, the repository becomes unwieldy and difficult to use. This creates the opposite of the intended effect—people stop using stored research and the total sum of an organization’s customer research leads to less customer knowledge.

Why a research repository?

Most organizations with a user research function acutely feel the pain of the problems above. If you don’t yet, it’s likely you will in the future as you scale design and research. Much of the pain can be eased through a thoughtful approach for how research data and insights are captured, analyzed, standardized, archived, and shared.

Imagine a ‘Wikipedia’ for your organization’s customer knowledge: a centralized, searchable database of research and insights that the entire company leverages to make better decisions. A single source of truth that empowers stakeholders to engage with research and fosters a culture of research across your organization. Designers, product managers, and developers effortlessly pull up insights that answer common questions and, in one click, can explore the evidence—and understand the context—that led to those insights.

A research repository is just that—one central place, or source of truth, where people in your organization can go to find the latest insights from your research team.

The characteristics of a successful research repository

We’ve talked to user researchers, designers, and product managers across a variety of organizations, and we’ve settled on five key characteristics that lead to the successful adoption of a research repository. We believe all successful repositories are retrievable, approachable, traceable, accessible, and secure.

Retrievable

Searching and finding research insights should be effortless. Members of the product team should be able to easily find answers to their questions, self-serve, without needing to write complex queries or spreadsheet formulas. Insights should be easily filtered and segmented by attributes important to your organization, like segment, country, market, demographic, or even NPS. Researchers need to be able to conduct secondary research by easily retrieving prior research and insights and feed that into current research projects.

Approachable

A research repository should be approachable not only for user researchers, but also for other members of the product team like designers, product managers, developers, and stakeholders like founders or executives.

Creating insights needs to be simple and delightful—your team needs to be bought in and want to use your repository, and they won’t if the experience is complicated, clunky, or slow. Stakeholders want to consume research insights in a lightweight, quick, and beautiful way, without needing to read multi-page reports or presentations. Anyone should be able to easily “remix” previous research and leverage an organization’s cumulative customer knowledge to make informed product decisions.

Traceable

Successful research repositories aren’t just a collection of disconnected insights or recommendations. They also need to provide a path back to the raw research data which led to the insights in the first place. Research findings should be connected to the source material that the researcher interpreted to inform a particular insight. References to raw research data or ‘evidence’ enable stakeholders to see where an insight came from and understand the original context of the research method.

Someone should be able to look back on a user research insight from years ago, review the evidence, and determine if it’s still relevant or has been interpreted correctly based on what is now known. Researchers should be able to look back to see if new insights could be developed from research data that was collected previously.

Accessible

A repository needs to empower your organization to leverage customer research data and insights, and for this to happen your entire team needs access. New people who join your organization should be given access on day one.

You shouldn’t have to worry about managing access permissions for folders in Drive or Dropbox – the ideal research repository should be ‘open by default’. Team members should deep-link to your living repository over email and chat as the source of truth, rather than send around presentations or reports as static attachments.

Secure

As a research repository will act as the single source of truth for your organization’s customer knowledge, it’s likely it will include personal information and potentially sensitive data from a range of industries.

For your research repository to be approved by your security, legal, and information technology teams, you’ll need to consider where customer data will be stored, what the processes are for retention and deletion, if data is encrypted or anonymized, and what security practices and policies are in place.

If your organization has customers in the European Union, you’ll need to ensure that your research repository is compliant with the General Data Protection Regulation (GDPR).

Starting a research repository

So you’re convinced and want to start building a research repository in your organization. How do you get started? Aside from working out the governance and operations that will support how people use and interact with your repository internally, the most critical decision is selecting a tool or platform for your research repository.

Flexible and generic tools

Most organizations start their research repository journey by assembling existing tools that are already used internally, typically because they’re accessible and inexpensive to set up. Teams cobble together faux research repositories with wikis like Confluence or Sharepoint, spreadsheets and databases like Excel or Airtable, issue-tracking tools like Jira and Trello, and storage tools like Google Drive or Dropbox.

As an example of using a flexible tool as a research repository solution, the folks at GitLab have used their own product as a repository for their UX research insights. Sarah Jones, UX Research Manager at GitLab describes their repository:

As everyone at GitLab has access and knows how to use the product, this approach has been successful for them. However, most research organizations don’t have the luxury of working on a product that can be manipulated in this way, and flexible solutions almost always inevitably fail in the long term for a variety of reasons:

• Searching and retrieving insights from these tools is cumbersome – search functionality is complex, limited, and unintuitive. In the case of file storage tools like Dropbox and Google Drive, searching within files is usually not possible.

• Insights are static and disconnected from the original research data. Recordings, transcripts, notes, and participant information has to be stored across many different tools and the evidence that supports an isolated insight is lost.

• They’re not fit for purpose. Flexible, horizontal tools aren’t designed for research and use unopinionated terminology like cells, sheets, rows, and records instead of concepts that are research-friendly like insights.

We often see people come to Dovetail as a purpose-built research repository solution after trying to build a repository using tools that aren’t built for that specific purpose.

Building a custom repository in-house

Unsatisfied with flexible, generic tools that aren’t fit for purpose, organizations with well-resourced research teams often look to design and build their own research repository in-house. Large enterprises like Microsoft, WeWork, and Nasdaq have all shared details of their own purpose-built research repository solution.

Microsoft’s research team have built their own repository called the Human Insights System (HITS). As Matt Duignan, Principal User Research Manager at Microsoft explains, to help scale HITS across the entire Microsoft organization without losing the integrity of their data, they designed for letting people find insights while retraining context:

The design of HITS enables research to be brought into conversations and make product decisions informed by customer data and evidence.

Similarly, WeWork built their own research repository solution internally, and Tomer Sharon, then the Head of User Experience at WeWork, announced the Polaris research repository in his article Democratising UX.

Following the article, an ‘open-source’ version of Polaris was released as a template on Airtable. Unfortunately, the researchers we’ve spoken to who have tried to create a research repository with this template have not had success, for the same reasons why tools like Confluence, Airtable, and Dropbox don’t make good research repositories.

With enough resources, building your own research repository can work since it can be tailored to specific needs and processes. However, given the complexity in building a bespoke research repository solution and the ongoing maintenance commitment, even large organizations rarely opt to invest in an internal solution.

Why we built Dovetail

We built Dovetail to be the platform for your organization’s user research, considering principles of retrievability, approachability, traceability, accessibility, and security from the ground up. With our platform, researchers, analysts, designers, and product managers can store user research data in one place, analyze qualitative data to discover patterns and insights, and share research findings with the rest of their organization.

Our team has decades of combined experience working in product teams at Atlassian, Canva, Intercom, and Microsoft. We’ve all seen first-hand the challenges that come with managing customer knowledge and insights, and we’re passionate about helping teams understand their customers, make better decisions, and do great work.

Search, segment, and retrieve data

We’ve built an appreciation of how important it is to enable stakeholders to self-serve customer data and research insights. Researchers are usually stretched thin relative to their counterparts in engineering, product management, and design, and insights need to be scaled across the entire organization.

We’ve built powerful, intuitive filter and segmentation controls that enable people to easily retrieve research data and customer insights. Research data can also be enriched with contextual information: demographic data like name, age, location; contact details like email address and phone number; properties like persona, company, or license; and even someone’s Net Promoter Score.

Approachable and enjoyable

A focus for us is making sure Dovetail is approachable and enjoyable to use. We know a lot of research software isn’t particularly intuitive, especially academic qualitative data analysis suites like MAXQDA, NVivo, and Dedoose. While very powerful, these QDA tools are complex and beyond the reach of most commercial researchers and stakeholders.

We try to make it easy for people to analyze unstructured data, create insights and embed references to raw data, and making insight reports look good by default.

Embed evidence to your raw data

In his article on Microsoft’s Human Insights Library, Matt Duignan talks about “real-time curation” as a way to encourage participation in the research library / repository, to avoid the problem of reports being deferred to the end of the process:

Rather than running out of time to turn insights into durable, standardized reports, an insights repository should instead organically evolve over time, with traceability back to the original source material and context.

Because Dovetail has features to help with analyzing common research methods like interviews, usability testing, and surveys, it allows you to maintain traceability through embedded references to notes, quotes, tags, photos, videos, recordings, and more.

Access for your entire organization

We don’t want our pricing to inhibit the effectiveness of research teams within organizations. To make Dovetail accessible to everyone, we’ve been intentional about offering unlimited, free viewer accounts for people in the organization to view research data, read insights, and comment. This means that our customers only pay for researchers that actively contribute data, analysis, and insights.

This pricing model has accelerated the consumption of research in our customer’s organizations. Gemma Sherwood, Intrepid Group’s UX Manager has shared with us that their UX team has now turned their attention to other departments like Marketing and Insights to get them involved in their research repository:

We’ve also invested in features that allow anyone with an email address at your organization to join your Dovetail workspace without needing to be invited and the ability to share your projects and insights with others who don’t yet have an account.

Secure, and private by design

We make security and data privacy foundational principles of everything we do, and we recognize the importance of adhering to regulations to advance information security and data privacy, including the GDPR.

We’ve chosen proven third party cloud providers, take regular data backups and test recovery, run penetration testing, encrypt all data at rest and in transit, conduct static code analysis and third-party vulnerability scanning, sanitize our logs, secure individual customers at the database level, and many other cloud security techniques.

The future of research repositories

We think more and more organizations that talk to customers and leverage customer data to improve their products and services are increasingly frustrated with the pains of storing, retrieving, and sharing insights from user research. We believe that a thoughtfully considered user research repository is critical to mitigating this pain and unlocking the potential of customer research in organizations worldwide.

In Dovetail, we think we’ve built a solid foundation on our principles of retrievability, approachability, traceability, accessibility, and security. Our team is motivated to make Dovetail the obvious platform for your user research and insights.

We’re at the beginning of this journey. We’ve got more to learn from our customers, and many exciting features planned for the future. If you’re interested trying a research repository in your organization, we’d encourage you to give Dovetail a go and sign up for a free trial—we’d love to hear your feedback!