Concept testing 101: what is it & when should you do it?

Concepts, hypotheses, and ideas are a crucial part of product development. But whether you’re improving an existing platform or creating a new one, it’s important to test and validate your concept with your target audience to establish what's most valuable to your users and worth pursuing. The key benefit of concept testing is it increases the likelihood that a new idea will meet market demand and user needs by evaluating what’s valuable to the public. 

Concept testing is a research method to test the assumptions of a target market's needs and their willingness to purchase a solution to solve their needs. Achieving this involves interviewing users about an idea and discovering their acceptance and willingness to purchase through user research. By collecting this information before development and investment occur, you can make crucial decisions before further time and resources investment has incurred. 

What concept testing is not

The concept testing due diligence achieved before launching a new platform or service isn’t the same as new features released. The difference between concept testing and new features is concept testing is the purpose of a platform, while a feature launch is a property of the concept. 

In short, features are an attribute, and there can only be one concept with several features undergoing iterations to improve the solution. 

Concept testing is a means of validating an idea and its business model, which requires market and user research to determine the minimum viable product (MVP), roadmap milestones, and the platform's vision.   

Consider concept testing as the intermediary between scoping the problem and defining a solution as part of the discovery. Identifying a clear problem statement and the target market is necessary as part of the research process before you test the platform's hypotheses. Once you know what you’re solving for who, you can then generate a hypothesis and brainstorm ideas that will become features to test for the solution you want to offer.

You can solve the same problem in several different ways, from design, function, technology, pricing, and segment, and it’s important to identify who you want to cater to. For example, there are several task management apps on the Google or iOS store, but how many embrace neurodiversity or inclusivity? And which ones target SaaS companies—would SaaS companies use apps, or would they prefer a desktop app?

You can see why identifying who you’re serving before discovering what they need is essential to validate you’re ultimately serving a market and there’s a need from the market. Once you’ve solidified the concept, you can then move on to designing prototypes to experience the reaction of your target audience to new features.

Why concept testing is essential:

Saves resources and capital

Of all the new startup platforms released into the market from businesses, about 95% flop. And a significant amount of resources are wasted when a product fails to generate revenue or reach traction. This includes product development team efforts and capital spent in creating the product and other operational, financial, and legal costs. 

Concept testing uses a reduced, controllable portion of your budget and minimal personal and reputational investment. It can prevent you from launching an idea that doesn’t solve market needs and that users are unwilling to pay for. 

Validates concepts

Concept testing offers validation that users are willing to purchase and support your idea, enabling support from stakeholders, community, investors, and other team members. Materializing a concept isn’t easy, especially without management exercising caution and risks. But having concept testing validation allows even the most cautious team members to gain confidence in favor of the idea. 

Creates new business opportunities

Concept testing creates the foundation and stepping stones for a new organization or startup to venture on and become a leader in the marketplace. It shows the idea’s essential MVP features and functions gathered from user and market research as a desperate necessity for potential and future users to buy into. When this happens, you can capitalize on the idea's best attributes in the marketing campaigns. 

After evaluating what matters most to the target audience, you can use the learning opportunities to modify or completely change the features to optimize its success.

You can do concept testing at various stages, particularly when you need to validate ideas and make feature and platform strategy decisions. Below are some specific examples of when to run a test:

Before launching the product

Conducting a concept test before pre-launch allows you to determine the market needs for the segments targeted and learn if users are willing to pay before you invest further capital and resources. 

Discovery and feature ideation

Concept testing is critical during the product discovery stage since it identifies the solutions you should continue improving for a specific user problem. This also allows you to check your team's assumptions and validate features with users before proceeding into the development stage. 

Design iterations

After kickstarting your product design process, you can test design concepts. For instance, if you’re developing a new website, you can validate the low-fidelity concepts, such as the layout, graphics, and the brand.  

What do your users really want?

Just upload your customer research and ask your insights hub - like magic.

Try magic search

You can test your ideas using the following:

Comparison testing

Comparison testing involves presenting at least two concepts to measure which one is favored more and performs better through measuring responses. To quantify respondents, you need to use rating or ranking questions to calculate which is scoring higher. Another approach is to increase the options available and ask respondents to select the best concept displayed. 

Without a clear research goal from the question and refined user research question or too small of a sample size group, the results can lack context and fail to specify why the respondents preferred one over others. Designing and structuring clear user research objectives is important for your analysis and will later help you successfully launch a new feature or design preference. 

Monadic testing

Monadic testing aims to analyze a single concept in-depth. It involves segmenting the target audience into multiple groups and presenting only one concept to each group. 

The method of each group viewing a single concept in isolation allows you to maintain in-depth follow-up questions in person and maintain brevity in a survey. This makes it easier to ask multiple follow-up questions about the different attributes of a concept, such as what they like and how they would improve the pricing or change the look and feel. 

Monadic testing is an in-depth approach that gives clarity with more context around why the audience prefers a specific concept. 

Sequential monadic testing

Sequential monadic tests are just like monadic tests, but they differ as each group sees all the concepts instead of just one concept isolated. This process involves user researchers 

randomizing concept orders to avoid research bias and repeating follow-up questions to gain further insights from the respondents. 

Due to the increased user research quality, the sequential monadic test uses a relatively small target audience size, as they are able to go further in depth. This makes testing multiple concepts easier, more cost-effective, and ideal for research with a tight budget. 

However, sequential monadic tests have more questions to be answered by respondents since each person sees all the concepts. These long questionnaires may lead to a reduced completion rate and cause non-response bias. However, the collected insights become shallow if researchers limit the number of questions to reduce the questionnaire length. 

Protomonadic testing

Protomonadic testing involves combining sequential monadic and comparison tests. This works by the respondents first evaluating multiple concepts and then choosing their preferred one. The main benefit is it helps validate sequential monadic results. Further, researchers can confirm if the concept chosen in the comparison test is compatible with the insights gathered about each idea. 

Concept testing requires a survey designed to record the participant's feelings about your idea. The collected data helps establish what the respondents like or dislike about your idea. To put theory into practice, we’ve designed some tips below for creating an effective concept test survey:

Develop a goal

Establishing a main goal for your survey makes it easy to develop questions that will gather critical insights about your concept. Think about the purpose/actual motive of the test and the particular information you want to learn from the participants to improve the idea or solution. That way, you can create a survey with meaningful questions that offer unique insights into your customer's perspective. 

Create a consistent structure

When you create the questions, make sure they are ordered and grouped together to create a smoother flow and make it easier for participants to answer them. For example, do not mix follow-up questions, and make sure you order questions one at a time. This will help your respondents focus, reduce content switching, and provide more detailed and centered responses. 

Include Likert scales

Likert scales ask respondents to rank their opinions on a scale ranging from 'strongly agree' to 'strongly disagree.' The scale helps create a quantifiable measurement to rate or score a question, making it easier to analyze the question's results. 

Use visuals

Use images when you need user feedback about a visual concept. For instance, if you’re testing new logos, you can present them and ask respondents to select the image most appealing to them. This practice offers an unbiased format and easily digestible results for the researcher. 

Include demographic survey questions

Demographic questions in a survey inform user researchers of patterns in the segments and groups and ultimately confirm they fit your target audience. For example, a respondent may say they disliked your idea, but that doesn't mean it’s bad. That respondent may not be your ideal client and thus has no interest in your offer. 

Including demographic survey questions determines whether the concept will effectively suit your ideal customers and helps you understand how different user groups are responding to your questions. 

Below are the common cases you can apply concept testing:

Product development

Companies use concept testing to make decisions before launching new products. It helps identify the features customers care about and the pain points they face with features to solve them. With a concept test, you’ll know what your customers expect, make changes, and successfully launch your products. 

Designing new homepage

The homepage is crucial, especially for eCommerce and SaaS businesses, since it is the first point of contact with potential customers. When redesigning it, you must achieve specific performance measures, including user experience (UX). Concept testing UX allows you to present the designs to your target audience, who interact with them to iron out flaws in the design and prepare for a perfect launch. 

Logo testing

A logo is a critical part of your company's brand. It’s the first thing customers see about the business and helps tell your story. Before designing a new logo, you need to know how the customers might react to the new design. Concept testing logos help to evaluate different designs that resonate with your customers. 

Offers and pricing

A company may want to give discounts for a new product or introduce a new pricing structure. Before advancing plans and resources, testing the customer's initial response and determining the features that excite them will allow you to learn if the customers are interested and what they are willing to pay. 

Ad testing

Testing website ads, images, and banners to identify the best possible combination is common among companies. With marketing concept testing, the business can get insights into outcomes such as which ad results in the most conversions or grabs the most attention. The feedback comes from consumers, so you can trust it and make your advertisement and marketing strategy accordingly.

There are many ways and best practices to improve concept testing. Here are some top tips for performing effective concept testing:

Set a specific goal

A precise testing goal allows you to quickly evaluate if a concept is worth advancing to the next stage of resources, time, and capital investment. For instance, your aim can be 'we want at least 90% of our target audience to find the feature useful'.

Ask the right questions

Ask and phrase questions in a manner that a customer can understand and give you rich feedback. The right research questions, without bias, will uncover the insights required to validate your concept or establish what it’s missing. 

Have the right target audience

Conduct concept testing to validate your idea with the audience that fits the demographics of your customer base. Recruit the right participants whom you can use their feedback as guidance to develop features they need and launch a successful product.

Below are some common errors in concept testing and how to solve them:

Mistake #1: Conducting concept testing on a single concept in isolation

Running a one concept test means you miss out on the full benefit of comparative research data over time. Consider what you’re testing and how you’ll use the information once you get it. 

Depending on the research goal and stage, you might ask whether you should conduct repeat exercises to see if your product development process is on the right track for a concept instead of trying to improve its feature.  

Mistake #2: Testing multiple concepts in a single survey

Comparing lots of concepts in a single survey makes your participant's experience long, potentially confusing, but mostly tiring. Giving a reasonable number of questions to your respondents shows that you respect the energy and time and allows you to focus on which questions are most important to solve for the business today. 

Mistake #3: Abandoning too early

Canceling an idea because your target audience doesn't show interest may be short-sighted. Research helps you know what your customers feel about your concept. A negative feeling doesn't mean your idea is not valuable. Perhaps they’re not the right audience, or the pricing is too high. Refocusing the user group, asking the right questions, or pivoting the concept is very common and happens post launch for businesses. 

Mistake #4: Forgetting empathy

Not having enough empathy for user needs and experience and assuming participants think as the researcher is a common challenge. For example, sharing the whole idea in the survey introduction is too much—you should keep it focused. Overcommunication and forgetting empathy means you forget what’s happening in your respondents’ day-to-day lives. You should keep it relevant, simple, and easy to digest. 

Mistake #5: Failure to adjust the survey to cover all geographies

Failure to test different geographical regions could reduce your chances of expanding early on or discovering other potential markets. Consider targeting specific regions and including the language of where you want to widen the respondent pool. 

Various companies have applied concept testing before launching their products. Below are three popular concept testing examples:

Tesla

Tesla used concept testing for their Model 3 in 2017 to gain customer insights. The company qualified participants who could put down a deposit after familiarizing themselves with its features and benefits. This concept testing strategy allowed Tesla to gather invaluable customer feedback and raise capital from investors. 

Lego

Another concept testing example involves Lego, the famous toy making company. Lego consistently faced challenges selling its products to female children across North America of primary and elementary school ages. The company invested in concept testing to understand young girls' play habits and preferences based on age and gender. 

Their research revealed that girls disliked standalone structures. They preferred to create full environments and focused more on structure details and interior layouts. These insights enabled Lego to design a new line of products for young girls across North America that was very successful. 

Yamaha

Yamaha is a world-leading musical instrument manufacturer. The company faced difficulties choosing between knobs and sliding faders in its Montage keyboard. Concept testing gave them valuable and accurate data, which informed their decision to use sliders. 

As you develop your concept, it’s necessary to present it to your target audience and gather insights for data-driven decision-making. It saves resources, capital, and time and validates ideas while helping discover new business opportunities. Concept testing is convenient since you can conduct it during ideation, in the early design phase, or even before the product launch.

Concept testing reduces business risk to make sure you dedicate time, energy, capital, and resources to the right idea. It offers validation, substantiates product and design, and contributes to marketing ideas before launching the product.

The sample size needed to perform a monadic test is extensive since the target audience is in multiple groups. Further, various concepts need testing, which makes the sample size more significant. A large sample size means the research costs more and takes more time to complete, analyze, and aggregate results to generate a meaningful report.