Usability vs preference—what’s more important?

The other day I was having a conversation with a few of my friends, and UX snuck its way into the chat (we can’t help it sometimes!). One friend mentioned their team was trying to assess a redesign. They had done a lot of testing to compare the original design with the new (and improved!) one, but they wanted to know if people liked the new design more than the old one. It turns out, the majority of people preferred the old design.

Now, this happens a lot. For example, I worked on a similar redesign for an app and participants mentioned they preferred the outdated 90s-style design rather than the more modern elements we thought would be a hit. My friend was at her wit’s end already having invested a lot of time and effort into the process. Should they revert to the old design and scrap the whole project?

Absolutely not. 

The problem with preference

Preference is a slippery concept, and measuring user preference is like hanging on to a fish. You might get it for a second, but it will soon disappear from your grasp. 

The problem with preference is that it is hugely subjective. For example, if I asked you if you prefer chocolate or vanilla ice cream, you could likely tell me which one you like more. Or if I asked if you liked winter or summer more, you might have an answer. 

But why? I much prefer vanilla ice cream to chocolate. I like the taste much more, as well as the consistency. And I love summer because I always tend to be cold and love walking outside in nature. On the other hand, you might like vanilla ice cream because it reminds you of a specific childhood memory, and you may like summer more because you love to swim outside. The problem with preference is that so many confounding variables go into a given answer, producing widely varied responses across individuals. 

You won’t be able to account and design for preferences. 

Additionally, when we ask participants what they prefer design-wise, they might be able to tell you, but we also might be forcing them into an unlikely situation. For example, how many times have you landed on a website, and they’ve asked you, “Would you like to view the website with a blue or purple background today?” Or, “would you like the banner to be an image that spans across the screen, or an image in a box, accompanied by text today?”

People aren’t used to making these decisions. One person might prefer a huge image banner with text overlayed on top of that, but it will likely be an accessibility nightmare. When we ask participants what they prefer, they might choose, but we can’t be sure why they made that choice or if it doesn’t violate any heuristics or design needs for others.

Instead, measure usability

Now that you’ve knocked preference off your to-do list, what should you be doing instead?

Have you ever used an ugly app or website? Something that was just so outdated or weird-looking but just worked? Despite the ugliness, you could quickly get in and out of it and achieve whatever goal you needed to get done. Have you ever used Amazon? I am not saying Amazon is ugly, but they certainly haven’t modernized it. Why has Amazon been able to get away with the same general design for so long? 

Because it is usable, and we, as humans, use products that help us achieve our goals, not because they are beautiful (with some exceptions, of course).

Just like Maslow’s hierarchy of needs for humans, there is a hierarchy of needs for users. Aaron Walter created this hierarchy, and it stands true. 

Before anything, a product needs to have a function. If you create something with no value or purpose, no one will feel compelled to use it (let’s refer back to Heinz’s purple ketchup or Segways).

Next, it has to be reliable. If a website or app keeps crashing or something we are using continues to break, we will move away from that product to find a replacement. 

Then we come to usable. If the product is impossible to learn or use, we will abandon it to find something more manageable. 

Finally, there is pleasurable. If the product hits on the above needs, people can appreciate the aesthetics or delightful aspects of the experience. People can only notice a beautifully redesigned product if it is first usable.

The cornerstones of usability

Instead of focusing on preference in your next test, focus on the usability of the product. During usability testing, you should focus on:

1. Efficiency—how quickly people can complete tasks

2. Effectiveness—whether or not people can achieve their necessary tasks

3. Satisfaction*—how people feel about the usability of your product 

By measuring these three usability aspects, you can benchmark your current design and compare the new design. If your latest experience is more efficient and effective than the old one, it is a good bet that more people will use your product or stay with your product for longer (hitting on acquisition and retention metrics!). 

*One quick note on satisfaction when it comes to usability. A lot of companies ask users how satisfied they are with a particular product. This is different from satisfaction when it comes to usability. When you measure satisfaction within the context of usability, it is about whether or not the user found the usability satisfactory, not the product. An excellent resource for this is the System Usability Scale (test-level satisfaction) or the Single-Ease Questionnaire (task-level satisfaction).

Still struggling with a redesign?

What if your product is functional, reliable, and usable, but people still hate your new design? I highly recommend that you read about the status quo bias. This bias causes us, humans, to prefer how things currently are and when things change from that baseline, we perceive this as a loss. We don’t like change, and seeing change impacts our ability to make sound decisions, which is why people might choose an outdated design over a new, sleek one.

eBay did a great job of overcoming this bias. They changed their yellow background to white, and complaints flooded in, so much so that they reverted to the old color. Then, slowly, over a few months, they changed the color from yellow to white, without any complaints. If you consider a more usable redesign, gradually making changes might be a great happy middle ground for your users.

Have you struggled launching new designs for your customers? Is preference something that comes up a lot in your research? Come and tell us more about it at our bustling community Slack channel.

Written by Nikki Anderson, User Research Lead & Instructor. Nikki is a User Research Lead and Instructor with over eight years of experience. She has worked in all different sizes of companies, ranging from a tiny start-up called ALICE to large corporation Zalando, and also as a freelancer. During this time, she has led a diverse range of end-to-end research projects across the world, specializing in generative user research. Nikki also owns her own company, User Research Academy, a community and education platform designed to help people get into the field of user research, or learn more about how user research impacts their current role. User Research Academy hosts online classes, content, as well as personalized mentorship opportunities with Nikki. She is extremely passionate about teaching and supporting others throughout their journey in user research. To spread the word of research and help others transition and grow in the field, she writes as a writer at dscout and Dovetail. Outside of the world of user research, you can find Nikki (happily) surrounded by animals, including her dog and two cats, reading on her Kindle, playing old-school video games like Pokemon and World of Warcraft, and writing fiction novels.