Question: How many users do you need to test with for a usability test?
Answer 1: = 5 users (Jakob Nielsen and Thomas Landauer, 1993).
Answer 2: = 15 users (Laurie Faulkner, 2004), PDF file.
So, which is it, 5 or 15? And why are we arguing about an extra 10 users, doesn't one need to test with at least 100 or more users for statistical significance, accuracy and validity?
Summary: UX efforts should improve their quality of data and informed decision-making by making sure feedback is coming from the right users.
Fake users are the wrong people who will provide you biased, tainted or innacurate and imprecise feedback. Precision is everything if you want to be successful with UX. Getting the right users, priceless.
At Experience Dynamics we learned this from doing it the other way (Hint: Your UX Agency does it this way most likely, ask them). Like many, we used to have a Market Research firm recruit for us, now we do it all meticulously in-house.
Market research firms provide recruiting (focus groups are happening all the time). But it's also very common to have user no-shows, luke-warm relevant feedback or someone that feels like a badly hired actor. We have not experienced any of that for many years-- that's when you know you are doing it right.
Summary: Usability testing is usually best conducted early on when you have concepts, wireframes or even static sketches or Photoshop compositions. There are really good reasons for early-on testing, related to the ROI of UX.
Sometimes some of the best usability advice comes from users directly. In some recent usability testing, a user said something which I found very perceptive.
He said: "The designs this company produces seem to be evolutionary, not revolutionary". He explained, "They tend to do small updates that don't really make a substantial difference or change the core problems with the interface...if I am going to spend my money on upgrading, I want something more revolutionary than evolutionary".
If you read nothing else on Demystifying Usability last year, read these top 3 blog posts from 2010...
UX blog post #1:These User Experience concepts I teach to my clients in in-house proprietary usability trainings:
How well you get your customers to their destinations with your design, and help users do what they need to do, is the difference that makes a difference in customer experience. If you are not doing it well, I guarantee you your competitors are or are trying to find a way to. In this post, I'll cover 5 strategic patterns that you need to lead the pack.
"Imitation is the highest form of flattery"...so the saying goes. The problem with borrowing design and user interface metaphors from other applications, websites or brands is that what works in one place might not work in another.
In this post we'll look at the pitfalls of copying design elements from other designs and what to do instead.
Maybe you have heard the saying "we'll take care of that in user training". The notion that problems users have can be resolved by user training is severely flawed. Yet entire departments rally around this belief and worse many companies seem to wrap product management around it.
The idea that you can educate users about how to use a user interface is misguided. The goal of usability is to create intuitive user interfaces. Intuitive means the design does not require understanding. Help and "user education" presupposes that both of these goals are possible.
Next week is World Usability Day, a day when the usability community gets out to raise awareness and visibility about the field and goals of usability engineering and user centered design.
"World Usability Day 2006 promotes the value of usability engineering
and user-centered design and the belief that every user has the
responsibility to ask for things that work better". (from the WUD site)
Here are a couple of items related to World Usability Day that we are doing at Experience Dynamics, a leading usability and user centered design consultancy.
The Importance of User Experience
Here's a poster that reflects some thoughts about user experience...all of the bottom row items (outcomes of positive user experiences) in the poster are based on empirical research. Let's review some of that research, a brief glimpse at the science behind what the poster is communicating...
About the poster project and translations into many languages below all these quotes (bottom of the post)...
Elements that contribute to a positive user experience: (the bottom row of the poster)
Loyalty > Trust > Perceived Credibility > Profitability > Intent to Return > Intent to Purchase > User Satisfaction > Word of Mouth