User testing: what analytics will never tell you
Analytics tells you where customers drop off. It never tells you why. And the “why” is where the fixes with real impact are hiding.
The numbers show that 70 % of people abandoned the pricing page. Whether they left because the price was too high or because they never found the price at all — the numbers won’t say. You only learn that by looking over a customer’s shoulder.
What user testing actually is
It isn’t a focus group. It isn’t a survey. It’s observation.
You sit next to a real person from your target audience — or watch them over a shared screen — and follow them through your site. You don’t give instructions. You don’t steer. You stay quiet and watch where they hesitate, where they get stuck, where they leave.
The participant thinks out loud, describing what they see and what surprises them. You take notes.
No special technology. No lab. Five people, an hour of your time, and a willingness to listen.
Why five people is enough
In 1993 Jakob Nielsen and Tom Landauer published an analysis that changed how UX research works: five users surface roughly 85 % of the significant usability problems on a site.
The sixth, seventh and eighth participants bring diminishing returns — the same problems start repeating. The added value of each extra session drops fast.
Five isn’t a compromise. It’s a deliberate choice that lets you test quickly, repeatedly and cheaply. Instead of one large twenty-person study a year, you run three rounds of five — and improve the site continuously rather than occasionally.
What we found on a real project
We worked with an online consulting platform. The operator knew conversions were low, but analytics couldn’t explain why. We ran a user test with five people from the target audience.
None of them hit a technical error. None of them left because the product failed to interest them.
Three barriers still came out — barriers analytics would never have surfaced:
A customer didn’t realise the product was online. She read the service description, browsed consultant profiles — and still assumed she’d be going to an in-person meeting. The site never said otherwise. It was so obvious internally that nobody had written it down.
The “Call” button caused anxiety. Not a technical problem. A psychological one. She didn’t want to call on impulse without knowing who she’d speak to or about what. The barrier wasn’t the price or the offer — it was one word on a button.
The consultant ratings looked fake. Every rating was five stars, with no text and no context. She said it out loud: “This looks fake.” Social proof meant to build trust was reducing it instead.
Every finding had a concrete fix. None required a redesign. These are the kinds of barrier that repeat across sites — the most common conversion mistakes are surprisingly consistent.
How to prepare a test
Recruiting. Look for people who match your audience but don’t know your site yet. A fresh pair of eyes is the whole point. Five is enough for a first round.
The scenario. Don’t give the participant tasks — give them a goal. The difference matters.
Wrong: “Find the pricing page and click Order.” Right: “Imagine you want to try a consultation for the first time. Do what you’d normally do.”
A task leads the participant by the hand. A goal reveals how the site actually works.
Think-aloud protocol. Ask them to narrate what they see and think. It feels unnatural at first — it’s fine to nudge gently: “What’s going through your head right now?”
What to watch. Three signals: where they hesitate (hesitation = ambiguity), where they ask a question (a question = missing information), where they leave without completing the action (leaving = a barrier).
What to do with the results
Don’t look for statistics — look for patterns. If three of five people hit the same problem, that’s a pattern. If one person does something unusual, it’s interesting but not yet a reason to redesign anything.
Prioritise by frequency and severity. A problem five out of five hit that stopped them cold gets fixed first. A problem one person hit that merely slowed them down can wait. How to think about prioritising fixes more broadly is covered in the guide to where to start raising conversions.
Simple fixes first. Test results surprisingly often lead to copy changes — adding a sentence, renaming a button, filling in a missing piece of information. Before anyone says “redesign”, go through what can be solved without a developer. How to write copy that leads people to action is a separate piece.
The takeaway
Analytics is irreplaceable. It tells you what is happening. Why it happens you only learn from customers themselves. If your analytics shows symptoms but you don’t know where to look for causes, the overview of why websites don’t make money is a good starting frame.
User testing isn’t a research luxury. It’s five people, one afternoon, no special equipment. And the output is specific barriers with specific fixes — not a general “improve the UX”.
If you want to know where customers hesitate on your site and why, that’s the direction Kaen works in — systematically, with concrete findings written down and a recommendation for what to change first.