Home/Blog/Modern Work & Data
Modern Work & Data · 11 min read

How to run customer interviews and usability tests that change decisions

The short answer

To run customer interviews that change decisions, start from the decision and its deadline, recruit by recent behaviour, get informed consent, and ask about specific past events instead of opinions or predictions. Watch five to eight people try the riskiest tasks. Then count how many people share each theme, write insights with evidence, and log the decision.

Want the tools, not just the method? The The Customer Interview and UX Research Kit kit turns this guide into ready-to-use templates, spreadsheets and printables, with an instant download.

Key takeaways
  • Write the decision, the decider and the date before you recruit anyone.
  • Ask about the last time something happened; avoid 'would you' and 'don't you think'.
  • Five users is a round size, not a guarantee: fix, then test again.
  • Count people per theme, not sticky notes.
  • Ask for recording consent out loud and give every recording a deletion date.
  • Use AI for a first pass only: redact first and verify every quote.

What are the steps to run customer research that changes a decision?

Most customer research fails before the first interview, because nobody agreed what it was for. The steps below put the decision first and the method second. Each step takes hours, not weeks, at the scale most product and marketing teams work.

  • 1. Write the decision, who makes it and by when, in one sentence with a verb (build, fix, drop, price, launch).
  • 2. Turn it into no more than three research questions, none of which names your solution.
  • 3. Choose the method for each question: interviews for why, usability tests for where people fail, surveys for how many.
  • 4. Recruit by recent behaviour with a short screener, and include people who use assistive technology.
  • 5. Get informed consent, ask out loud before recording, and set a deletion date for every recording.
  • 6. Write questions about specific past events and check them for leading or hypothetical wording.
  • 7. Run sessions with a moderator and a note-taker; debrief for 15 minutes after each.
  • 8. Synthesise as a team: one observation per note, themes counted by people, insights with evidence.
  • 9. Share an answer-first readout and log each decision with an owner and a date.

Why should research start from the decision rather than the feature?

A study that cannot change a decision is a cost with no return. If a team asks users whether they would like a feature it has already planned, positive answers confirm the plan and negative answers are explained away. Writing the decision first, with a deadline and a named decider, forces the team to say what evidence would change its mind.

Government digital teams build the same discipline in. The UK Government Digital Service asks teams to analyse findings together soon after each round, so no single person or stakeholder decides what the research means; that only works when everyone knows which decision the findings serve. The useful test is simple: if every possible result leads to the same action, do not run the study.

What questions should you ask in a customer interview?

Ask about specific past events, not opinions or predictions. In a classic experiment, people who watched the same car accident gave average speed estimates from 40.8 mph to 31.8 mph depending only on whether the question said the cars 'smashed' or 'contacted' each other. Wording shapes memory.

Predictions are even weaker. A meta-analysis of 47 experiments found that a medium-to-large change in what people intended to do produced only a small-to-medium change in what they did. So 'Would you use a feature that lets you work out with friends?' asks for a guess. 'Tell me about the last time you exercised with someone else' asks for a memory, and memories of specific events are the best material an interview can give you.

TrapWeak questionBetter question
LeadingDon't you think set-up is confusing?Walk me through how you set it up.
HypotheticalWould you use a friends challenge?Tell me about the last time you exercised with someone.
Double-barrelledHow often do you exercise and what stops you?Ask two separate questions.
ClosedDo you like the app?What happened the last time you opened it?
LoadedHow much do you love the short workouts?Tell me about the last workout you finished.

How many interviews and usability tests do you need?

For interviews, the useful idea is saturation: the point where new sessions add few new codes. An early experiment with 60 interviews found 92% of one group's codes after 12 interviews. A 2022 systematic review of 23 studies (17 with empirical data) found saturation within 9 to 17 interviews, mostly in fairly similar groups with narrow questions. Plan 5 to 12 per distinct group and stop when new sessions add little.

For usability tests, five users is a popular rule with a weak floor. When researchers tested 60 users and sampled random sets of five, some sets found 99% of the problems and others only 55%; with ten users the worst set found 80%, and with twenty, 95%. Test five to eight people, fix what you find, and test again with new people.

How do you run a usability test?

Write three to five tasks that give a goal and a context but never name the button or menu to use: 'You can exercise on Monday, Wednesday and Friday. Set up your weekly plan for those days' rather than 'Tap the plan builder and choose three days'. Write the success criterion for each task before the first session, and pilot the tasks with a colleague. A pilot nearly always finds one task that gives the answer away.

In each session, ask the participant to think aloud, reassure them that you are testing the product and not them, and stay mostly quiet. If you have to help, the task counts as a failure. Record task success as 1 or 0, the time taken, and what the person said. With six people, report '2 of 6', not '33%'.

After the tasks, the System Usability Scale gives a quick, free measure of perceived usability. Then list every problem, and have two people rate each one from 0 (not a problem) to 4 (blocks the task) without conferring. Evaluators often disagree about which problems matter, so discuss any gap of two points or more. Rank by severity times the number of people affected, but put anything that blocks a task on the fix list, however rare.

How do you recruit participants and handle consent?

Screen on recent behaviour, not identity or opinion: 'How did your free trial end?' finds the people you need more reliably than 'Are you a fitness enthusiast?'. Add a disqualifier for people who work in research or for competitors, an attention check, and an optional question about what would help people take part, such as captions, extra time or a phone call instead of video.

Recruit through at least two channels so you do not hear only from your happiest customers, and include people who use assistive technology when the product must work for them; the CDC reports that more than 1 in 4 US adults have some type of disability. Pay people for their time at the start of the session, and over-book by one or two, because no-shows are normal.

Give people an information sheet before the session that says who is doing the research and why, what you collect, how it will be used and shared, that taking part is voluntary, how long data is kept and who is watching. The GOV.UK Service Manual lists exactly these points and adds two practical rules: keep evidence of consent with the data it covers, and stop and delete if someone withdraws.

Use separate opt-ins for taking part, recording and quotes, and ask for recording consent out loud before you press record. The US Belmont Report's three tests are a good check on any process: did people get the information, did they understand it, and were they free to say no? Recording and privacy laws differ by place, so check your local rules with your privacy lead. University students should ask their ethics office or IRB before contacting anyone.

Measure where you are

Reading helps; measuring tells you what to work on. These AI-graded assessments on AssessAll pair with this topic:

What does a worked example look like?

Here is one fictional, composite study from a 70-person fitness app. The team wanted to build a social challenge feature for about $180,000. Instead of asking users whether they wanted it, the designer ran eight interviews, a usability test with six new users and a survey sent to a random 2,400 of 12,000 recent trial users.

The survey row shows why intent questions need a behaviour question next to them: 117 people said they would be likely to use a friends challenge, but only 14 named the lack of one as their main reason for stopping. The team paused the feature and fixed onboarding first.

MeasureResultWhat it meant
Interviews: restarting after a break6 of 8 people, both segmentsStrong theme
Interviews: wanting to exercise with friends2 of 8 peopleWeak signal
Usability: set a weekly plan2 of 6 succeededFix before release
Survey: main reason was a break97 of 286Largest single reason
Survey: main reason was friends14 of 286Small
Survey: 'likely' to use friends challenge117 of 286Interest is not behaviour
Survey margin of error (95%)About plus or minus 5.7 pointsSampling error only

How do you turn interview notes into insights?

Write one observation per note: what the person said or did, with their participant ID, not what you think it means. Group notes into themes as a team soon after the sessions; group analysis reduces any one person's bias. Then count distinct people per theme, not notes, because one talkative participant can fill a wall.

Turn each theme into an insight statement: who needs what, because why, so we will do what, followed by the evidence count. A theme that only one person raises can still matter if it blocks a whole group, such as a screen reader user who cannot complete a task. Counting supports judgement; it does not replace it.

What are the most common mistakes?

These six mistakes cause most wasted research. Each has a simple fix.

  • Asking whether people would use a feature. Fix: ask about the last time they faced the problem.
  • Recruiting only friends, colleagues and power users. Fix: screen on the behaviour that matters, through two or more channels.
  • Treating the recording notice as consent. Fix: consent out loud before recording, with a deletion date.
  • Ranking usability issues by frequency alone. Fix: rate severity with two people; anything that blocks a task goes on the fix list.
  • Quoting decimals from small or opt-in surveys. Fix: show the base and the margin of error, and report counts under 100.
  • Trusting AI summaries. Fix: redact first, verify every quote against the transcript, and have a person own each theme.

Can AI help with user research analysis?

Yes, as a first pass. A 2026 rapid review of generative AI in qualitative research found useful agreement on some coding but surface-level themes, missed nuance and fabricated or misattributed quotes in four of the five studies that tested quote extraction. Use your organisation's approved tool, remove names and identifying details first, code one transcript at a time, verify every quote word for word and keep counts in your own spreadsheet.

If you want templates, a synthesis workbook and small tools that redact transcripts and verify quotes, the Customer Interview and UX Research Kit from Bodhih Training includes them. To check your survey-writing skills, AssessAll's Survey and Questionnaire Design Assessment is a useful benchmark, and Jobulary can turn what you practise into an individual development plan.

Bodhih Pro Kit

Run your next study with every file ready

The Customer Interview and UX Research Kit includes the plan, screener, consent form, guides, synthesis workbook, practice transcripts and tools used in this guide.

Sources

  1. GOV.UK Service Manual (Government Digital Service): Getting users' consent for research
  2. GOV.UK Service Manual (Government Digital Service): Analyse a research session
  3. GOV.UK Service Manual (Government Digital Service): Using moderated usability testing
  4. US Department of Health and Human Services, Office for Human Research Protections: The Belmont Report
  5. US Centers for Disease Control and Prevention: Disability Impacts All of Us (infographic)
  6. Social Science & Medicine (Europe PMC record): Hennink, M. and Kaiser, B. N. Sample sizes for saturation in qualitative research: A systematic review of empirical tests. Social Science & Medicine 292, 114523
  7. Field Methods (copy hosted by Harvard University, Department of Sociology): Guest, G., Bunce, A. and Johnson, L. How Many Interviews Are Enough? An Experiment with Data Saturation and Variability. Field Methods 18(1), 59-82
  8. Behavior Research Methods, Instruments, & Computers (Europe PMC record): Faulkner, L. Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. BRMIC 35(3), 379-383
  9. Psychological Bulletin (Europe PMC record): Webb, T. L. and Sheeran, P. Does changing behavioral intentions engender behavior change? A meta-analysis of the experimental evidence. Psychological Bulletin 132(2), 249-268
  10. Journal of Verbal Learning and Verbal Behavior (copy hosted by University of California, Irvine): Loftus, E. F. and Palmer, J. C. Reconstruction of automobile destruction: An example of the interaction between language and memory. JVLVB 13, 585-589
  11. Journal of Medical Internet Research: Gierbolini-Rivera, R. D. et al. Generative Artificial Intelligence for Qualitative Methods in Health Research: Rapid Review. JMIR 28, e98551
  12. American Association for Public Opinion Research (AAPOR): Margin of Sampling Error / Credibility Interval
Common questions

Questions people ask next

How long should a customer interview be?

Thirty to 45 minutes suits most product interviews: five minutes for consent and warm-up, about 20 minutes on the story of the last time, ten on workarounds and five to close.

Should I pay research participants?

Usually yes. Pay a proportionate amount for their time, and pay at the start so they know it does not depend on their answers.

What is the difference between a research question and an interview question?

A research question is what your team needs to learn. An interview question is what you ask a participant, usually about a specific past event. Never ask participants your research question directly.

What is think-aloud in usability testing?

Asking participants to say what they see, expect and are trying to do while they use a product. The moderator stays mostly quiet and uses neutral prompts such as 'What are you thinking now?'

Is a SUS score a percentage?

No. The System Usability Scale gives a score from 0 to 100, but it is not a percentage. Compare it with benchmarks, an often-quoted average is around 68, and with your own previous rounds.

Does a margin of error apply to every survey?

No. It applies to probability samples and covers sampling error only. It does not apply to opt-in polls and does not capture nonresponse or question wording.

Do I need ethics approval for user research?

Inside a company, follow your privacy and research policies. At a university, ask your ethics office or IRB before contacting anyone, because class projects and publishable research may be treated differently.