How to Run User Testing Sessions
User testing is the fastest way to find usability problems before they reach production. Watching real users interact with your product reveals issues that no amount of internal review can catch. Even 5 tests reveal approximately 80% of usability problems.
This guide covers moderated user testing: sessions where a facilitator guides the participant through tasks while observing their behavior.
Step-by-step guide
Step 1: Define test objectives
What are you testing: a new feature, a redesigned workflow, or the entire onboarding experience? Define 3-5 tasks the participant will attempt. Tasks should be realistic scenarios: "You want to create a spec for a new search feature" rather than "Click the Create button."
Step 2: Recruit representative participants
Recruit 5-8 participants who match your target user profile. Do not test with colleagues or friends: they know too much about the product. Use services like UserTesting.com, recruit from your user base, or use social media to find participants who match your criteria.
Step 3: Write a test script
Create a script with: introduction (purpose, ground rules, permission to record), warm-up questions (about their role and experience), task scenarios (what they will attempt), and closing (open-ended feedback, next steps). The script ensures consistency across sessions.
Step 4: Moderate without leading
During the session, ask the participant to think aloud: "Tell me what you are thinking as you go through this." Do not guide them: "Have you tried clicking that button?" is leading. Instead: "I notice you paused. What are you looking at?" The goal is to observe, not teach.
Step 5: Capture observations systematically
For each task, record: whether they completed it (yes/no/with help), time taken, errors made, confusion points, and direct quotes. After each session, write a 3-sentence summary of the key finding. This prevents the last session from overshadowing earlier ones.
Step 6: Synthesize and prioritize findings
After all sessions, compile findings. Prioritize by: severity (blocks task completion, causes confusion, or cosmetic issue) and frequency (appeared in 1 session or 5 sessions). Issues that block completion in 3+ sessions are urgent. Cosmetic issues that appeared once can wait.
Common mistakes
Leading the participant
Saying "You can click the blue button to continue" teaches the participant instead of observing them. When they struggle, wait. Their struggle is the data. Intervene only if they are completely stuck and the session cannot continue.
Testing with 1-2 participants
1-2 participants can be misleading. One person's confusion might be an outlier. 5 participants reveal approximately 80% of usability issues. If budget is limited, 5 short sessions are better than 2 long ones.
Only testing the happy path
Include tasks that test error states, empty states, and edge cases. Users encounter these scenarios in real life. A product that works perfectly on the happy path but breaks on the first error is not usable.
Not sharing findings with the team
User testing findings that stay in the PM's notes do not improve the product. Share video clips of key moments, present findings in sprint planning, and create tickets for the most critical issues.
Tips
- Record sessions (with permission) so the team can watch key moments
- Invite one engineer and one designer to observe each session live
- Test early with prototypes: finding issues at the prototype stage saves weeks of development
- Use a rainbow spreadsheet to track findings across participants and identify patterns
How Vantage helps
Vantage generates clickable HTML prototypes from PRDs that can be used directly in user testing sessions. Testing with generated prototypes before development begins catches usability issues at the spec stage, when changes are cheap.