Creating an Effective User Testing Discussion Guide

There are a lot of moving pieces even in simple usability tests. A discussion guide is a document that outlines how each individual session should be structured, providing structure and organization to ensure usability tests run smoothly and yield the desired insights. If a usability session is a “play”, then the discussion guide is its “script”. Here are some tips on how to create and implement an effective discussion guide:

Importance of creating a discussion guide

With a discussion guide, you will enjoy:

• • Smoother tests, with less bumps. All parties involved can keep it moving like a well-oiled machine. Share this guide with all researchers ahead of the test so they understand how the session will be run.
  • Assurance that all questions and tasks are covered. Ensures that all questions and tasks get covered. Important to getting the metrics that are needed so that decisions/ recommendations can be made.
  • A guide for each participant helps when comparing notes and analyzing results later. Each person involved in the test has a blank guide for each participant. Here, they can write notes and observations throughout the test.

What to include in a discussion guide

A discussion guide can include:

• • End-to-end session flow
  • Scenarios/ Tasks
  • Questions
  • Guidelines/Rules

How to create and implement a discussion guide

1. 1. Determine the session flow: Begin by determining the end-to-end flow of the user testing session. This should include the introduction, setup, tasks, debrief, and closing. Having a clear outline of the session flow will help ensure that all necessary components are covered and that the session stays on track.
   2. Identify scenarios and tasks: Once the session flow is established, identify the scenarios and tasks that participants will be asked to complete. These should be based on the learning objectives for the user testing and should be designed to help elicit the desired insights. For example, if the goal of the user testing is to evaluate the ease of use of a new mobile app, scenarios might include navigating to a specific feature, completing a purchase, or submitting a form.
   3. Develop questions: Along with scenarios and tasks, develop a list of questions to ask participants throughout the session. These questions should be designed to help gather qualitative feedback about the user experience, as well as any pain points or areas of confusion.
   4. Set guidelines/rules: In addition to tasks and questions, set clear guidelines and rules for the user testing session. For example, participants should be asked to think aloud during the session, and researchers should avoid leading questions or influencing the participant’s behavior.
   5. Review and revise: Once the discussion guide is complete, review it with the rest of the user testing team to ensure that all necessary components are included and that the guide is clear and easy to follow. Revise as needed to ensure that the guide is tailored to the specific learning objectives and user testing context.
   6. Implement the discussion guide: Finally, implement the discussion guide during the user testing sessions. Make sure that all researchers are familiar with the guide and that it is followed closely during the sessions. Encourage note-taking during the session to ensure that all feedback and observations are captured accurately.