Here are answers to some frequently asked questions that I receive about user testing:
What is SUS and why should I integrate it into my user test?
SUS or System Usability Scale is a questionnaire that measures user perception of satisfaction and usability of a product or service. The SUS questionnaire consists of standardized questions about how easy-to-use a system is. The SUS number can be benchmarked against other companies and industries. Integrating SUS into your user test can help measure the effectiveness of your product and improve its overall usability.
How do I find and recruit participants for user testing?
To find participants for user testing, you can start by simply asking existing customers or recruiting from representative users if you have access to them. Another way is to post a request on your social media channels and target social media channels that your potential users are likely to be on. You can also outsource the recruiting process to a recruiter, who can screen, schedule, and remind participants about their test appointment, and handle additional administrative duties such as administering incentives for participants (i.e. gifts or money), and in some cases travel/parking expenses.
How many tasks should I include in my user test?
You should limit your user test to 6 to 12 tasks due to time constraints and users’ mental exhaustion. This varies based on the device being used (desktop, mobile, etc.).
How many participants are needed in a user test?
As a rule of thumb, test with at least five users. Always recruit at least one more participant than you need, just in case one person cancels. According to usability expert Jakob Nielsen, testing with five users will catch 85% of the usability issues. To catch the remaining 15% of issues, you can test with 15 users.
At what points in my project should I conduct user tests?
Testing should be done throughout every stage of the product’s design, and continue to be done after the product launches. Products do not have to be complete to be tested. Testing can be done on the entire product or on just a few select screens. Testing can be done on a live site or a test site. Products being tested can be in draft form such as paper sketches, wireframes, or HTML prototypes.
What is the standard structure for scenarios and tasks?
During the user test, you will have several tasks that you want to target to attempt to complete. Each task should follow a standard structure, including setting context, providing the user with a specific task, and following up by asking a question to gauge the user’s experience. For example, to learn how well a person was able to navigate to a type of product, you could have them start on the home page and say, “You are here to buy a fire alarm. Where would you go to do that?”
How do I prepare for things that might go wrong?
Prepare for things to go wrong (and something always does). Consider the following:
- Some participants will be a few minutes late. If they are, but you still want to use them, what are the lowest priority tasks or questions that you will cut out?
- The prototype software could stop working or have a bug. Try to have a backup – such as paper screenshots – if you think this is a possibility.
- In a remote study, some participants will have difficulty using the video conferencing tool. Know in advance how the screen looks to them, what they should do, and common things that can go wrong so you can guide them through the
Can I use internal stakeholders as users?
It is not recommended to use internal stakeholders as test participants. It is important to select users who represent your actual user base to ensure that the feedback received is relevant and accurate. While internal stakeholders may provide valuable insights into what your users might value, they are not actual users and may miss important insights. However, if business stakeholders want to be involved in the testing process, pilot tests can be a great opportunity to include them.
What is the difference between user testing and usability testing?
In this resource, we use the terms usability testing and user testing interchangeably because the process is similar and they are often done at the same time. However, the goals of these two types of testing are different. Usability testing aims to identify critical problems that prevent users from completing tasks or achieving their goals with the product. On the other hand, user testing aims to understand whether the user needs the product and if it provides value to them. By conducting both types of testing, you can gain a comprehensive understanding of how users interact with your product and how you can improve their experience.
