What is Moderated Usability Testing?

Kassidy Kelley , Managing Editor

December 30th, 2022

Successful usability testing takes time and expertise to plan and execute. Moderated usability testing involves a process of user research, participant recruitment, testing user flows, and analyzing data. The end results will uncover prioritized problem areas that, when addressed, will provide new ways to satisfy and retain your users. 

Different UX methods are dynamically used depending on the team’s goals, product, and audience. However, the favored method of “moderated usability testing” is critical to truly understanding the problems your users face. Uncovering these pain points leads to actionable insights for transforming your product’s user experience.

Moderated usability testing gives you better control over testing environments and realistic/accurate insights, lets you fully understand the user’s experience, and leverages UX experts to design and run tests. The result is a more holistic, human-centric comprehension of the user’s journey while interacting with critical components of the product. This process is an integral part of an overall trend toward better usability, focusing on exciting, intuitive digital experiences that users will enjoy and return to.

Why usability testing is critically important

Moderated usability testing can be done in person or in remote sessions that allow back and forth between a participant and facilitator. This process includes planning, management, recruitment, recordings, analysis, and more. This teaches you about user behavior, identifies problems before release, and uncovers opportunities to better align with user expectations.

Often, usability testing irons out common issues that need to be addressed. Problems like not accounting for varying mobile resolutions, cluttered navigation, side-scrolling, or confusing content can wreak havoc on usability.

This can significantly impact the overall success of a product because once a user base decides your app is unusable, even an improved version may not gain traction. After all, the user remembers their precious dissatisfaction. In fact, while many companies are tempted to skip usability testing to save money or release faster, this can lead to disaster. Not only do you risk reputational damage, but you also face increased costs and delays, as fixing problems in production is slower and more expensive than fixing them in dev.

In fact, usability testing always leads to better products, with faster rollout and scale-up in the long run. Even though skipping usability testing can speed you up initially, rolling out bad products leads to more delays, upset users, and higher costs – if not abject failure.

We’re heading into a world where consumers demand great digital experiences, or they will move on to another product. Superb usability, which meets user needs no matter their hardware or technical ability, shows that you listen and care about your users. This is all but guaranteed with robust moderated usability testing and almost impossible without it.

How to conduct remote usability testing

Remotely moderated usability testing is scalable and less expensive than in-person testing sessions. An excellent remote usability testing strategy must include proper planning, management, tester recruitment, recording, and analysis. This means more than just giving users your product and letting them fly independently. Great testing scenarios define precisely what you’re looking to test – whether that’s checkout, signup, onboarding, or any other discreet functionality. What also matters? The specific testing team you are testing with. Recruiting realistic users that would typically use your product will provide real representative issues with your product. Start by setting clear goals – for example, “reduce dropout during signups by 50%” rather than “make signups easier.” From there, you can clearly define criteria for testers, ask the right pre-session questions, assign the right tasks, and successfully interview participants after testing. 

During testing, keep tabs on participant reactions. Things like what users say, whether they asked for help from the moderator, and notes on any pertinent information about how they interact with the product. Once a session is complete, there is a large amount of data from the testing session to analyze that will be critical indicators of your platform’s user experience. If the usability testing is conducted successfully, you will not only have an abundance of insights, but then you need to prioritize what issues are the most important to address and how to remedy them.

Moderated usability test plan

Setting up a usability test involves carefully creating a test scenario based on your current testing goals. The classic moderated usability test structure includes the following steps:

• Identify your target audience: Age, gender, location, usage habits, and tech familiarity
• Conduct pre-session interviews: Ask about lifestyle, background, tech habits, and other interests
• Proceed with actual testing tasks: These tasks should be pre-selected based on the goal of the test session; at this stage, record testers to get their vocal reactions at the moment as well as facial expressions and other physical actions
• Conduct post-session interviews: Get the tester’s opinions and general thoughts on the product, as well as issues they encountered and ask follow-up questions on any areas the user particularly struggled with.
• Send a post-test survey: get a quantifiable metric of your product’s user experience by creating a SUS survey.
• Analyze results and write a test report: This is your opportunity to look for patterns and apply data analysis tools to your results

Before you open testing to a broad group, try running a pilot test with one or two participants to ensure your test structure works. Once you’ve ironed out the procedure, scale up with a remote QA testing team and a bigger testing pool.