Request info

5 Ways to Test for Consistency

Consistency is critical to quality. Patient consumers who give your product multiple chances will never know what to expect if product results and behavior is inconsistent. But impatient people (those who ditch out after one feature glitch) will be lost customers, possibly forever.

Good QA should naturally have an eye on consistency. That’s why we work so hard to make sure as many variable combinations (whether those variables be device, motivation, or location) are covered as possible within the given cycle timeline and resources.

Moving beyond strategic test planning, here are five things to look out for to nail quality consistently.

We can all agree that accuracy is important. Consistency is another aspect that needs attention.

Michael Stahl

1. Consistency in published APIs

It’s important to make a good impression on all of your users, from those who will touch just the basic features to those who will rely on your extensive APIs.

Ways to remedy this include assigning one person ownership of the names and attributes used in the API syntax and for the QA department to review all APIs, especially when the documentation is too large for one person to handle alone.

2. Meeting market standards

Consistency isn’t about you (the business).

It’s about them (the customer).

What this really boils down to is that you need not only make sure you’re consistent with yourself, but with the rest of the world, or at least the market and industry standards.

Here’s a simple example: let’s say that a user fails to login to your mobile website. They get the login error message for using the wrong password, and both their password and email address have disappeared from the form upon reload.

But in all the other mobile websites they use every day, if they type the wrong password, then at least their email or username is retained so they don’t have to type it in again.

In this scenario, your company’s mobile website inconsistent with the rest of the customer’s life. Websites and apps (whether mobile or desktop) just keep getting better and better, and you have to keep up with expectations.

By understanding the pace of your industry, your competitors, your customers, and their needs, QA testers can keep an eye out for inconsistencies that will impact the perception of your company and the user experience.

3. Style and navigation consistency

Icons, colors, element placement and order…all of these things should be consistent so that when a user is switching between devices, there’s no confusion.  For multiple versions of a product, and even within a single app, QA testers can evaluate UI consistency with the ultimate goal of improving user adoption.

4. Rolling out massive upgrades

Well, what we want from software is consistency and insights. And that’s where things get tricky.

Most people are familiar with a time when their favorite app got a whole new look–and lost a lot of usability. With big changes, things get missed. A lack of mobile responsiveness across devices can be a huge problem when an app gets a new UI.

Consistency is a virtue for trains: what we want from a philosopher is insights, whether he comes by them consistently or not.

Stephen Vizinczey

To maintain product quality during updates and big rollouts, the QA team can help make the delivery schedule more doable for the entire organization. Which users should get the upgrade first? At what speed? Rather than feel frazzled by having to rapidly test major code changes, QA can help advocate for release times that will better serve all users.

5. Consistent documentation for testers, developers, and users

Perfectly consistent documentation is critical for:

  • Requirements and analysis
  • Test planning and instructions
  • User guides

But it’s too easy to forget about external documentation. How often do you review user guides? There may be PDF manuals published on the website or sent via email autoresponder that use different names or procedures than what is currently featured in the app.

The QA department can keep a log of all customer-facing materials, and review them for any inconsistencies following big software changes, then notify the marketing or customer support team if anything is found to be out of date.

Offering not only relevant, but consistent documentation to users can go a long way towards reinforcing the opinion of your company as high quality.

Don’t let inconsistencies slide (unless they don’t matter)

Even though consistency is a huge factor of quality, it’s alarmingly easy to let little inconsistencies slip past–whether in dealings from department to department, with external developers, or even with users.

It might be okay to accept one kind of inconsistency when consistency with something else is more important to people who matter.

Michael Bolton

Put on your “consistency glasses” for a minute, and look through them. Think about these five examples and consider any other areas within your organization where inconsistencies might be lurking.

When taking any steps towards improving QA processes, be sure to pause and make special preparations for maintaining consistency in (nearly) all things.

Jumbotron image