What test cases should be automated (and which shouldn’t)

Developing high-quality apps involves pressure to make tradeoffs on speed, quality, and features to meet deadlines for release. This tension between speed and quality comes to a head with QA: you need a functional product but can’t afford weeks of turnaround time. 

Still, automation is not the solution at every stage. There are some QA metrics you just can’t test in a lab. Here’s a guide to what test cases should be automated, which ones should not, and how fused testing can change the game.

What test cases should be automated

Automation plays a critical role in a holistic software testing strategy. It’s a repeatable, efficient approach — especially for unit, API, and UI tests.

1. Unit Tests

Unit tests evaluate source code to validate that components work as expected. Tests are generally written in the same language as the underlying code and run quickly. Automated UI tests help catch issues earlier in the development process and create a faster cycle for feedback.

2. API Tests

Application program interface (API) tests simulate communication to ensure it functions as intended. You can ensure functionality by testing an application’s communication protocols and business logic. API test automation simplifies regression testing each time changes are made to the software.

3. UI Tests

UI tests simulate end-user interactions with the software’s interface. Automation makes testing easier and more in-depth. With today’s shorter release cycles, automation is necessary to speed up time to market.

Candidates for automation also depend heavily on where your org may see the highest ROI. Other test cases to consider: 

  • Cross-browser platform tests
  • Data environment tests
  • Performance tests
  • Software integration tests
  • Regression tests

Before running for maximum automation, beware: input data for automated testing can take considerable time and meticulous attention to detail to configure different test scenarios. So, evaluating what test cases should be automated means focusing on cost vs. time, tests you need to repeat frequently, or what automated tests will help scale beyond what manual testing could handle efficiently.

Jumbotron image

Three manual test cases that show what humans can do

While automated testing offers immediate test execution, manual testing provides the flexibility and coverage to verify app performance on real devices outside the test lab in localized environments. Manual testing is essential for verifying new features and offering actionable reproduction instructions for issues missed by flaky or broken automated scripts. 

When determining manual test cases, here are three great places to start:

1. Exploratory Testing

2. Usability Testing

While automation can test components, it struggles to evaluate the holistic software experience. Manual testing can help uncover software errors or friction points beyond the code base. Evaluating the customer experience is not the best candidate for automation; humans need to evaluate emotional responses, intuitive experience, and innovative features. 

3. Localization Testing

You’ll also want to stick with manual testing anytime you need a human reaction to something. It also makes sense to do manual testing when it would take longer to develop the testing script or when actions aren’t repeatable. Other test cases to consider: 

  • Payments testing 
  • Captcha verification
  • Visual testing
  • Real-world performance testing
  • End-to-end testing

How to fuse manual and automated testing 

Automated and manual testing strategies don’t have to work in isolation. The best piece of advice to what test cases should be automated is this: some. Not all. It depends! But let go of the mindset that automation is always right. 

To adeptly introduce automation into your manual testing process or to develop a fused testing strategy,  you’ll need the platform, process, people, and open architecture to accelerate testing.

  • Platform: Acts on signals from CI/CD, test automation, and issue management tools in a single platform.
  • Process: Coordinates transition between manual and automated workflow, including parallel testing, manual continuation and validation, automated fallback, and more.
  • People: In any development environment, you must have the right people to drive the process.
  • Open Architecture: An extensible approach lowers switching costs between automation frameworks.