What test cases should be automated (and which shouldn’t)

Developing high-quality apps involves pressure to make tradeoffs on speed, quality, and features to meet deadlines for release. This tension between speed and quality comes to a head with QA: you need a functional product but can’t afford weeks of turnaround time. 

You can’t skip QA: the true cost of software bugs – the direct cost of mitigating the defects and the indirect cost of decreased consumer trust – is extraordinary. But, you can optimize your QA strategy by embracing automation alongside manual testing. 

Whenever you can comfortably automate a test, you can reduce the workload and see benefits such as quicker time to release, lower costs, increased accuracy, and scale. Nearly a quarter of companies that initiated automated software testing showed an ROI within the first six months.

Still, automation is not the solution at every stage. There are some QA metrics you just can’t test in a lab. Here’s a guide to what test cases should be automated, which ones should not, and how fused testing can change the game.

What test cases should be automated

Automation plays a critical role in a holistic software testing strategy. It’s a repeatable, efficient approach — especially for unit, API, and UI tests.

1. Unit Tests

Unit tests evaluate source code to validate that components work as expected. Tests are generally written in the same language as the underlying code and run quickly. Automated UI tests help catch issues earlier in the development process and create a faster cycle for feedback.

2. API Tests

Application program interface (API) tests simulate communication to ensure it functions as intended. You can ensure functionality by testing an application’s communication protocols and business logic. API test automation simplifies regression testing each time changes are made to the software.

3. UI Tests

UI tests simulate end-user interactions with the software’s interface. Automation makes testing easier and more in-depth. With today’s shorter release cycles, automation is necessary to speed up time to market.

Candidates for automation also depend heavily on where your org may see the highest ROI. Other test cases to consider: 

• Cross-browser platform tests
• Data environment tests
• Performance tests
• Software integration tests
• Regression tests

Before running for maximum automation, beware: input data for automated testing can take considerable time and meticulous attention to detail to configure different test scenarios. So, evaluating what test cases should be automated means focusing on cost vs. time, tests you need to repeat frequently, or what automated tests will help scale beyond what manual testing could handle efficiently.

Learn how to create a more holistic QA strategy that fuses automated scripts with on-demand manual testing. Download our eBook.

Three manual test cases that show what humans can do

While automated testing offers immediate test execution, manual testing provides the flexibility and coverage to verify app performance on real devices outside the test lab in localized environments. Manual testing is essential for verifying new features and offering actionable reproduction instructions for issues missed by flaky or broken automated scripts. 

When determining manual test cases, here are three great places to start:

1. Exploratory Testing

Users don’t often follow logical patterns when navigating software or apps; they jump around and move unpredictably. Structured exploratory testing services can help find bugs that get past developers and uncover any issues missed from automated scripts before release. Expert manual testers can also take unplanned paths through software to cut through the noise and target high priority bugs, learn more about exploratory testing.

2. Usability Testing

While automation can test components, it struggles to evaluate the holistic software experience. Manual testing can help uncover software errors or friction points beyond the code base. Evaluating the customer experience is not the best candidate for automation; humans need to evaluate emotional responses, intuitive experience, and innovative features. 

3. Localization Testing

If your app reaches across borders, prepare for things to get lost in translation. And when apps are incompatible with culture, language, or devices, users won’t (or sometimes can’t!) accurately use your app. Localization testing looks for these flaws in design, language, and UI and can only be done with manual testers in different locations.

You’ll also want to stick with manual testing anytime you need a human reaction to something. It also makes sense to do manual testing when it would take longer to develop the testing script or when actions aren’t repeatable. Other test cases to consider: 

• Payments testing 
• Captcha verification
• Visual testing
• Real-world performance testing
• End-to-end testing

How to fuse manual and automated testing 

Automated and manual testing strategies don’t have to work in isolation. The best piece of advice to what test cases should be automated is this: some. Not all. It depends! But let go of the mindset that automation is always right. 

While most teams already combine manual and automated workloads to some degree, fused testing allows teams to work together more collaboratively. Fused testing merges machine and human resources in a blended workflow that enables shift-left and shift-right principles. Continuous testing requires parallel testing and transitions between manual and automation testing to produce optimal results.

To adeptly introduce automation into your manual testing process or to develop a fused testing strategy,  you’ll need the platform, process, people, and open architecture to accelerate testing.

• Platform: Acts on signals from CI/CD, test automation, and issue management tools in a single platform.
• Process: Coordinates transition between manual and automated workflow, including parallel testing, manual continuation and validation, automated fallback, and more.
• People: In any development environment, you must have the right people to drive the process.
• Open Architecture: An extensible approach lowers switching costs between automation frameworks.