Are you ready to take on QA automation?

Kassidy Kelley , Managing Editor

August 15th, 2022

Modern engineering and product teams must meet an increasingly high bar for quality and speed. While QA automation is marketed as a quick way to increase cadence to meet consumer demands, fusing manual and automated strategies creates a more effective, failsafe approach. Automated testing ensures consistency and immediate test execution, while manual testing offers flexibility and actionable insights. 

Yes, automation has immense benefits: accelerating repeatable tests, broadening coverage, critical flexibility, and cost savings. But automation is not a one-step process. You must first invest in staff + external tools to manage automated testing, identify candidates for automation, and have a continuation plan for flakey scripts.

First, assess your automation readiness with our checklist to reap the powerful benefits of automated QA testing.

Step 1: Perfect your manual testing

Ok… perfect is a strong word. But the first step to assessing if you’re ready for automated testing is to ensure your manual testing is efficient and effective. 

While automated testing has the allure of speed and reduced human labor requirements in a tight market, manual testing is the backbone of your automation journey. Manual functional, exploratory, payments, location, real device, and regression tests help scale early development efforts. These tests identify the repeatable input and output parameters that become candidates for automation. You have to create excellent quality manual tests before automation, or you run the risk of automating bugs into your automated test suite. 

Hastily writing scripts to automate test cycles and get back to development is tempting, but automating bugs is a major issue. And for QE’s who are converting manual test cases into automated scripts, it’s tempting to write the entire end-to-end manual test case into code. Unfortunately, both approaches lead to messy test suites and more maintenance time in the long run. Following software development principles is vital when writing automated test scripts, such as source code management, using design patterns, and investing in frameworks.

• Have plentiful manual test cases that are easily repeatable, efficient, and effective. 
• Consider software development principles when writing automated test scripts and developing an automated QA strategy.

Step 2: Know what to automate and what needs a human touch

Before you dive headfirst into a fully automated suite, you need to identify candidates for automation. Use the test automation pyramid as a guiding principle.

Where manual testing excels: 

• As a precursor: If you try to automate before starting or perfecting manual tests, you could automate bugs into your automation.
• Feature testing allows testers to independently test a new feature until it’s relatively bug-free without compromising the functionality of the entire project.
• UI and UX testing
• Failed tests. If an automated test fails, it’s usually up to a human to judge whether the test or the product code is at fault.
• Exploratory testing for higher-level assessment of complex business flow and real-life situations such as interruptions and display image orientation. 

Where automation excels:

• Unit tests. Start with unit tests to ensure that an automated testing suite is broken down into small parts, reducing your automated testing operation’s flakiness and run time. 
• API tests run pretty fast, but not as quickly as unit tests. They are also reliable because they often run without providing false results/ flaky tests. 
• Supporting DevOps to improve results velocity and provide development teams fast feedback.
• Repetitive and data-intensive tests.
• Happy path testing using known inputs and a clearly defined expected output.

While building your QA automation strategy, be mindful of time. It may take weeks to months to develop, write, and build a full test automation suite. In the meantime, manual testers can serve as backup for overnight and weekend regression testing while you work to build your automated suite. 

• Determine candidates for automation.
• Determine what tests should remain in human hands (and when you need manual fallback).
• Create a roadmap for automation using the test automation pyramid.

Step 3: Stack internal + external talent

To tackle QA automation, software companies need dedicated and knowledgeable Quality Engineers. Easier said than hired. A major hiccup on the road towards automation is the scarcity of qualified tech talent and the cost of recruiting and maintaining a full-time staff. 

Before executing your automation strategy, take stock of your internal talent. Identify holes for improvement and where you need to invest time and resources to take on automation successfully. Which employees have the skillset to write, implement and maintain scripts? Who can interpret the signals created by automated testing platforms? Who can build workflows and connect integrations? 

Heading to external partners and their integrated tools is vital to address capacity needs. For example, Testlio’s freelance network includes on-demand QEs to create, run, and optimize automated test scripts. They leverage frameworks like Appium and Selenium and help integrate with test automation platforms like Applitools, BrowserStack, Headspin, mabl, Sauce Labs, and Suitest. 

• Have a mix of QA and QE resources to oversee automation script creation, execution, and maintenance. 
• Create plans to translate and action the signals/results being generated by test automation tools.

Step 4: Tackle third-party tools, frameworks, and integrations

Now that you have a handle on your internal talent and have manual QA and QEs at your disposal, step four is to become familiar with automated testing tools and how they work. 

Spend time building a kickass tech stack that helps handle automated test creation, execution, feedback, and test runs. Use partners to maximize your investment in automated testing capabilities.

If you’re using Testlio’s QA testing platform, we have a selection of partners that integrate into our platform to maximize automation where needed. While they all provide the traditional benefits of software testing (releasing with confidence, coverage across devices, and O/S combinations), they also have unique features that squeeze out every last inch of ROI from your automated testing journey. Check out our partners: Applitools, BrowserStack, Headspin, SuiteTest, mabl, and Sauce Labs.

As part of your CI/CD pipeline or Agile planning processes, create a game plan for how to best stack and utilize these tools. Which tool are you using for what feature? How will you integrate signals from each into a more effective strategy? Embrace new partnerships to keep up with frequent release cycles, maintain a staff of expert software testers, and address capacity needs. 

• Make test automation investments in tools like Applitools, BrowserStack, Headspin, mabl, and Sauce Labs.
• Plan to integrate CI/CD, automated testing, and task management tools with a manual testing platform.

Step 5: Plan for automation failure 

The final step in this checklist is planning for the inevitable hiccups that occur with QA automation. Automation is notorious for coming to a screeching halt in the presence of flakey scripts. Implement fast-follow workflows where expert manual testers validate the results of automated scripts.

Take an incremental approach to automation. For example, before your first automated run,

it can take weeks to build a framework, design tests, write the code, and “test the tests.” The risk is that the app may have changed too much by running your first automation.

Start with a single test to automate. Then, incrementally add to that test, keeping the test suites running daily and updated alongside the app as it’s updated. Adapt frameworks like test-driven development (TDD) to your automation strategy.

When automating with the TDD methodology, devs start with small unit tests and write simple scripts to automate before returning to automating another element. Prepare to transition and adapt other existing initiatives (like shifting right or left) to match QA and QE needs. 

Finally, in the presence of these failures, be ready to tinker with your strategy. If automated runs continuously produce noisy or flaky results, convert them to manual runs until you can rework the scripts. Conversely, if manual testers run the same repetitive tests with clear results, shuffle their runs into the “to be automated” pile.  

• Implement fast-follow workflows where expert manual testers validate the results of automated scripts.
• Connect and adapt existing test automation initiatives (eg. shift left, shift right, continuous development) to reap higher ROI.
• Convert flaky automated runs into manual and repetitive manual runs into automated.