What is regression testing?Before we dive into regression testing, let’s first define it and understand what a regression is.Software regression testing is a type of software testing that ensures changes and additions to software code have not introduced new issues into previously working functionality. A software regression then is any unwanted change that occurs from code changes. A real-world example of this is you take your car to a mechanic to get the air conditioning fixed, and when you get it back, the air conditioning is fixed but the gas tank sensor no longer works. Bummer.To raise the stakes further, imagine you’re working on an airplane engine. You need to replace a component – say, a turbine disc. After you’ve done the job, you wouldn’t just check that the disk works in isolation; you’d also test the entire engine to ensure everything else functions as expected. For example, you’d want to ensure that you haven’t accidentally impacted how gases leave the combuster, that the compressor no longer maintains proper cabin pressure, or that the new disc caused any other damages.In software, a regression can happen after a new feature is implemented. When you change or add a piece of code (like replacing the disc in the airplane), you need to not only verify that the new code works as intended but also run regression tests against the existing functions of the software (like the other parts of the plane) to make sure they haven’t been inadvertently affected by your changes.Let’s say an email photo sharing service introduces video compatibility, but after the capability is released, the basic function of sharing photos to a set group of email addresses is completely broken. Given that a regression is an unintended change, then regression testing is the process of hunting for these changes. The regression testing definition is simply an unintended issue caused by code change.A top 8 regression testing test case checklistThere are many scenarios when it makes the most sense to perform regression tests. To explain the test cases, we’ll look at automating regression tests for an e-commerce website that recently added a discount coupon feature. Here are the top 8 test cases selected, tied with the regression testing strategy:Cases with frequent defects: User registration often had login errors. The login process needs thorough regression testing to ensure seamless user access. Obvious functionalities: Product browsing, a highly expected feature, must be tested to ensure the product information is correctly displayed. Cases that verify core features of the product: The checkout and payment process, a core e-commerce feature, requires testing to maintain trust and reliability. Cases of functionalities that have undergone recent changes: The newly added discount coupon feature requires rigorous testing to ensure it functions as intended. All complex and integration test cases: The shopping cart function, integrated with multiple features (product browsing, checkout, payment gateways), needs detailed testing to avoid failures. Boundary value test cases: Testing edge cases, such as maximum and minimum quantities for product purchases, ensures the system can handle limit conditions. A sample of Successful test cases: Test user registration and successful purchase with a discount coupon is essential for a smooth user experience. A sample of Failure test cases: Attempt invalid actions such as applying expired discount coupons or registering with an existing email to confirm error handling.Why should regression testing be automated?The words “automated” and “regression testing” go hand-in-hand. When companies first consider how they can maximize their QA teams’ potential through automation, they look to regression tests. This is because regression tests are naturally good candidates for automation. In general, automated test cases should be:Stable Repeated frequently Simple Worth maintainingBecause regression testing is for existing functionality, it requires constant repetition. Every release cycle should include a fair amount of regression testing to ensure that no new development causes bugs or breaks.When regression testing is automated, it allows for checks into a variety of changes and frees up testers to conduct manual exploration into more unusual cases in the production environment. Not all regressions are caused by new features or the consequences of routine bug fixes. They can also be caused by database updates or new browser versions, in the case of a web app. A regression can also be an issue with efficiency and speed. Automating those cases which are stable and repeatable allows manual testers to spend more time on testing various environments and for merging more complicated cases together at a higher level.Related: 8 steps to implement a manual regression testing strategyChallenges and how to overcome themAs with other forms of automation, regression testing is subject to the challenge of test case maintenance. The more test cases we automate, the more we can verify quality for existing functionality. But the more test cases we automate, the more test cases we must maintain. If we write the test cases too loosely, they’re too easily passed, but if we write them rigidly, they’ll need to updated and rewritten as the system changes.The question of what to automate can only be answered with a deep dive into an individual application, but a good starting point when determining regression test candidates is to ask which components of the system have been affected by a code change?With each new release, you can list out or create a mindmap for affected areas, then determine at the unit testing level, integration testing level, and system testing level which cases will need to be repeated and will be worthwhile to maintain.When internal resources are in short supply or automated testing seems too daunting, using crowdsourced testing for overnight or weekend regression testing works wonders. For example, a leading provider of hotel accommodations with over 10 Million installs crowdsourced their regression testing to augmented their small in-house QA team. What had taken the internal team three weeks to test is crowdsourced to Testlio’s network and completed over weekends. The process is more efficient than adding additional full-time testers and provides the speed and flexibility necessary for today’s continuous development initiatives. Testlio’s do it for me (DIFM) approach to managed QA testing immediately takes the pressure off the in-house team so they can focus on higher-level testing. The entire process includes:Twenty (20) testers to work in 2 to 3-hour bursts to knock out the regression testing over weekends POS and localization testing Comprehensive device, OS, and real user app version upgrade coverage, as not all of their users immediately install the latest updatesFor more details on this and other use cases, read about Testlio’s regression testing capability or drop us a line.Automated regression testing toolsTools like Selenium and vTest can be used for automating functional as well as regression testing, allowing you to achieve more in a single platform. With Selenium, you can automate the test scripts, as well as the result reports. While the execution of the tests are light years faster than manual tests, everything else about automated testing takes more time, which also means it can initially cost more money. Writing the tests and setting up the suite requires an initial time-consuming effort that will only pay off if the pre-defined tests truly overlap with an overall strategy to create excellent app coverage. Automated regression testing servicesIn the long run, automated testing can save an organization money spent on human resources, but only if they are fully prepared for the investment. With any turnover in QA employees, the proactive and retroactive processes for maintaining automated test suites can easily fall by the wayside. If a business wants to experience long-term cost savings by automatically testing for software regressions, but doesn’t have the internal resources to set up and maintain the suite, then working with an external service provider is an excellent option. Eight examples of services offered by best-in-class automated testing companies include:Alignment and integration of your engineering workflows with the automated software testing service’s testing process The design, build and maintenance of accurate test scripts The management of test runs and setups for you The review and validation of test results and defects, while also taking care of noise and flaky tests Real device utilization Access to build management tools Automation run systems The integration of reporting and insightsSome benefits are:Fit automated testing into an overall test strategy Get results continuously and for upcoming releases No need to commit human resources No risk of forfeiting ROI with inevitable organizational changesIf a customer asked you to demonstrate to him, within an hour, that your newest software is ready to use, what tests would you run? Are these the same tests that you are now performing in your first hour of regression testing? They should be.Steven WoodyPracticing selectivenessUltimately, regardless of WHO does it and HOW it is done, regression testing requires that we practice selectiveness. What features and functions are affected by new code?What is the minimum amount of test cases that can cover the maximum amount of code?Which unit level tests can be merged or eliminated?By always (always!) being selective about what test cases require automation in the first place, you create a more stable automation suite that is less subject to inevitable software changes. Do you need help with your regression testing? Testlio can help, schedule a free demo today.