Apps used to be so large that they came on CDs. Today, software updates are small enough to download in the background of our smartphones. As our expectations for technology have increased, engineers have kept pace by continuously delivering smaller enhancements and feature upgrades to the end user.
In a recent Atlassian survey, 80% of respondents used agile processes to build products faster and 65% of respondents relied on continuous delivery shorten their release cycles.
Continuous delivery requires teams to develop a “test early and often” mentality. Saving QA for the end of release cycles risks discovering a large issue that could require extensive reworking.
Testing early and often can be challenging for QA teams. How do you test something that is incomplete? The wrong strategies can cause the need to retest everything after sprint completion, slowing down the release schedule. Luckily, this conundrum can be solved with a mix of communication, refined testing methods, and advanced tools.
What is (and isn’t) continuous delivery?
Continuous delivery is a DevOps practice that results in small, continuous updates and releases. The development process produces code that is simultaneously tested and as such completely ready for release at the end of the sprint. Continuous delivery is the practical, product-minded concept often coupled with the overall DevOps approach, which includes large company culture shifts and the breakdown of business and sales silos.
The practice ensures that code can be quickly deployed to production by moving all code changes to a production-like environment during a sprint. So rather than save testing until the end, small changes are continuously tested. That way at the end of the sprint, the changes are ready for release.
Code can be deployed, but it doesn’t have to be. With continuous delivery, teams make the decision to deploy manually. They can do a final round of manual testing, or wait for an additional change.
Continuous deployment, on the other hand, automates the entire process all the way through release. Deployment happens straight away and any manual review or testing occurs in the actual production environment.
The right model differs for every team, but because so many organizations are hesitant to take automation to the deployment level, we’re focusing here on continuous delivery, the more widely used DevOps practice.
Reduce manual processes for requirements inputs
Continuous delivery requires that teams automate wherever possible, and not just in QA. Building and documenting processes must be automated as well.
One area that teams still struggle to automate is requirements. Is everyone working from the right inputs?
If not, quality can’t be built in from the start. If requirements are ambiguous, then getting QA and development to “done and done” at the same time is near impossible.
Most teams still rely on Excel spreadsheets and Word docs to track and share requirements for updates. This makes any changes to requirements harder to implement during sprints. Because they’re open to interpretation, requirements can be read differently, causing QA to design test plans out of line with the finished build — which slows everything up.
Certain quality management solutions offer better traceability from test plans to requirements and helps everyone on the team stay in the loop on any changes. A higher level of team communication and more accurate test planning reduces ambiguity when QA and development is happening side by side.
Test in a realistic, production-like test environment
A production-like environment is the backbone of QA for continuous delivery. As soon as components are ready, they have to be tested in a life-like environment, before the entire version or update is ready.
Creating a test environment with real deployments and services is essential to continuously validate applications from end-to-end. This staging environment will be used to test each small change so that testing moves just as quickly as development. The test environment must mimic real production conditions as closely as possible, including connectivity, database activity, and number of servers.
Conduct testing at the unit level and deliver continuous feedback
Once they’ve build a staging environment, QA teams can test components as they’re made available.
Chief Technologist at CA Technologies Naga Jayadev puts it this way:
Imagine if Boeing built a plane like the way we build software today. If we build the fuselage, the wings, the engine, and we say, “We got to test it out.” Would you want to be the pilot on that first flight when we’ve not tested anything and we try to fly the plane? You don’t, right? We can’t fly the plane and say, “Let’s go replace the pilot if it failed.”
We, unfortunately, do that in software today. One of the ways to look at a small piece of the puzzle is to say, “Let’s take a look at how Boeing is building the plane.” They build the fuselage, they test it in a wind tunnel. The fuselage is tested in isolation in a wind tunnel to make sure that that performs very well against the specifications it’s got.
You don’t see Boeing’s flights fail in their first test. There might be some problems, but those problems are discovered in that isolation testing.
Our goal is to decouple all of these components and test everything in isolation in a wind tunnel.
Continuous delivery is all about testing like Boeing. There are several benefits to this approach:
- Testers can drill testing down and isolate problem components
- Testers can help identify the causes of any bugs
- Testers can deliver more information to development, making fixes faster
- There is less risk as the sprint progresses
- Releases are faster
Use service visualization to test integrations and break through bottlenecks
But what about communication between components?
Units and features can’t only be tested in isolation. Yet for continuous delivery to be possible, teams can’t afford to wait until the build is completely integrated to test the full release.
Service virtualization eases that strain by mimicking the missing components and allowing engineers to test cases that stretch across features and functions.
Third-party services that are integrated with an app or site can often cause QA bottlenecks if they are either unavailable at the time, too costly to use for testing, or haven’t been integrated with the product yet. With service virtualization tools, teams can create components that can be used repeatedly in both development and testing environments.
Continuous delivery makes it possible for organizations to deliver high quality software faster than before. The process leverages manual testing to confirm and validate new builds and to monitor their performance, rather than to identify large, costly defects.
The challenge for any QA team is getting started. Jayadev recommends reaching for the lowest hanging fruit. Identify the biggest QA bottlenecks, the biggest pain points, or what will have the most impact, whether it’s automating for better version control or reviewing a build before deployment. The most important strategy is to continually remove any blockages that keep the delivery process from moving along smoothly.
For expert testing that keeps pace with your team, get in touch with us.