Mitigating risk is a chief responsibility of every QA tester. How can we identify problem areas and prioritize testing? How can we discover risk early on in the sprint to help keep development on deadline?
All QA testing should address risk to some extent, but a truly risk-based approach works by maximizing the amount of risk that can be mitigated by testing.
It’s all about encouraging QAs to be hyper-aware of the factors that can cause project failure and use testing to combat them.
Why risk-based testing matters more than ever
Agile development and timely deployments rely on testing that occurs as early on in the sprint as possible. The more behind QA gets in terms of coverage deadlines or requirements changes, the more risk there will be in the code changes as they near release.
To be successful, risk-based testing must be collaborative. QA managers need to actively seek out the information needed to assess the content of the next sprint. Product managers, developers and other testers should be consulted during sprint planning to conduct a holistic risk assessment.
Mitigating risk has always been an important element of software development. Particularly with enterprise software, where security or performance issues can ruin the workday for a large organization. But now, more than ever, with everyone from executives to customer support on board with collaborative, fast-paced agile development, risks have to be uncovered early on.
The four main types of risk that affect a product
Risk isn’t just technical. Risk is created throughout an organization, not only during coding. Here are the four main types that affect software:
- Project risks – Has this sprint been particularly confusing? Maybe there have been staff changes that have caused some gaps in knowledge transfer, an issue with a third-party service, or difficulty getting relevant feedback from clients or shareholders. Some risks are created by the human element of software and the management circumstances surrounding the project.
- Product risks – These types of risks are far easier for testers to identify. Product risks can be functions of the product that are new and as of yet under-tested or that are historically prone to failure.
- Technical risks – Some risks aren’t necessarily the fault of the product but are larger technical concerns that are based on infrastructure, on calls to external web services, or on anything else that can affect overall security and performance.
- Business risks – Some features are so critical to customers that if they ever did fail, it would mean chaos. Any mission-critical aspects of a product may always be regarded as high-risk simply because they mean so much to the business.
Factors that contribute to risk and how to calculate it
Because risk is not just about data or the newness of a feature, its calculation is qualitative. But that doesn’t mean risk assessment isn’t incredibly useful.
Items are typically categorized as low risk, medium risk, high risk, very high risk.
Having those categories is far more useful during QA than one big lump of apocalyptic fear. So how do you create them?
You’ll first want to start with any historical data that you do have. What does your backlog of bugs tell you about future testing? Where are problems for the upcoming sprint likely to be concentrated?
Next, identify all immediately recognizable risk factors. What comes to mind right away? Maybe it’s vacation time taken by someone who excels at code review. Maybe it’s a feature that is particularly challenging to unit test.
Once historical and obvious risks are identified, go down the list of the four types of risks (project, product, technical, business) and simply brainstorm what else should be filled in for each. Incorporate input from others on the team. You’ll also want to examine the product map and review all features and functions for concerns.
It’s helpful to identify the bulk of risks for that particular sprint before assigning low-high values so that everything can be taken into consideration and weighed together.
When assigning qualitative values to each factor, consider likelihood versus impact. If a particular feature has a low likelihood to fail, but an extremely high impact of potential failure, then it is a high-risk factor.
How to respond to risk with the right levels of testing
Let’s start with a project risk as an example. Say that two QA testers will be on vacation at the end of the sprint you’re currently in planning mode for. Are there certain scripts you need them to write before they go? Maybe a developer on the team can help with exploratory testing during the sprint’s last couple of days? Some risks will require planning in terms of resources and time.
But most risks that you identify will help inform your QA strategy in terms of prioritization and coverage.
For low-risk features, maybe you only test the code changes but don’t do regression testing. For features of moderate risk, you’ll likely do a mix of partial regression testing (on the basic, feature-critical tasks) and testing on the code changes.
With really high-risk features, you’ll layer up as many methods as possible:
- Complete regression testing
- Use cases that cause appropriate coverage overlap
- End-to-end path coverage
- Testing through the lens of a variety of relevant user roles
The more strategies you layer over high-risk features throughout a sprint, the more likely you are to help the team solve problems quickly.
Some things are always risky: integrations with third-party services or other apps, data security, scalability, and performance. Mixing continuously risky product components with one-off concerns during test planning means that you’re not losing sight of overall product quality or failing to cover new code changes.
Even with a risk-based approach to testing, things still go wrong. When this happens, it can be helpful to write a “lessons learned” report. Get input from others in your department or team as to what contributed to an event of data corruption, or a failed user task, or a big crash. Simply asking around, uncovering some root causes, and drafting a quick, shareable write-up can make it so that any large issues aren’t just something you don’t want to repeat, but something you know HOW not to repeat.
For testing that heightens the user experience and integrates with your tools and your team, get in touch with us.