Quality assurance managers give no guidelines to exploratory testers. They just say, “Go ahead and explore!”
When it comes to agile development, exploratory testing is the pea in the pod. For new startups that have low budgets, exploratory testing is typically all they can afford. (Writing scripts is time intensive.)
So, this approach is widely adopted in certain arenas. But what about the holdouts? Let’s take a look at some of the big misconceptions that keep large corporations both in and outside of regulated industries from switching existing teams to an exploratory method.
News flash: exploratory testing doesn’t mean unstructured. It’s not ad hoc and it’s not freestyle.
Instead, QA managers give direction ahead of time. In fact, direction-giving is so commonplace that there are even different ways to describe it.
- Goal – what you want the tester to be able to achieve
- Lens – what type of user you want the tester to pretend to be
- Charter – the functionality or region you want the tester to explore
Each company comes up with their own method of divvying out guidance.
There’s no way to monitor exploratory testing so it can’t possibly count towards covering the entire product. It’s just throwaway work to do at the end if we have time. Wrong!
That mentality—while super common—is probably more representative of poor management than any natural failings of the exploratory approach.
In reality, exploratory testing has the ability to produce more documentation than your favorite bureaucracy. First, you’ve got output logs from the test system that records specific actions taken in detail.
Next, you’ve got reports submitted from the testers. Each company will have their own standards. Some reports can reveal time spent on test planning versus test execution as well as the proportion of testing relevant to the charter versus testing whose conception arose on opportunity.
Before testing even begins, most companies create a set of exit criteria—exactly where product performance needs to be before release.
Whether your approach makes use of output logs, session reports written by the tester, or both, you should have some breakdowns of the sessions, the coverage, the bugs, and the issues.
All that’s needed, then, is a tool that can search, locate, display, combine, and analyze that trove of data. You can create pie charts and metric goal markers until your coffee pot runs dry.
If you’ve got the FDA breathing down your neck, we all feel for you. Remember: regulators exist to keep us vulnerable humans from dying—not to stifle innovation. The FDA understands that software development is moving towards agile (and exploratory along with it).
Providing weak, insufficient evidence is absolutely something you don’t want to do come audit day. Reports, logs, risk assessments, charter documentation, and everything you need to successfully manage a testing team in the first place can demonstrate a professional, regimented approach to finding critical bugs.
Further proof? NASA has favored exploratory testing for decades.
Want a reason to adopt Exploratory Testing in your organisation? If it’s good enough for Nasa it’s good enough for us!
Even when testers are running off a script, they’re not robots. They still suggest changes to the test script writer as necessary or (gasp!) make the changes themselves.
So, while exploratory testing does require a lot of skill, so does manual testing in general. While exploratory testers are more engaged and do more critical thinking, it’s not like scripted testers do none. Your existing team can probably handle the switch.
Of course, it helps if a tester has some experience, but it isn’t necessary. Many testers now start their careers with the exploratory method. It’s absolutely possible to teach someone to be a tester without giving them specific use cases.
They just need to understand functions and features and have the ability to come up with ideas.
Because it is more flowing and less fragmented, exploratory testing actually feels more natural. One way to get a tester started is a with a resource folder that shows what the product is and what it’s supposed to do. Quick reads like marketing collateral, user guides, and demo presentation slides are an initial burst of inspiration for a tester tasked with coming up with use cases on the fly.
Cheat sheets can also serve as idea generation tools. A cheat sheet can be a list of questions that act like a double check system—helping the tester decide if they’ve run through everything in specific areas.
You’ll want to strike the balance between double checking for necessary testing and allowing for personal interpretation.
All types of testing have their place.
Misconceptions are just lame. You wouldn’t want to miss out on the benefits of project speed and app improvement because of misinformation.
And if you were secretly hoping that exploratory meant little documentation, we’re really sorry to have burst that bubble. If you want to freestyle, go right ahead. We’ll keep the metrics.