While exploratory testing has been around for over a decade, there are still a lot of misunderstandings about it.
Just because it’s user-inspired, doesn’t mean it’s disorganized. Just because it requires multitasking, doesn’t mean it’s difficult to keep track of.
Actually, exploratory testing can help discover bugs that traditional, scripted testing can fail to detect. Why is that? Let’s find out.
The definition of exploratory testing
Exploratory testing is a method of manual software testing where test case designing, planning, and executing all happen concurrently as the tester uncovers more and more of the app.
More than just “testing on the fly,” exploratory testing requires a great deal of experience and creativity to be effective. You can’t just assign it to interns and hope they know how to cover the app effectively.
To provide comprehensive app coverage, testers can be strategically split up based on any number of different factors:
- Feature sets
- Actions and large desired outcomes
- Types of user accounts
- Testers’ ability to work like users in certain features or areas
- Feature maturity level
In order for each tester to truly cover what part of the software has been assigned to them requires both innovation and experience. For example, when testing an app for managing a family calendar of tasks, events, appointments, and to-do items, a tester needs experience with what sort of features might be prone to cause bugs, such as complicated mobile gestures, and they also need to stay in the moment and creatively assess new testing opportunities, such as adding two collaborators to a to-do list instead of one in order to test the appropriate notifications.
Exploratory testing vs ad hoc
Let’s start with ad hoc testing so we can understand what exploratory testing isn’t. Ad hoc testing is an informal and very random testing approach. It’s not strategically organized. It doesn’t encourage testers to divide and conquer. It doesn’t have any set of specific goals or desired actions.
Exploratory testing is similar to ad hoc testing in that it involves a more natural, user-centric approach than running clearly defined test cases. But it’s different because exploratory testing does involve some strategy to get the most out of time spent in the app.
For this reason, exploratory testing acts as a great compliment when it is difficult to write test cases for certain features or desired results.
All in all, exploratory testing is more organized, strategic, and thought out. The test manager will have some motivation for how product or user areas are assigned, and the tester will have some level of understanding about what the app does to make sure they’re hitting it from all angles.
An example of exploratory testing
Seeing is believing, right? You might be wondering what exploratory testing actually looks like and how it works.
Well, it looks like hanging out in an app and writing bug reports when you find an issue. There are no pre-written test cases in a test management system, but a tester will likely have some sort of brief with a quick description of what feature areas and user types they’re supposed to test.
Here’s an example of exploratory testing by agile testing consultant and trainer Alan Richardson.
In this video, he walks through his accidental discovery of a bug in Github search while he was building a new Chrome browser extension for testers.
Why exploratory testing is essential
Now that we understand fully what exploratory testing is, let’s dive into why it matters.
- Similar to real user behavior – Using this method, testers explore the part of the app, user type, or objective assigned to them just as a real user would. This helps ensure that important steps and moments aren’t missed, which is common with scripted test cases.
- Realistic device coverage – The behavior isn’t the only thing that’s real. The devices are too. Testers explore the app using their own devices, devices at the office, their spouse or friends’ devices, etc.
- Allows for testing different environments – In addition to device types, testers can also cover other environmental factors such as internet quality, presence or absence of wifi, device storage, device battery level, etc.
- Replace or overlaps scripted testing – Exploratory testing can cover parts of an app that scripted testing struggles with, such as complex features and gestures. The method can also overlap more structured testing to ensure that nothing is missed even full coverage is supposed to be offered by the test cases.
- Specific yet also flexible – This type of testing is “specific yet flexible” which is an amazing feat in the software testing world, where scripted testing is extremely rigid and pre-planned and ad hoc testing is completely random.
- Perfect for localization – There’s no fast and easy way to test for localization. With exploratory testing, testers utilizing the app in their native language can check clarity issues, UX issues such as line breaks and character count, missing localized text, as well as translation problems.
- More accurate for regression testing – The exploratory method can be used as a form of regression testing to ensure that users aren’t having to struggle with bugs in older parts of the app. While automation is a popular way of handling regression testing, it isn’t enough. The exploratory method offers realistic results.
Related: Four Exploratory Testing Examples
Tools & best practices that help you do it right
Let’s take a look at some of the tools and best practices for exploratory testing so you can maximize this method.
- Nimbus – Because Nimbus makes it easy to capture sections of a page and scrolling screenshots, it’s a great tool for testers.
- Loom – Though screen video recording tool Loom has exploded in its use cases and popularity, it began as a testing tool, and it still a great one for filming the reproduction of bugs.
- Test management system – On a test management platform like Testlio, testers can follow an exploratory test structure with detailed instructions, including focus and out of scope areas. This ensures testers explore the parts of the app that make sense for the client and avoid wasting time on things that aren’t relevant. Furthermore, there should be transparency with all testers on the team, so everyone can see and search what has already been reported. This way, they won’t duplicate it.
- Stopwatch app – It can be smart for testers to use a stopwatch when testing. This way they can know how long it’s taking them to complete objectives and actions and make sure that’s a reasonable amount of time.
Best practices for exploratory testing
You’ve probably gleaned this from the post so far, but the most important thing you can do is make sure that there’s some sort of strategic management with exploratory testing. As mentioned, there aren’t scripts, but there should be specific briefs.
For example, a test manager might ask one tester to test every part of account management and settings features. She might ask two testers to pair up as a standard user and an account administrator with approval functions to test the review, approval, and publishing workflows. Strategy can also extend to device coverage, location coverage, etc.
You also have to be realistic about the number of testers required for exploratory coverage. Especially if device types and language are a concern, you need to get the right number of testers on your team to be effective.
How to get more testers on board to explore your app
Exploratory testing is the perfect complement to agile testing. In a way, it’s the most agile form of testing available, because the cycle of test planning and execution is super short and fast.
Every application can benefit from exploratory testing. But you need skilled testers on board to help.
Here are your options:
- Hiring and insourcing – At times, hiring additional testers is the right move, but if you don’t have a need for a full-time position, then it’s unnecessary and cost-prohibitive. In addition, you have to make sure this person is managed effectively.
- Crowdsourcing – While crowdsourced testing has some benefits (testers from around the world working while you’re sleeping or weekending), the downsides aren’t usually worthwhile. These downsides include no vetting of skills and no tester management.
- Networked testing – By tapping into a global network of skilled, vetted testers and fully managing their efforts for clients, networked testing offers both flexibility and ease of mind. This method is gaining popularity.
Exploratory testing should be done on every app, but it can’t be done by just anyone. It requires skill, creativity, and testing experience. From user behavior to devices and environments, it provides insights that other methods can’t.