What Is Exploratory Testing? Testlio May 17th, 2024 If regular scripted testing follows clear directions on a map, exploratory testing ventures off the beaten path to uncover hidden defects. This guide will teach you what exploratory testing is, why it’s critical, how to conduct exploratory testing sessions, and techniques to maximize your results. Table of Contents What Is Exploratory Testing? What Is Exploratory Testing in Agile? Why Use Exploratory Testing When to Use Exploratory Testing Types of Exploratory Testing Pros and Cons of Exploratory Testing How To Do Exploratory Testing Exploratory Testing Techniques Conclusion What Is Exploratory Testing? Exploratory testing is a form of software testing in which testers design and execute tests on the fly, without relying on predefined test cases. It involves simultaneous learning, test design, and test execution. Testers use their creativity, intuition, and experience to probe the application for defects that scripted testing may miss. Rather than following a set of steps and expected results, exploratory testers dynamically generate and modify their tests based on the application’s actual behavior. They ask questions like “What happens if I do this?” or “What should the system do in this scenario?” and devise tests to answer those questions. For example, when testing an e-commerce site’s checkout process, an exploratory tester may try entering a negative quantity or dollar amount to see if the system catches those invalid inputs. This unscripted probing can uncover defects that only appear in unexpected user paths. What Is Exploratory Testing in Agile? In agile development, this style of testing provides immediate feedback when there isn’t time to develop and run a full suite of scripted test cases. It allows testers to quickly learn about an application and adapt their testing to maximize defect discovery with each sprint. For example, let’s say an agile team is developing a new e-commerce website. They might use exploratory testing during each sprint to rapidly validate core user flows like browsing products, adding items to the cart, and checking out. The tester can freely navigate the site, trying different scenarios and looking for issues. They might discover that the checkout breaks when applying certain coupon codes, or that the product search doesn’t handle synonyms well. By uncovering these types of defects early through exploratory sessions, the team can fix them before the next sprint demo. It also excels at validating the user experience, not just functionality. Agile teams can use exploratory sessions to assess subjective qualities like usability, accessibility, and emotional response that are difficult to verify with automated tests. Continuing the e-commerce example, an exploratory tester could evaluate how intuitive the site navigation and product categorization is for users. They might identify that the “Add to Cart” button doesn’t have enough color contrast for those with visual impairments, or that the site copy has an inconsistent tone that damages the brand perception. Insights like these allow you to iterate not just on code quality but on crafting an outstanding customer experience. Without the open-ended nature of exploratory testing, many of these subjective concerns could go unnoticed. Need help prioritizing QA in your Agile planning? Head here. Why Use Exploratory Testing It’s impossible to find every bug by only following predefined scripts. Even the most comprehensive test plan will have gaps because it’s based on assumptions about how users should interact with software, not how they actually do. Exploratory testing adds another layer of verification by intelligently probing areas that structured test cases don’t cover. It excels at finding bugs spanning multiple features, subsystems, or technologies that are missed by siloed scripted tests. It can dig up complex defects that only appear under intricate scenarios or edge cases, like a banking app allowing users to overdraft their accounts when making rapid back-to-back withdrawals. It spots user experience issues like confusing navigation or unhelpful error messages that aren’t captured in the requirements. Take Basecamp, for example. The popular project management tool does extensive exploratory testing, especially for its mobile apps. With so many third-party dependencies, different devices, and possible configurations, relying only on scripted tests isn’t practical. Instead, Basecamp has its developers and designers regularly dogfood the apps, performing unscripted testing of common user flows and new features. They make a point of trying scenarios that span the seams between the mobile app, API, backend systems, and even companion apps like email. This hands-on testing by the development team, in addition to scripted tests, helps Basecamp catch interaction bugs early and deliver a more seamless user experience. When to Use Exploratory Testing It’s especially valuable early in development before requirements are nailed down. Testers can start learning the system and identifying risk areas right away, rather than waiting for a full specification and test plan. Quick rounds of testing provide immediate feedback to shape the product design. Even organizations that rely heavily on automated and manual scripted tests find value in exploratory testing. Microsoft is well known for extensively automating test cases for Windows and Office, but it still uses exploratory testing to complement those scripted tests. Exploratory testers dig into new features more deeply than automated tests. They probe error handling and application states that are hard to script. They approach the applications with a user mindset, finding issues in the end-to-end experience that siloed test cases miss. Microsoft has found that this combination of automated scripted tests to check for regressions, plus exploratory testing to discover new defects, maximizes test coverage and application quality. What Is A Simple Example Of Exploratory Testing Consider a team developing a new search feature for an e-commerce mobile app. To give fast feedback, they decide to do some testing before writing a full set of test cases. The exploratory tester starts by searching for basic terms related to products sold on the site, like “shoes” or “t-shirts.” They verify relevant results are returned in an easy-to-read format with crucial details like price, ratings, and shipping options. Next, the tester tries some more advanced searches. They search for very broad terms that return hundreds of results, like “electronics,” and very narrow terms that return only a few results, like “navy blue long sleeve cashmere sweater.” The tester checks how well the search handles synonyms by searching for “slacks” and “dress pants” to see if they return similar results. Next, they try searching for a competitor’s product names to see if the app suggests relevant alternatives. To assess the tolerance for user error, the tester enters misspelled queries like “running shoos” or “Nikee sneakers” and verifies the app suggests the correct spelling. The tester also attempts searching with special characters like “men’s dress shirts & ties” and “Levi’s 501 jeans” to see if those trip up the search. As they test, the explorer adapts their strategy based on the results. If searches for synonyms fail, they dig deeper into that area, trying different word variations. If misspellings are handled well, they deprioritize those tests to investigate riskier scenarios like searching for vulgar language. Throughout the session, the tester records their searches, results, and any defects found. They report their findings to the team, who fix the most critical issues before release and integrate the other ideas into the regular test plan. By performing exploratory testing early, the team identifies major defects when they are easier to fix. They also generate dozens of new test ideas with minimal time invested in test case design. Types of Exploratory Testing While exploratory testing is guided by the tester’s instincts and experience, there are several distinct styles that provide different benefits. Understanding the different testing types allows testers and teams to select the ideal approach for their situation. Unstructured and Structured At the highest level, it can be divided into two main categories: unstructured and structured. Unstructured exploratory testing, also known as freestyle testing, has no formal rules or guidelines. Testers are free to interact with the system in any way they see fit, following their intuition and curiosity. This type of improvisational testing is ideal for quickly getting a feel for the system, smoke testing new builds, or investigating particular areas of concern. For example, when a new feature is implemented, testers may spend some time in unstructured exploration to gauge its behavior and look for any obvious problems before proceeding to more systematic testing. In contrast, structured exploratory testing involves some level of planning and organization. The degree of structure can vary, from loose guidelines to comprehensive charters outlining specific testing goals and strategies. This ensures that the testing effort is focused, efficient, and aligned with project priorities. Strategy-Based Testing One common form of structured testing is strategy-based testing. Here, experienced testers employ specific strategies and techniques to perform focused, rigorous testing of the system. Some common strategies include: Boundary value analysis: Inputting extremely high, low, or edge-case values to stress the input handling Error guessing: Attempting common error scenarios like entering invalid data or navigating pages out of order State transition modeling: Mapping out the different states of the system and the transitions between them Testers select the strategies that are most applicable to the system under test and their goals for the testing session. For instance, boundary value analysis is helpful when testing a search feature to ensure it gracefully handles searches with no results or an extremely large number of results. Scenario-Based Testing Another structured approach is scenario-based testing, where testing is guided by realistic user scenarios. The team develops detailed scenarios that describe how an end user would actually interact with the system to accomplish a goal. Testers then use these scenarios to steer their exploration. As an example, an ecommerce website could have a scenario like: Search for a product Add the product to the shopping cart View the shopping cart Proceed through checkout Enter payment information and complete the order By walking through the path a real user would take, the tester can evaluate the overall workflow and usability while watching for any technical defects. Scenarios are often derived from user stories, use cases, or real customer journeys. Collaborative Testing Approaches Although exploratory testing can be an individual activity, it’s often beneficial to take a collaborative approach. When multiple testers work together on a testing problem, they bring diverse perspectives and complementary skills that enhance the overall effectiveness. One common collaborative testing approach is pairing an experienced tester with a new tester or domain expert. The experienced tester shares testing skills and techniques, while the less experienced participant contributes fresh viewpoints and application knowledge. For instance, a new tester may ask questions or try workflows that an experienced tester has become accustomed to, uncovering hidden assumptions and issues. Another collaborative approach is mob testing, where a whole team comes together for an intense testing session. This could include developers, product owners, UX designers, or other stakeholders in addition to testers. Mob testing helps build a shared understanding of system quality and gets many diverse eyes on the product simultaneously. Focused Sessions with Charter-Based and Timing-Boxed Testing When there is a need for a bit more formality, charters and timeboxes provide a lightweight structure. With charter-based testing, the tester or testing team develops a concise mission statement for their testing session, often focusing on a specific quality attribute like usability, security, or performance. The charter guides the tester’s mindset and focus during the open exploration. Timing-boxed testing applies a time constraint, challenging testers to uncover as many important defects as possible within a limited window, typically a few hours. This creates a sense of urgency and emphasizes finding the most critical issues quickly. Charters and timeboxes are often used together to create focused, mission-oriented testing sessions that produce tangible results. For example, a team may schedule a two-hour charter-based session with the mission of evaluating the accessibility of a new user registration workflow. This keeps the team on track and accountable. Pros and Cons of Exploratory Testing Like any testing technique, although powerful there are pros and cons to exploratory testing as listed below. Exploratory Testing Advantages Exploratory Testing Limitations Can catch defects that fall between the cracks of predefined test cases by deeply probing the application logic and user experience Can be difficult to know how much of the application was actually covered since testers are making up test scenarios on the fly Provides a safety net for finding issues not anticipated by specifications or unit tests Two testers exploring the same feature are likely to use very different data and paths, making results hard to reproduce Fast and cheap to start since there aren’t test scripts to write and maintain For applications that require documented, traceable test coverage for compliance reasons, exploratory testing alone usually isn’t sufficient Testers can begin exploratory sessions as soon as there is working code, without needing a complete feature or formal test plan Requires a higher skill level than simply following predefined test scripts Highly efficient for getting broad test coverage quickly Testers need strong critical thinking abilities and enough domain knowledge to pick the most important scenarios to probe Valuable when requirements are changing frequently, as exploratory testers can adapt their approach on the fly Testers have to be comfortable with uncertainty, making quick decisions about what to do next based on what they are observing Effective way to assess the actual user experience, not just raw functionality, by simulating realistic user behaviors May need to invest in training and coaching since it takes practice to get good at it. Comparing the pros and cons Despite these drawbacks, you will find that a combination of scripted and exploratory testing strikes the right balance between test coverage, reliability, and efficiency. How To Do Exploratory Testing Step 1: Strategize and Plan Before beginning, strategize which areas of the application to focus on. Analyze the product’s risks to determine the most critical user flows and features that need in-depth probing. Review the planned changes for the release to identify areas likely to have flaws based on the development team’s past issues. Use this input to generate time-boxed exploratory testing charters that will guide the team’s efforts without constraining their creativity. For a music streaming app, an example charter might be: “You have 60 minutes to test the process of creating a new playlist on iOS devices. Verify that users can add both individual songs and entire albums to a playlist, reorder tracks, and share the playlist on social media.” Step 2: Set Guidelines and Documentation Requirements To keep testing efficient, set clear guidelines and documentation requirements. Specify the key data testers should capture during their sessions, like the workflow tested, the variations tried, the results observed, and any notable questions or ideas to investigate further. Standardize how testers should write up the defects they find so developers can quickly reproduce and fix them. Using an exploratory testing management tool to record the testing activities and their results can make this process easier. Step 3: Adopt User Personas and Pair Testing Urge exploratory testers to adopt different user personas during their sessions. They should put themselves in the shoes of a new user, a power user, and a casual user to see if the application meets those distinct needs. Encourage testers to work in pairs so they can feed off each other’s ideas and question their assumptions as they work. For applications with specialized domain knowledge, consider bringing in a subject matter expert to join the testing. A tax accountant might catch issues in tax prep software that testers miss, while a warehouse manager could predict problems with a logistics app. Step 4: Monitor Progress and Hold Debriefs As the testing is happening, check in with the testers periodically to see what they’ve covered so far and what strategies they’re using. Test leads should help get testers unstuck if they run out of ideas and redirect them to other areas if they’ve saturated their charter. For larger efforts, hold a short debrief at the end of each day so the testing team can share interesting defects, demonstrate new techniques, and decide where to focus their efforts next. Step 5: Review Results and Plan Next Steps After finishing a round of exploratory testing, carefully review all the bugs logged and session notes. Analyze the issues found to identify themes that might represent deeper risks worth investigating more. Look for user flows that generated a lot of questions from testers and consider doing additional targeted exploratory testing or research with real users to get answers. Discuss the results with managers and developers so everyone has visibility into the quality level of the application and what it will take to get it ready to ship. Step 6: Repeat the Process Finally, plan to repeat the cycle again as the application evolves. Like any testing style it isn’t a one-and-done activity. As the development team fixes known bugs and adds new features, exploratory testers should reassess the product to find issues with the latest changes. Dedicate time for exploratory testing in every sprint so it becomes a habit. Exploratory Testing Techniques We’ll explore testing techniques suitable for both beginner and advanced QA professionals. Whether you’re new to exploratory testing or looking to take your skills to the next level, you’ll find practical tips and real-world examples to help you get started. Beginner Exploratory Testing Techniques Creating User Scenarios One of the foundational skills in exploratory testing is being able to think like a user. Testers should put themselves in the user’s shoes and construct realistic scenarios that describe how they would interact with the application to complete common workflows. For example, let’s say you’re testing a new e-commerce application. A core user story is the ability to add products to a shopping cart and then check out. To construct test scenarios around this story, you might come up with the following: A user browses the product catalog, adds several items to their cart, then views the cart to verify the correct products and quantities were added. A user adds a product to the cart, continues shopping, but then decides to remove the item from their cart before checking out. A user with an expired account attempts to add a product to their cart. The application should gracefully handle this error case and prompt the user to reactivate their account. A user tries to add a product that is out of stock to their cart. The application should display an out of stock message and not allow the item to be added. 2. Going Off-Script Don’t be afraid to color outside the lines. In addition to the basic user flows, try doing unconventional actions like: Rapidly double-clicking buttons or links to see if the application can handle the extra events gracefully. Using the browser back and refresh buttons in the middle of a multi-step workflow to ensure the application maintains its state correctly. Entering form fields out of order or submitting a form with missing required fields. Entering very long strings of text, special characters, or even code snippets like SQL or HTML into text fields to see how the application responds. Uploading very large files or unsupported file types to test validation rules and error handling. The goal is not just to verify that features work as expected, but to actively attempt to “break” the software by performing unpredictable actions that real users might try. Testers need to put on their hacker hat and construct creative ways to misuse the application. More often than not, this probing will uncover defects and inconsistencies that would never be caught by a test script. 3. Proactively Preventing Defects Exploratory testing isn’t just about finding defects after the fact – it’s also about proactively identifying potential issues. Testers should carefully analyze requirements documentation and design artifacts to spot missing functionality or ambiguous specifications before development even begins. For example, imagine you’re reviewing the design mockups for a new feature that will allow users to export data to a CSV file. Some potential gaps you might identify could include: The design doesn’t specify what should happen if the user doesn’t have permissions to export data. Do they see an error message, or does the export silently fail? There are no details on how the export will be triggered. Is it a button, a link, or a menu option? On which screens will it be available? The design mentions that users can select the columns to include in the export, but doesn’t explain how this selection will be made. Will it be a multi-select box, checkboxes, or something else? By proactively identifying these types of gaps and getting clarification from stakeholders, testers can avoid rework and prevent many issues from ever making it into the finished software. Creating visual flow diagrams of the planned user journeys is another great way to map out the application and highlight potential problem areas. These flow diagrams might reveal overly complex user flows, dead-end paths, or places where the application flow doesn’t align with business processes. Finding Gaps in Requirements and Workflows One fruitful avenue for exploratory testing is to analyze the application’s design and requirements for potential gaps in functionality before they manifest as defects or additional development work. Many software teams document requirements and workflows graphically or in writing. Examining graphical representations of workflows can help testers spot missing pieces, while written requirements may require you to map out the intended functionality yourself to identify gaps. When analyzing an application for gaps, be sure to give yourself ample time to thoroughly investigate. Consider how and where the application interfaces with databases and other connected systems, exchanging data back and forth. Check if page data refreshes at appropriate times and in the right ways. If the app includes email or SMS messaging capabilities, either for communicating with users or for internal purposes, carefully test this functionality for issues, as hookups with third-party messaging services are notorious sources of bugs. 4. Mapping Out Test Cases with Mind Maps Mind maps—graphical representations of ideas and their relationships—are a go-to tool for project managers to ensure nothing falls through the cracks. They enable you to take a specific function, user story, or workflow and diagram all the related scenarios you’ll want to test. Mind maps for testing can quickly branch out and grow quite large, but that’s the beauty of them—they help you identify numerous paths through the application that could harbor defects. Let’s consider an example of using a mind map to explore test cases for a restaurant delivery app. Such an app has to handle a huge variety of menu options, item combinations, customizations, payment options, delivery instructions, and so on. As a tester, you can map out flows to exercise things like: Editing or canceling an order, both with and without saving changes Handling payment failures and retries Supporting multiple payment options, including cash on delivery Relaying delivery details like “leave with the doorman” to the driver Advanced Exploratory Testing Techniques 1. Understanding Developer Blind Spots One key to effective exploratory testing is analyzing the strengths and weaknesses of the development team. Through working with developers over time, perceptive testers learn to recognize patterns in how each developer operates. Frequently, developers only test code on their local machine before handing it off to QA, due to time constraints. They may do minimal code review without actually running tests. In these cases, defects will be easy to spot right away, without requiring much creativity from the tester. However, some developers are more thorough, writing unit tests and running the application after merging code. When developers put in this extra effort, testers need to dig deeper and exercise more creativity to find defects, focusing on user flows and edge cases the developer likely didn’t consider. Developers may also introduce bugs by failing to implement all requirements listed in the user story. During an initial design review, testers can do some exploratory testing of the proposed workflow, uncovering conflicts or gaps before any code is written. Observing product owners is also important – if requirements are incomplete or the product owner lacks understanding of end-to-end flows, defects are sure to crop up. 2. Exploring End-to-End User Flows Creating exploratory testing stories that walk through realistic user flows from start to finish is an effective way to uncover workflow issues. For example, consider an application that allows patients to manage their prescriptions and interact with their medical team: The tester could start by logging in as a patient and verifying all their info displays correctly, including doctor assignments and medication details. They could then walk through the medication review process in various states, making sure appropriate alerts get triggered. Opening up integrated systems like lab results, the tester could ensure patient data stays consistent across all touchpoints. Searching, sorting and filtering data in different ways may uncover display issues. Throughout the process, the tester is role-playing as a real user, experimenting with different paths and observing how the system responds. This type of organic, curious exploration often leads to discovering defects that would be missed by scripted testing. Dissecting Backend Processes and Integrations Experienced testers learn to look beneath the surface of the application, investigating backend processes and third-party integrations that enable core functionality. Analyzing database connections, data feeds from APIs, and scheduled backend jobs can uncover weak points and failure scenarios. Email and SMS notifications triggered by user actions are common integration points. Testers should experiment with these flows, purposely trying to make them fail. Security settings and user access controls are also fertile ground for testing, especially if they involve external systems. Enabling application settings in conflicting combinations is another effective exploratory testing method. For example, in a medical app that enforces regulatory compliance, experimenting with contradictory configuration options could uncover serious defects that put patient safety at risk. Putting Yourself in the Customer’s Shoes It’s easy to get tunnel vision as a tester, focusing on granular user stories and losing sight of the big picture customer experience. Effective exploratory testing requires constantly putting yourself in the user’s shoes as you move through workflows. Testers should explore the application by role, concentrating only on what a doctor, nurse, patient or administrator experiences in their typical journey. Avoid falling into familiar paths – remember that users often deviate from expected workflows, getting interrupted, making mistakes, or using the app in unintended ways. For example, users may start a multi-step data entry process but need to abandon it partway through. Or they may accidentally hit the wrong sequence of keystrokes. Testers should mirror these real-world scenarios, discovering how forgiving the application is to human error. 3. Pushing the Application to Its Breaking Point Ultimately, the fearless exploratory tester seeks to break the application, pushing it to its limits. Testers intimately familiar with the application know its weak spots and points of brittleness. Exploiting these potential faults, within the limits of security access, is the goal. Trying to overload backend servers and databases with unexpected usage patterns and edge case data can uncover critical issues before they impact production. The best testers proactively look for ways to make things break rather than timidly avoiding such scenarios. Experienced testers know that if they can break something in the test environment, it’s very likely users can (and will) do the same in production. Proactively discovering and fixing these defects, however obscure, is the hallmark of comprehensive testing. By relentlessly exercising creativity and curiosity, testers can harden the application against the chaos of real-world usage. Conclusion While exploratory testing requires skill and experience to perform effectively, it pays dividends in uncovering costly issues before they reach users. When thoughtfully integrated with other testing techniques, exploratory testing helps teams efficiently deliver high-quality, delightful software. It provides an additional layer of validation on top of automated checks and unit tests. Exploratory testing excels at revealing problems in the user experience, cross-functional workflows, and edge-case scenarios that are hard to anticipate in advance. And it scales well by enabling the whole team, not just dedicated testers, to assess the application with a fresh perspective continuously. The best way to improve your exploratory testing skills is through practice and reflection. Pick an application you’re working on and spend 30 minutes freely exploring it. Try some of the techniques from this guide and see what you discover. Share your observations with a teammate and brainstorm ideas for what to try next. Make exploratory testing a regular habit and watch your defect detection abilities grow. You can also learn more about Testlio’s exploratory testing services and how they can help accelerate your team’s testing efforts. Testlio’s global community of vetted testers can quickly spin up exploratory testing cycles on top of your existing automated tests and in-house QA.