What is Sanity Testing and how is it different from Smoke Testing? Sanity Testing and Smoke Testing are widely used terms in software testing. Sanity Testing is carried out when a small change such as a bug fix has been made, to ensure that everything, including the fix, is working as it should. Smoke Testing is done when a larger chunk of code is changed, such as a new feature development, to check if the main workflows of the application are working as expected. Knowing when to apply each type of testing enables companies to plan their testing strategies smartly and save precious hours of testing time. The main differences between Sanity and Smoke testing help decide when each type of test should be performed. Related: Customer-focused testing strategies that don’t break the bank Breadth of Testing Scope vs. Depth of Testing Sanity Testing is done for a small coverage area – a narrower scope, but it dives into the main areas of the scope and surrounding functionality. Suppose your software application has a 3-step registration process that could be initiated from its native apps on iOS, Android, Desktop, Web Browser as well as the website. A bug in step 2 of the process, once it’s verified to be fixed on all the platforms, would require sanity testing of the registration process and its dependent flows. Usually, this is done across 2-3 most used application platforms. Sanity testing of this sort generally follows a wider-scale regression. Smoke Testing, on the other hand, covers a wider range of functionality, usually right after a feature is developed. It checks the main flows of the application that lead up to the newly developed functionality and the main acceptance criteria of the developed feature. Working with the previous example, if a 4th step was added to signup, the first time the testing team gets the build, they would test the complete registration flow across all application platforms. Smoke tests also are run to “pre-qualify” the app for other tests, the basic flows need to work before committing to a more extensive suite of tests. Regression Testing vs Acceptance Testing Sanity testing is a subset of Regression Testing. It is generally done when running an entire Regression testing cycle over a small code change or bug fix would not make sense. But the test cases run during Sanity testing are subsets of the Regression test cases of the functionality in scope. For example, if the icons on a mobile app screen are moved, as part of a UX improvement, their associated functionality will be tested, but the icons of all other application screens would not need to be tested. Smoke testing is a subset of Acceptance Testing. The Smoke test verifies that the prerequisites of the feature test cases can be achieved in the software build that has been provided by development, and then moves ahead to verify the main acceptance criteria of the feature. Usually, when the test cases of a feature are designed, certain test cases related to major functionality and high priority acceptance criteria are marked as Smoke tests. Related: 8 steps to implement a manual regression testing strategy Release Decision vs. Test Decision Sanity Testing generally determines whether the build is ready to be released to clients or to production. Failing a sanity test means the fixes in the build are not working as expected or breaking surrounding functionality, so releasing the build would not solve the intended problem. In fact, it may result in a release-revert, where it might just be better to stop the release and send the build back to development. Say you have an e-commerce app with a product gallery view. The code has been optimized and the gallery view will now need a sanity test run for the areas surrounding it e.g. product detail view, interactions on the product detail view, etc. Smoke Testing generally determines whether the build is ready to be taken up by the testing team for more detailed feature testing. Smoke tests failing indicate several items are not working. This, in turn, means that other features that haven’t been run through a Smoke test may also be broken if the basic ones aren’t working the way they should. If the product cannot pass the smoke test, it’s generally not worth testing further – the build needs to go back to development for analysis and triage. Stable Build vs. Unstable Build Sanity Testing is done over relatively stable builds. This could be a bug fix for a build that is already in production, or a few fixes on a new feature build going into production, a build which by this time would have gone through several rounds of feature acceptance testing and regression testing. If a social media app has a bug when uploading images, the app overall is stable because it is in production with an active user base. When the bug is fixed, the sanity tests around the fix will look for stability and outcomes that were in the production app before the bug fix. Smoke Testing, on the other hand, is performed on all builds, whether or not major code changes have been done. This could be a minor optimization or a whole new feature. It’s usually the first build handed over to the testing team, though testing teams may perform Smoke tests on the first few builds they receive from development teams after initial testing cycles, based on build quality. As such, it is always good to time Smoke Tests to complete before the development team moves onto the next chunk of development e.g. overnight or over the weekend. Scripted vs. Unscripted Sanity tests are more often scripted and automated because they are derived from existing Regression tests. Automating Sanity tests can save time and provide faster results for the development team to take action on. Smoke tests are also scripted and automated, but since they target larger changes in code, there may be more maintenance required to keep the Smoke tests up to date. Automation scripts are pieces of code in themselves and need to be verified before they can be deployed over builds. So before automation scripts can be run, the builds must be stable. Scripted sanity testing can be done for the login process of an application, both for the UI and the API, only when the features are working the way they are supposed to whereas the smoke test will be used to identify whether the feature is ready for further testing or not. For testing that gets your product ready for release on schedule, contact us today for a demo.