Request info

Differences Between Sanity And Smoke Testing

Sanity Vs Smoke Testing

The main differences between Sanity and Smoke testing help decide when each type of test should be performed.

Knowing when to apply each type of testing enables companies to plan their testing strategies smartly and save precious hours of testing time.

What Is Sanity Testing?

Sanity Testing is carried out when a small change such as a bug fix has been made, to ensure that everything, including the fix, is working as it should.

What Is Smoke Testing?

Smoke Testing is done when a larger chunk of code is changed, such as a new feature development, to check if the main workflows of the application are working as expected.

Related: Customer-focused testing strategies that don’t break the bank

Difference between sanity and smoke testing

Breadth of Testing Scope vs. Depth of Testing

Sanity Testing is done for a small coverage area – a narrower scope, but it dives into the main areas of the scope and surrounding functionality. Suppose your software application has a 3-step registration process that could be initiated from its native apps on iOS, Android, Desktop, Web Browser as well as the website. A bug in step 2 of the process, once it’s verified to be fixed on all the platforms, would require sanity testing of the registration process and its dependent flows. Usually, this is done across 2-3 most used application platforms. Sanity testing of this sort generally follows a wider-scale regression.

Smoke Testing, on the other hand, covers a wider range of functionality, usually right after a feature is developed. It checks the main flows of the application that lead up to the newly developed functionality and the main acceptance criteria of the developed feature. Working with the previous example, if a 4th step was added to signup, the first time the testing team gets the build, they would test the complete registration flow across all application platforms. Smoke tests also are run to “pre-qualify” the app for other tests, the basic flows need to work before committing to a more extensive suite of tests.  

Regression Testing vs Acceptance Testing

Smoke testing is a subset of Acceptance Testing. The Smoke test verifies that the prerequisites of the feature test cases can be achieved in the software build that has been provided by development, and then moves ahead to verify the main acceptance criteria of the feature. Usually, when the test cases of a feature are designed, certain test cases related to major functionality and high priority acceptance criteria are marked as Smoke tests. 

Related: 8 steps to implement a manual regression testing strategy

Release Decision vs. Test Decision

Smoke Testing generally determines whether the build is ready to be taken up by the testing team for more detailed feature testing. Smoke tests failing indicate several items are not working. This, in turn, means that other features that haven’t been run through a Smoke test may also be broken if the basic ones aren’t working the way they should. If the product cannot pass the smoke test, it’s generally not worth testing further – the build needs to go back to development for analysis and triage.

Stable Build vs. Unstable Build

Sanity Testing is done over relatively stable builds. This could be a bug fix for a build that is already in production, or a few fixes on a new feature build going into production, a build which by this time would have gone through several rounds of feature acceptance testing and regression testing. If a social media app has a bug when uploading images, the app overall is stable because it is in production with an active user base. When the bug is fixed, the sanity tests around the fix will look for stability and outcomes that were in the production app before the bug fix.

Smoke Testing, on the other hand, is performed on all builds, whether or not major code changes have been done. This could be a minor optimization or a whole new feature. It’s usually the first build handed over to the testing team, though testing teams may perform Smoke tests on the first few builds they receive from development teams after initial testing cycles, based on build quality. As such, it is always good to time Smoke Tests to complete before the development team moves onto the next chunk of development e.g. overnight or over the weekend.

Scripted vs. Unscripted

Scripted sanity testing can be done for the login process of an application, both for the UI and the API, only when the features are working the way they are supposed to whereas the smoke test will be used to identify whether the feature is ready for further testing or not.