8 steps to building an effective QA test summary report for mobile applications

Lauren Gilmore , Dog owner, expat, gin lover. Allegedly wise to the ways of digital marketing, PR, and social media. Currently waging a war on mediocrity in communication and storytelling.

January 29th, 2020

While QA mobile app testing is of the utmost importance, how you report your findings is equally imperative. The entire end-to-end application testing process should be recorded so that stakeholders understand the project situation and product quality.

Armed with this final report, management can decide if your mobile app is ready to be released to the market.

What is a QA test summary report?

A QA test summary report provides details of any important information uncovered during any phase of testing.

With the prime objective of explaining various details and activities about the tests performed, the summary report includes assessments of the quality of the testing effort, the quality of the software system under test, and statistics derived from incident reports. 

It also allows project stakeholders to review testing efforts with respect to the test plan ensuring transparency and accountability throughout the organization.

What does a QA test report contain?

Though each test summary report is based on individual company format and practice, a typical test summary report template generally contains: 

• A summary of test activities and final test results
• An assessment of how well the testing is performed

While subjective, this includes: 

• Important information uncovered by the tests conducted
• Quality of testing effort
• Quality of the mobile application
• Statistics derived from incident reports
• Different types of testing performed and time taken for each of them

Based on the test report, stakeholders can: 

• Evaluate the quality of the tested product
• Decide on the software release. For example, if the test report informs that there are many defects remaining in the product, the stakeholder can delay the release until all the defects are fixed.

Eight steps to building an effective QA test summary report

1. Project information

All information on the project such as the project name, product name, and version should be described in the test report.

2. Purpose

Test reports provide a summary of the results of tests performed as outlined within the testing themselves. It shows to the best extent the testing environment, test results, and examples of bug reports.

3. Test objective and scope

The test summary report should include the objective of each round of testing, such as Unit Test, Performance Test, System Test… etc.

This section also explains the functions in scope and out of scope for testing. As well as any items that are not tested and why (constraints, dependencies, restrictions, etc).

4. Test summary

This section includes the summary of testing activity in general such as:

• The number of test cases executed, along with the execution results: pass/ fail / blocked
• The numbers of new issues found: highlight or list the most significant issues, especially if any issues are considered blocking issues
• Pass percentage
• Fail percentage
• High-level comments, including a discussion of blocking issues, testing gaps, and recommendations
• Instead of simply listing this information, make it visually apparent by using graphs and charts. Mentioning these metrics in this form is a good way to visually represent data. In addition to the graphics, provide a link for the reader to drill down into details.

5. Environment

Devices and software information your app is implemented on.

Android devices:

• Model
• OS version
• Processor architecture
• Resolution
• GPU
• RAM, Mb

iOS devices:

• Model
• OS version
• Resolution

6. Test results

List all functionalities that were checked and subsequent defects. 

• Details on test execution results, including links to issues for failed & blocked test cases
• Total number of bugs
• Status of bugs (open, closed, responding)
• Number of bugs open, resolved, closed
• Breakdown by severity and priority

7. Exit Criteria

Each test type requires distinct entry and exit criteria for each testing phase ensuring that testing strategy objectives and product requirements are met.

In this section, exit criteria should be clearly defined and it should be obvious that all planned test cases were executed and all critical defects were closed. 

Include an assessment on whether the product under test has met the exit criteria, being very specific for any areas of the criteria that were not met. 

8. Definitions, acronyms, and abbreviations

Some companies use their own definitions for certain words and abbreviate others that may not be general knowledge. Consider including an index of sorts for these so new team members or unfamiliar stakeholders understand things clearly. 

The QA test summary report is an important communication tool between QA managers and stakeholders and should explain activities performed as part of the QA testing in a way recipients can understand it.

A good qa test summary report should be: 

• Detailed: Provide a detailed description of the testing activity. Resist including abstract information or departmental-only verbiage as to not confuse readers.
• Clear: All information in the test report should be short and easily understandable.
• Standard: Test reports should follow a standard, consistent template making it easy for stakeholders to review.
• Specific: It’s called a “test summary report” for a reason… it’s not an essay. Focus on the main object and points.

Conclusion

Test summary reports are an important deliverable. After performing exhaustive testing and explaining results, stakeholders are given evidence of whether or not to move forward with the product as it is. This extremely important step in QA mobile app testing allows your company to be effective and efficient in such a demanding race to launch. 

Ready to test? Our dedicated testing professionals are here to help. Contact us today to schedule a demo.