10 QA Statistics Dayana Mayfield , Dayana is a QA engineer turned technology writer living in Milan, Italy. She's always down for a smoothie. March 2nd, 2017 Marketers and IT experts regularly produce fascinating research demonstrating their industry’s impact on the bottom line—but what about QA? QA is known to directly affect the customer experience, which will soon overshadow price and product as the key brand differentiator. With the growing sophistication and adoption of UX metrics, it’s becoming easier to measure the impact of product improvements and quality enhancements on user experience and engagement. Agile development—widely adopted over the last decade—allows QA to move earlier up in the development process. By prioritizing QA and including QA as early on as possible, teams can move quickly and still maintain quality (as opposed to leaving testing until the end). Now, the best teams in the world use DevOps practices to share knowledge and break down barriers between core business functions. Development, QA, and business aren’t silos anymore. Sales teams are often beta testers, and QA engineers are often given a seat at the table during the requirements phase. All in service of the customer. Everyone at every level is in charge of keeping customers happy. Here are ten stats that prove that QA teams are doing their part. Less than 25% of app users return after day one Retaining mobile app users is notoriously difficult. The majority of apps fail to inspire user loyalty. After the first day of use, just 25% of app users return on the second day and retention rates continue to drop steeply. Interestingly, Android apps have slightly higher retention rates than iOS apps across all verticals, according to Appboy. But the averages for both platforms are still dismal. So, how can product teams combat low retention? Overall, Appboy recommends improved onboarding campaigns, omnichannel engagement, and usage-based notification campaigns. All of these are prime opportunities for exploratory testing, which is a must for verifying quality in such complex customization efforts. Only 29% of mobile developers do exploratory testing And yet, many mobile developers still don’t have the resources for the exploratory method. That’s a shame since exploratory testing (in which testers cover specific use cases but without tightly written scripts) provides a lot of insight into true user behavior. Only 29% of mobile developers actually do exploratory testing, according to Forrester. Shockingly, Forrester found that QA is not a priority in mobile development for the majority of teams. Lack of mobile application testing resources (both human and device) keeps too many products from focusing on quality, and they fall to the bottom of the pack. The average app loses 95% of users after 90 days The retention difference between top-performing apps and the rest of the pack is stark. The average app loses over 95% of new users after 90 days, according to Quettra. Even the top 10 apps on Google Play lose just under 50% of users over three months. The key to success is to get the users hooked during that critical first 3-7 day period. Ankit Jain, CEO of Quettra on the retention disparity By relying on testers who can think and act like users, product teams can nail those critical first impressions and test the accuracy of personalized onboarding campaigns. Android app testing and iOS app testing are crucial to success. High-performing product teams have 200x more frequent code deployments So what sets those top performing apps apart? Frequent releases. Continuous development is THE answer to meeting customer expectations. The more successful an app is, the more often it’s updated. Etsy has reported deploying 80 times per day, while (between its various products) Amazon can deploy thousands of times per day. Puppet’s 2016 State of DevOps Report finds that high-performing product teams outperform the low-performers by an insane amount: we’re talking 200x the frequency of deployments. With deployments that fast and frequent, the old adage that testing makes it impossible to maintain both speed and quality is completely blown out of the water. Continuous QA keeps up with continuous development as more and more organizations “shift left.” When you shift left, instead of testing quality only at the end, you have multiple feedback loops along the way to ensure that software gets delivered to users quickly, at a high level of quality. Puppet High-performing product teams spend 22% less time fixing issues Puppet also found that teams who integrate testing as a partnership with the entire development process spend 22% less time on unplanned work. Dev teams who build quality in and work with QA early on have more time for new, value-adding work. Of course there will always be old issues to fix. But teams who have been successful at the shift-left approach to QA are able to devote 49% of their time to developing new features that will delight customers, as opposed to other teams who can only devote 38% of their time to new work due to a more traditional approach to QA. Top-reviewed iOS apps are just 28 days old on average There’s one area where a commitment to QA is commonly known to always pay off: app store reviews. No product team wants issues to show up in app store reviews. Critiques of the product? Maybe. But bugs that could have been fixed? Absolutely not. Continuous deployment and continuous QA correlate directly with high app store reviews. The top 100 apps on Google Play and the App Store are updated more frequently, while the lowest rated apps are updated infrequently. Frequent releases are also correlated with less volatility in reviews, meaning that quality and customer perception remain consistent. 67% of consumers cite bad experiences as reason for churn People ignore the apps that are on their phone every single day. How many of your apps do you actually use? Probably not very many. People either get the app or they just completely forget. Andrew Chen Simply forgetting an unremarkable app is a fairly neutral reason for churn. But it isn’t always safe to assume that churn is due to… The user no longer needing the product The user being too busy The user forgetting the product Often times, churn is the direct result of a bad experience. Every industry has different methods for avoiding bad customer experiences, and when it comes to tech, the answer lies largely with QA. Only 4% of unhappy customers complain It’s common to worry about what customers say. But what about what they don’t say? Considering that only 4% of customers complain directly to a company about a poor experience, there’s a lot more negative feedback than what can be found in app store reviews and social media posts. Exploratory testing helps to not only keep negative feedback out of public channels but to discover what that unspoken feedback might possibly be. Testers who can think like users can reveal issues and areas for improvement that would have otherwise caused more silent churning. The QA budget should be as much as 25% of development For a few years now, the World Quality Report has recommended that companies keep the QA budget to 25% of development costs. That universal recommendation comes from an understanding of what’s “reasonable” to maintain quality, security, and improve the customer experience while not bogging down the potential for new development. Actual QA budgets of large enterprises rose from 18% to 35% over a period of four years, and then dropped to 31% in 2016. IT teams surveyed collectively predict that their QA budgets will soar to 40% of development costs by 2019. While that does reveal that enterprises recognize the value of QA (and are ramping up QA efforts throughout development cycles), the truth is that most startups don’t come close to spending 25% of development on QA and need to gradually divert more resources to quality as growth allows. 52% of IT teams site higher amounts of releases as the reason for higher QA budgets So what accounts for that higher predicted spend on QA? Well, keeping customers happy requires continuous development. And increased digital innovation requires increased test cycles. As more teams adopt a DevOps approach, there will be a higher frequency of releases, and a greater need for continuous, seamless QA. While some industry leaders have suspected that DevOps adoption would disband QA, it’s actually creating more synergy and more dependence when it comes to meeting deadlines. Still, the World Quality Report recommends that product teams keep QA budgets to 25% by extending their in-house capabilities, automating where possible, and bringing in the right testing tools. Testlio provides custom QA testing solutions to top teams. To learn more about customer-centric approach to quality, get in touch with us.