The quality industry has changed a lot in the six years I’ve worked with mobile engineering managers and VPs.
I went from telling mobile engineering executives why they needed to monitor user crashes in a live environment, to stressing the importance of automated testing so they can monitor crashes in a pre-production environment, to finding a balance of the two (at Testlio) by educating VPs on the importance of testing pre-release with both automation and manual strategies.
Quality is a boardroom issue
In many ways, my career over the last 5-6 years has converged in a similar way to the quality industry as a whole. Companies are trying to move faster and faster to keep up with their mobile users and like in any high-growth environment, pain points and gaps are surfaced. So while they continued this fast-paced progression, attention from the executive level, as well as budget and responsibility, has been poured onto quality. We’ve seen a huge uptick year-over-year in how much money companies are spending on testing. Today, testing and quality makeup 26% of the average IT budget and will grow 32% of IT spend by 2020, according to the World Quality report.
Every day we see mobile development teams that, solid as they are, stumble by moving quicker than they’re capable of. How do they recover and prevent future defects while increasing production speed?
Avoiding the mobile graveyard
Users are also changing. They’re becoming more and more demanding, they expect everything to work and work well, and they want it now. Users are not as forgiving as an internal team member because they aren’t sitting next to the product and design teams to understand the intent of what your company is trying to build. They either like it or they don’t, and they’re not afraid to tell you about it.
It’s important now, more than ever, to not only care about quality and maintain it but move fast. And the importance of speed only seems to increase over time. Which means a focus on quality has to increase as well because one without the other can be disastrous.
The founder and CTO of Appurify (now Google’s Firebase Test Lab) built their platform to warn developers about what’s still in their code and when they’re moving too quickly. It was so clear to him that product teams still did not have a way to properly identify the bugs that truly mattered before getting an app into the hands of their users. He was previously CTO at Zynga from 2011-2012 and focused on releasing Mafia Wars 2. When Zynga shut down the game some asked whether it was too complicated or failed to gain critical mass, but the real cause of the app’s decline was because of a single bug that forced the app to crash during the demo screen on a specific device (iPhone) on a specific network (AT&T).
There were no tools for Zynga to find this specific issue. There were a number of platforms and automation tools out at the time, but nothing that had the ability to surface a defect under such unique conditions. The CTO and his entire team spent 3-4 months post-launch tirelessly searching for the issue. A single bug effectively wasted nine months of development time and millions of dollars. Zynga had no other choice but to shut the game down.
The risk of overextending QA
I’ve worked closely with central QA services within large corporations whose role is to serve anywhere between four and twelve business units. These organizations learned quickly that having one central team with such a fluctuation of asks and many competing priorities wasn’t going to scale. Seemingly overnight, each business unit built out their own dedicated QA team to support development. It was clear that QA needed to be closer to engineering (even if some still felt they should be kept separate).
Because of this switch, QA teams took on more responsibilities. Instead of simply scratching the surface of testing for a dozen products, they were empowered to dive deep into one or two products and get to the core of any issues far before their users would.
As quality continues to top the checklist of testing priorities, having the resources to test swiftly without missing any pre-release bugs is more important than ever.
Quality isn’t a “nice to have” anymore. It’s the difference between becoming a household name and getting lost among the millions of similar apps that users can choose from.