Thrive360 Case Study

“Testlio is practically indispensable. I’ve been especially impressed with how easy it is to get organized and actionable information back. I trust that once I send off a build or a test run, that’s it, QA is taken care of. I never have to wait on Testlio, and that is a huge relief on my side.” 

John Best, CTO

Thrive360 provides mental and emotional support for people across the globe. Provided by businesses to employees as a wellness benefit, their offerings include courses and programs that promote emotional skills like mindfulness and change management. 

With six different apps across multiple platforms like mobile, web, and virtual reality, Testlio works directly with Thrive360’s CTO John Best to ensure quality for their weekly releases.

The Challenge

QA that could keep up with weekly releases across multiples apps, locations and devices

Thrive360 faced a common startup challenge. They had a small but growing engineering team supporting a global customer base and an ambitious release cadence. While John had considered having engineers run QA themselves, he realized that each week’s run would eat up an entire day’s time from half of his team – time that they don’t have. 

Dedicated QA was needed, but bringing it in-house would divert resources and slow down hiring engineers. Additionally, Thrive360 needed to test six different apps, used around the globe and spanning device types from mobile phones to VR headsets. They were constantly releasing new content from different providers, and it all needed to work perfectly for customers. A US-based engineering team without a device lab wasn’t equipped to surface issues for so many different use cases.

Missing coverage and delayed releases wasn’t an option — app quality and user experience is paramount for Thrive360. They needed expansive coverage, fast, without depleting their budget to get it.

The solution

Testlio deploys a large, burstable, cost-effective networked testing team

With Testlio, testing at Thrive360 is a dependable and painless part of the software development lifecycle instead of a UX liability, bottleneck, or budget-breaker.

Currently, Testlio provides a team of 19 testers in 8 locations across 24 device/OS combinations that represent Thrive360’s customers. The on-demand testing team flexes in and out of assignments to accelerate testing results. Instead of three people spending eight hours testing, Testlio’s burstable team breaks the assignment into a greater number of two-hour tasks. According to John, Testlio provides results three times faster with this model, allowing Thrive360 to keep up with weekly releases. 

“QA velocity is really fast, plus with a variety of people testing I also know it is really thorough,” John says. “Quality would suffer if we didn’t have this in place and that would absolutely impact our bottom line.”

Testlio’s model is also significantly more cost-effective than hiring a global testing team and equipping them with all of the necessary devices. Networked testing allows a growing startup like Thrive360 to get enterprise-scale coverage and confidence without an enterprise-scale budget.

The outcome

Global testing coverage and accelerated results that don’t break the bank

• 80 new issues discovered last quarter by testers in 8 countries with 24 device/OS combinations
• 3x faster testing velocity via a large testing team breaking assignments into two-hour sprints
• 40% savings compared to in-house with a global network of testers and baked-in device costs

“If I could give one piece of advice to a CTO in my position, it would be to hire external testing resources. Seriously. I always thought that QA testing was something engineers did themselves — I test my code rigorously and expect others to do that as well. Working with Testlio has made me see that there’s a much stronger way of doing it. I cannot foresee a time within the next couple years where it would make sense to switch to an internal QA team.”

John Best, CTO