Strava Case Study

“Testlio’s team better approximates how real users expect our product to work. Their combination of experienced testers and ability to meet tight deadlines has made them invaluable to our success.” 

Benjamin Wolak, Mobile QA Engineer

Strava was developed by two former rowers who wanted to recreate – on a massive scale – the camaraderie and competition they experienced as teammates on the Harvard crew team. Today, Strava is the leading social platform for athletes and the largest sports community in the world. With over 98 million athletes in more than 195 countries, Strava’s mobile apps and website connect millions of active people every day, giving them simple and fun ways to stay motivated and compete without having to be in the same place at the same time.

THE CHALLENGE

Strava needed to accelerate releases while maintaining a high bar for quality

When Strava first partnered with Testlio, they were a rapidly growing startup releasing only a handful of times per year and struggling to ensure quality in new markets. 

They had ambitions of rapid-fire biweekly releases and global expansion but didn’t have enough QA resources to execute at that level. As development accelerated, testing was becoming a bottleneck. Their speed of innovation was slowing down at a make-or-break moment in company history.

“Scaling testing in-house can be difficult and time consuming,” said Benjamin Wolak, a Mobile QA Engineer at Strava. “It is easy for us, as internal testers, to apply the inside knowledge that we have to justify functionality that a user would perceive as either broken or confusing.” 

In addition to functional biases, their internal testers were unable to identify localization issues like translation errors and problems with new character sets in target markets.

The solution

Strava chooses Testlio to accelerate testing and ensure quality in new locations

Strava needed a testing partner that could make their ambitions of confident, rapid global releases a reality. They were looking for a vendor with a global footprint and the ability to move fast without compromising on a high-performing app experience for their high-performing athletes. To keep pace with their accelerated release cadence and commitment to delivering a great app around the world, Strava chose Testlio. 

In the first week of each two-week cycle, Strava provides Testlio with a build by Wednesday and Testlio reports results for Strava the next day. Testlio achieves these rapid turnaround times with burstable testing, where a team of 20 expert testers swarm the app in two-hour windows. 

“The best part is the flexibility,” said Benjamin. “Testlio lets us add more testers or increase the number of testing cycles whenever we want. We have access to a much larger pool of testers and perspectives than we could ever employ internally.”

Strava also leverages Testlio for testing in non-Latin languages. As needed, Strava asks Testlio to place an emphasis on different character sets and keyboards, or give extra attention to an area of QA that is especially challenging to handle internally.

The outcome

Strava releases six times faster while catching critical issues

Together, Strava and Testlio have maintained a world-class experience for Strava’s users while releasing six times more often. 

Reaching beyond internal biases and bringing in testers who could see Strava with fresh eyes as users do has meant fewer crashes and rollbacks. Benjamin recounted, “In one case, a tester found a scenario that we missed involving the camera position. There’s a good chance Strava would have done an emergency dot-release for the issue if it hadn’t been caught before it reached users.”  

Additionally, Testlio’s global testers consistently catch localized issues before they become UX liabilities. For example, Strava leaned on Testlio when their new mentions feature needed to perform with non-Latin character sets and keyboards. 

“A tester in China found a major issue where multiple mentions could be created,” he said. “This was especially important because, although it was a major usability issue for our Chinese users, we don’t get many non-English speakers contacting our support team. It might have impacted a lot of users before it was reported.” 

As for speed, Benjamin can count on Testlio for reliable, fast testing that enables a dependable release schedule and continuous innovation. “High-quality QA is essential to on-time releases. Testlio ensures that we get detailed, rapid feedback early in our release cycles.” 

Since partnering with Testlio in 2016, Strava has grown from 20 million athletes to nearly 100 million and expanded into countless new markets, while maintaining a 4.8 star app store rating. In a competitive fitness market, Strava can’t afford to slow down or sacrifice quality. And with Testlio, they don’t have to.

“Testlio knows how important speed is to our growth. I love Testlio’s flexibility and willingness to accommodate our needs. When I send Testlio a release, I know it will be ready.”

Benjamin Wolak