Build vs. buy? A startup’s case for outsourcing QA

So your AWS bill is on fire and suddenly your all-hands takes up multiple screens on Zoom. Users are doubling. Your investors are thrilled to see your startup hitting its stride. 

But your rapid growth has led to a couple of scary-looking bugs in production. Users in new locations and on previously unseen devices are popping up in your activity log. Your unit tests aren’t cutting it anymore, and an untimely bug puts your growing user base at risk.

Is it time to post a job listing for your first full-time QA hire? Or give your small QA team that extra headcount they’ve been asking for? One alternative is to partner with a crowd-testing vendor. And for many companies like you, outsourcing QA is a great choice – economically and for coverage, quality, and speed.

Enterprise coverage without being an enterprise

While your team is small, your user base is in the thousands. You work from one location, they’re all over the world. Your team uses the iPhone 11, but users favor the Galaxy S22. The industry leader you’re up against doesn’t have this problem – they have global headquarters with fully-stocked device labs. 

It’s that exact challenge that Vinovest, a fine wine fintech startup, faced after a series A round and rapid development.

“For each of our employees, there were only so many devices. Plus our team was based in LA and we had customers in China and Europe. We would need a huge company to sustain coverage of everything, and even still, was it worth it?” said Conrad Kurth, Director of Engineering when making the buy vs. build decision. 

Kurth eventually made favored outsourcing QA to a crowd testing partner to cover the locations and devices of his user base and provide enterprise coverage without having an enterprise scale or budget.

Vinovest brought in Testlio as a full-service QA partner with expert testers across the globe. Read the case study, here.

The impact on customer experience has been dramatic. In the first quarter of working together, Testlio reported 191 bugs ranging from broken sign-up flows to small (yet critical) details like fluctuating exchange rates. Since partnering with Testlio, Vinovest has launched in China and released native mobile apps, all at the level of quality their customers expect.

Conrad calculated that Testlio’s comprehensive solution with managers, testers, and software saves Vinovest over $300,000 annually compared to building their own team, with a breadth of locations and devices covered that would have been impossible to staff in-house.

Keep developers coding

Speed is paramount when you’re trying to disrupt an entire industry, one two-week sprint at a time. An in-house testing team can be slow and relying on developers to split their time between coding and QA lacks efficiency. As telehealth platform Bright Health’s feature set expanded, testing pulled developers away from what they love – writing code.

“With sprint planning, demo, grooming, and retro, our developers are already losing one day. That’s 10% of a typical two week sprint. Adding a day of regressions across many different devices and another full day to fix bugs and prepare the release means we lose 30% of a sprint to this type of overhead,” said Chad Pavliska, Bright Health Senior Director and Mobile Architect.

Every hour of dev time matters for on-time releases, so tackling the growing test suite under a developer-as-QA approach was not sustainable.

Bright Health decided on outsourcing QA with Testlio to free up dev time to accelerate development without impacting quality. Removing operational overhead means more energy for other areas like development and product design. After one month of working with Testlio, Bright Health’s developers started writing 10% more code.

Not only are engaged developers more productive, they’re happier.

Bright Health has maintained high levels of employee satisfaction while growing exponentially, reinforced by earning a spot on Forbes’ America’s Best Startup Employers list for 2021. With a nod to Chad’s leadership, his team of highly experienced mobile application engineers has a 98% developer satisfaction rating and zero turnover.

Make the most of your latest funding round

Last but not least, your Series A isn’t going to last forever. As a startup, you have to make every dollar count. A major factor in the build vs. buy decision is the huge cost of standing up an internal QA team.

From our whitepaper on the ROI of crowd testing, our research uncovered that crowd-testing clients save 30-45% on QA compared to in-house operations. That doesn’t include the cost of maintaining a device lab, recruiting employees, or building internal QA systems and processes. 

“For a startup, the overhead of QA is very high. Testlio provides everything we need and more, without the operational expenses, which helps us scale,” Kurth said.

Operational constraints can feel like handcuffs when you’re trying to build the plane and fly it. Every function, whether that’s QA or customer success, requires operational bandwidth to spin up and scale. Crowdtesting means one less operation to get off the ground and maintain as your company hits hypergrowth.

Greater coverage for devices and locations. An unbiased look at your product. Accelerated testing velocity. Zero operational overhead. Cost savings. A crowd-testing vendor (maybe Testlio, the #1 crowd testing vendor on G2) can help achieve all of the with a clear path to high satisfaction and ROI.