Bright Health Case Study

“We think of Testlio as part of our team. They’re integrated into our processes. And it pays dividends in less friction, cost, and anxiety.” 

Brian Gambs, CTO

Bright Health Group (NYSE: BHG) is an integrated healthcare company leveraging technology to bring together financing and delivery of care. Their healthcare platform includes a suite of consumer and provider solutions, including DocSquad, a two-sided telehealth marketplace with a mix of virtual modalities including real-time chat, audio, and video.

THE CHALLENGE

As Bright Health’s feature set expanded, testing pulled developers away from what they love – writing code

Faced with a tight labor market, Bright Health Senior Director and Mobile Architect Chad Pavliska entered the second half of 2020 with an ambitious directive – recruit a highly experienced mobile dev team and launch a mobile-first telehealth application in a year or less. 

Because healthcare is not immune to the global shortage of skilled software developers, the pressure to build and release a product makes recruiting and retaining engineers a top priority. 

Energized by the challenge, Chad tapped into his network to recruit a team of 14 experienced Flutter mobile software engineers. 

Months before going to production, the DocSquad engineers performed complete regression tests on their code. Unfortunately, as the feature set expanded, managing regression each sprint meant losing valuable time to write code. “With sprint planning, demo, grooming, and retro, our developers are already losing one day. That’s 10% of a typical two week sprint. Adding a day of regressions across many different devices and another full day to fix bugs and prepare the release means we lose 30% of a sprint to this type of overhead,” said Chad.

Getting to feature-complete can consume much of a sprint. Every hour of dev time matters for on-time releases, so tackling the growing test suite under a developer-as-QA approach was not sustainable. Bright Health needed a way to free up dev time to accelerate development without impacting quality.

The solution

Bright Health leveraged Testlio’s managed services to scale testing and coverage

Bright Health CTO Brian Gambs had previously worked with Testlio at another company and asked Chad if he’d like the extra help for DocSquad. Chad immediately saw the opportunity to share the QA burden with Testlio so his developers could do what they love – write code and build features.

“In a competitive job market, you can’t afford to lose people,” said Chad. “A happy developer is a productive developer.”

Working with Testlio also meant that Chad could focus on his strengths without the added weight of building and managing an in-house QA team. “Recruiting is hard, and there was no guarantee that we’d find the right individual to set up all the QA processes, write tests, and conduct manual QA. That’s a pretty senior-level role, and good luck finding that in this market,” he said.

Testlio supports Bright Health with a managed testing services team familiar with HIPAA and PHI compliance, plus a pool of over 40 on-demand crowdsourced testers. While the DocSquad dev team adds unit tests within their build process, they no longer take on the weight of full regression testing. 

Types of testing and coverage

• Full regression testing each sprint with complete device/OS/platform coverage
• Ad hoc testing for big feature test runs before shipping
• Localization testing with native Spanish speakers to improve the patient experience
• Virtual experience testing for two-sided modalities, including real-time chat, audio, and video

“Having a different set of eyes running regressions on various platforms is helpful,” said Chad. “The fact that the Testlio Network can keep opposite hours and do overnight testing means we can be code-complete by 5 p.m., send the builds to Testlio, and have a full regression report the next morning.”

The outcome

Bright Health increased developer efficiency and feature-readiness at 2x ROI with Testlio

Releasing a new telehealth platform takes discipline, effective systems, and peak employee performance. To give developers more time to write code and get their product to market on time, Bright Health shared the testing responsibility and accountability for quality with Testlio. The partnership economics were significantly more favorable than building a device lab, subscribing to test management tools, and conducting in-house QA.

In a one-month time-to-value, devs started writing 10% more code. This extra output helped them advance to feature complete a full day each sprint. It also helped maintain on-time production-ready releases every two weeks for an entire year. 

Engaged employees are more likely to produce high-quality products and less likely to seek new employment. Bright Health has maintained high levels of employee satisfaction while growing exponentially, reinforced by earning a spot on Forbes’ America’s Best Startup Employers list for 2021. With a nod to Chad’s leadership, his team of highly experienced mobile application engineers has a 98% developer satisfaction rating and zero turnover. The ultimate reward for the DocSquad team is a 4.9 end-user App Store rating and rapid patient and clinician adoption.

As the DocSquad platform matures, a decline in overall bugs will enable them to fix high-priority issues quickly and further optimize their testing processes through efficient manual and automated testing.

“Testlio has helped us buy back 10% of our development time. This means more story points per sprint and functionality to the business. We literally move feature-complete forward one day each sprint.” 

Chad Pavliska