Speakeasy Co. Case Study

How Speakeasy Co. quadrupled their release cadence and increased quality by partnering with Testlio

3X reduction

in average time to live for client sites

56

new devices covered

1 month

time to value

“Testlio is far easier to work with than other testing vendors. It took only a month to see the value of the partnership, and now we release five sites per week where we used to release five sites per month.”

Ryan Mitchell, Director of Program Management, Speakeasy Co.

Speakeasy Co. is an e-commerce and logistics platform that allows beverage alcohol brands to sell directly to consumers from their own website without disrupting the three-tier system.

Company stage

Growth Startup

Industry

Commerce and Retail

Products

Web and Mobile Web

The challenge

Speakeasy Co. was struggling to keep up with QA as demand hit all-time highs

The pandemic has amplified consumer demand for alcohol. However, it has also put pressure on distilleries to find new ways to reach their buyers and sell alcohol and merchandise directly from their website while remaining compliant. Speakeasy Co. grew rapidly during this time but found that internal testing limitations were a bottleneck to meeting demand with quality.

“We were trying to keep up with a massive increase in demand with a small team, so our testing was quick and dirty. I would check things on one device and browser to make sure that the basic nuts and bolts were there, but we weren’t able to do deep dives, and things were getting missed,” Recalls Ryan Mitchell, Director of Program Management at Speakeasy Co. 

Because Speakeasy serves as customer support and an eCommerce platform developer for their clients, they would hear reports directly from end-users when issues arise. Reproducing and fixing these issues on live sites was creating extra work to find the root cause, taking time away from development, and proving to be very difficult with Speakeasy’s small yet agile internal team as they attempted to grow their business and get new sites out the door.

Speakeasy needed to change if they were going to keep up with tremendous demand and growth. Ryan, a former QA professional, knew that delivering on their promises to clients started with increasing the velocity and quality of their testing operation.

Speakeasy Co. is an e-commerce and logistics platform that allows beverage alcohol brands to sell directly to consumers from their own website without disrupting the three-tier system.

THE SOLUTION

Speakeasy brought in Testlio for fast, flexible e-commerce software testing

Devices

56

locations

9

expert testers

20+

Test types

Exploratory
Regression
Real device

Ryan was looking for three things in a testing solution that would get sites out the door faster: fast onboarding, flexibility, and quick feedback. He was wary of making an internal hire due to the time commitment of recruiting and onboarding experienced testing professionals. However, he also had previous experience working with an inflexible testing vendor that required that he match internal processes to their QA templates. 

With Testlio, he found a partner who could spin up testing faster than an internal hire and match their process to his needs while accelerating test cycles.

“Ease of onboarding was huge. If we don’t have time to hire, we definitely don’t have time to build a new testing process to match a vendor’s templates. From day one, our Testlio Engagement Manager made it clear that all they needed to ramp up was a look at our product. We sent them a simple spreadsheet of test cases, and they took it and ran with it.”

This flexibility carried over from onboarding into Speakeasy’s ad-hoc bug reproduction. Instead of sending five paragraphs of detailed test steps that matched a vendor’s template, Ryan could point to a page where end-users were reporting issues and Testlio would investigate and reproduce the issue on multiple devices without handholding. 

To reduce testing bottlenecks and keep up with Speakeasy’s project plan, Testlio’s global testing team conducted runs overnight. 

“Sometimes I’ll get out of the shower in the morning and my phone will buzz, and I’ll see a Slack message from Testlio. I have the test results before I even sit down in my chair to start work,” says Ryan. 

THE OUTCOMES

Speakeasy quadrupled the number of sites launched per month and gained confidence for future growth

Within one month, Speakeasy started reaping the benefits of their Testlio partnership and saw that they were able to get a client’s site developed, tested, and released confidently 3x faster. 

“We went from releasing five sites per month to five per week. And that happened pretty dang quick.” Says Ryan. 

Ryan saves 15-30 minutes per issue or test by sending Testlio links to sites instead of detailed instructions like he had done with previous vendors.

In addition to time savings on testing, Speakeasy receives more coverage, testing on 56 device/ OS combinations compared to Ryan’s one device and one browser, and validating the configuration of sites – shipping rates, taxes, etc. – that they weren’t checking previously. This increase in coverage means fewer issues in production. It allows Speakeasy to focus on getting high-quality, revenue-driving sites out of the door instead of fielding, reproducing, and fixing end-user complaints.

Testlio has played a role in major wins for the Speakeasy business.

“For one brand, we were expecting a couple of tens of thousands dollars in sales and did $600,000 of sales in one day. Having the site tested and turned around quickly certainly helped our sales. For another site, we went from concept to live in four days with zero issues in production. That would never have happened without Testlio.”

With Testlio, Speakeasy has a testing infrastructure that equips them for future growth and the confidence to expand upmarket. 

“It’s a different game when you’re going after the leading brands in the alcohol industry. Testlio has given us, as a startup, confidence that we can go after those brands and deliver a high-quality product.”

Testlio allows us to focus on what matters – growing our business. We not only test faster, but we release with less issues which means we spend less time going back and fixing issues.

Ryan Mitchell, Director of Program Management, Speakeasy Co.

Case Studies

LGND Case Study

LGND Case Study

LGND.art cut testing time from 20 hours to 15 minutes and built a true CI/CD pipeline. Learn how they did it in this case study.

Quizlet Case Study

Quizlet Case Study

Quizlet needed a new crowd testing partner they could trust. After switching to Testlio, they found more P1 issues with zero release delays.

Strava Case Study

Strava Case Study

How Strava keeps app quality in first place and accelerated their release cadence by 6x with Testlio.