Leading Furniture and Home Goods Retailer Case Study

The Challenge:

In retail, consumer trust can make or break brands. This is especially true for e-commerce businesses, where customers expect flawless online experiences, uninterrupted checkout processes, and cross-platform functionality on their phones and web apps. In this highly competitive space, even minor issues can steer customers away and toward competitors. Given its user base of over 21 million people, a single bug in user experience can result in significant revenue losses for the furniture retailer. 

With an upcoming launch in Ireland, the company needed to address the shortcomings of its previous provider, which struggled to own quality strategy autonomously, communicate clearly, and provide a full picture of regionalized bugs occurring across multiple geographies.

Discover and Resolve More Issues

The retailer’s previous QA provider struggled to identify and resolve a large volume of issues, which was less than ideal. The QA team needed both quantity and quality of bugs to remain priorities for their software testing initiatives. The team preferred to have visibility on all bugs, regardless of severity, and address them as and when they had the capacity. 

Staff In-Market Resources for Better Outcomes

The company’s previous QA partner couldn’t accurately perform in-market localization, device, and payment tests with only two dedicated resources assigned to the retailer. The lack of in-market resources meant the company had to rely on VPNs and simulated tests to ensure their mobile and web apps performed as intended across seven geographies. However, this could result in missed issues such as currency conversion, legal compliance, etc.

Ensure Efficiency With Around-the-Clock Testing 

With operations worldwide, having 24/7 testing resources is critical to the retailer’s success. Their previous QA vendor couldn’t offer efficient staffing models beyond their two dedicated testing managers. This ultimately pushed the company to seek other vendors that could provide quick and efficient testing cycles to keep up with their product builds and innovations. 

The Solution:

The leading brand sought a QA provider with a global presence and in-market resources to scale testing and ensure extraordinary user experiences across its seven operational regions. 

The QA team contacted several software testing companies to run a pilot project for them. They designed a rating system to ensure they found the best solution for their needs. 

The Pilot Project

As one of the companies in the running, Testlio quickly mobilized high-quality and vetted in-market freelancers for a 10-day pilot project. Testlio’s project leveraged 20 testers who identified 60 issues across various device/OS combinations and platforms, with 30 categorized as critical. After a successful pilot, the eCommerce giant chose Testlio as their QA vendor to help ensure seamless digital experiences across countries and brands. 

Today, the retailer leverages Testlio’s in-market resources in five countries. As a proven and flexible solutions provider, Testlio’s team performs functional, exploratory, localization, payments, accessibility, and UX testing across their web and mobile applications. 

Ensuring Both Quantity and Quality of Issues

The retailer’s most significant challenge with their previous provider was the quantity of issues they received. While the quality of bugs is essential to the company, the QA team prefers to stay on top of and monitor all bugs, regardless of their severity. 

With Testlio’s crowdsourcing solutions, the company can uncover 31x more bugs and resolve 15X more issues than before. A weekly cadence for app testing and a bi-weekly cadence for web testing ensures we can perform comprehensive QA and find both the quantity and quality of bugs the QA team expects. Our team meets quarterly with the retailer’s team to adapt our testing processes and share results from all test runs performed, enhancing collaboration and efficiency. 

Dynamic Staffing Model

Testlio’s unique client services model ensures around-the-clock testing and quick turnaround times by seamlessly handing off projects across different time zones in America, EMEA, and APAC regions. With on-demand resources available 24/7, our in-house testing team coordinates with each other to ensure all testing for app builds happens in a 24-hour cycle. Every week, the furniture retailer sends their build to Testlio’s American testing team, which starts the testing process and hands it off to EMEA, who then hands it off to APAC and back to America to ensure efficiency and coverage. This approach allows the retailer to maximize time spent on testing without sacrificing the quality of test runs and reports. Faster test runs also allow the company to resolve issues faster than before.

Comprehensive Payment Testing for Flawless Checkouts

The retail giant also relies on Testlio to regularly ensure the efficacy of its payment system to reduce cart abandonments and failed payments. Testlio’s innovative approach to payment testing, particularly during critical sales events like Cyber5, involves “time machine” testing and using VPNs to simulate future transaction scenarios. This process is crucial for ensuring the integrity and performance of the payment flow during peak periods across various regions and devices. 

In-Market Resources for Accurate Testing Outcomes

On-demand access to Testlio’s global network of 10k+ skilled freelancers ensures app localization and includes testing region-specific payment methods like Klarna with in-market resources for optimal testing outcomes. With in-market resources, the retailer can confidently release new builds, features, and payment methods for users in different regions. 

Bi-Directional Integrations for Improved Efficiency and Collaboration

Testlio’s integration with the retailer’s Jira and Xray systems provides real-time insights and feedback across 18 Jira boards. This level of integration is vital for optimizing coordination across various development teams and ensuring the company doesn’t overlook any details.

Outcomes:

The retailer’s partnership with Testlio has enabled them to deliver faster and better releases more confidently. 

Preventing Potential Vulnerabilities With Ease

Testlio’s customized approach helps them uncover 31x and resolve 15x more issues to ensure peace of mind. In the last quarter alone, the Testlio team uncovered 600+ issues across their web and mobile app. The emphasis on even the smallest issues enables the company to accurately track and monitor potential vulnerabilities. 

Seamless and Ongoing Collaboration

Testlio’s platform integrates with several critical tools in DevOps toolchains, allowing the company’s team to communicate with their testing team in real time. Testlio provides reports and insights across 18 Jira boards for various departments. From weekly cadences to ad-hoc testing, the retailer can quickly and seamlessly communicate with Testlio’s team to prioritize and address critical issues. 

Testlio’s dedicated engagement and testing managers meet with the company every quarter to present results, optimize strategy, and offer recommendations for improvement. The QA leads can also contact their dedicated managers anytime to flag issues or concerns and suggest improvements. This open line of communication helps the retailer constantly improve its processes to serve its growing customer base better. 

1-Day Turnarounds

Testlio’s intelligent staffing model enables us to deliver results for app builds in a 1-day testing cycle. Our testing managers coordinate internally to ensure seamless hand-off for off-hours testing by leveraging freelancers in EMEA, APAC, and the Americas. Since each region has dedicated team leads assigned to the retailer, quality is maintained throughout test runs and regions to ensure accurate test results and outcomes. 

Global Testing for an International Brand

Testlio’s network of experts, conveniently located worldwide, allows the client to unlock quick testing cycles, improving testing efficiency and accuracy. This approach has helped streamline the company’s QA processes and positioned it for success in existing and new markets, especially with the upcoming focus on UX testing to boost user growth and adoption. 

By partnering with Testlio, the retailer successfully launched their app in Ireland and has since consistently ensured high-quality builds and minimal bugs with our crowdsourcing solution. The ability to test during off-hours, with on-demand in-market resources, and in a 24-hour cycle has consistently enabled the retailer to deliver extraordinary customer experiences for users in different regions.