End-to-end Fintech Case Study

Company overview

This mobile-first financial technology company specializes in investment advice and micro-investing for over 9 million global customers. They push major releases for their mobile app five times per month on average and have a leading app store rating of 4.75.

The Challenge

• Rapid growth and limited QA resources lead the client to frequently release without full device coverage.
• Releases without full device coverage lead to a bad customer experience on several device/OS combinations, ultimately causing customer churn.
• A small in-house QA team required a software testing services provider to help manage end-to-end testing.
• Pressing issues meant the client needed help as soon as possible with limited time for onboarding.

The Solution

After completing an ROI analysis of hiring more internal QA or relying on a software testing partner, the client identified significant cost savings through Testlio’s fully managed networked testing offering. Learn how they calculated the ROI.

Given the client’s desire to begin testing as soon as possible, Testlio created an expedited onboarding experience and built a fully-managed testing team to do all the heavy lifting.

Example of Fully-Managed Onboarding Experience

Testlio clients select how they want to manage their testing within a continuum of fully-managed, end-to-end, white glove testing services or co-managed with shared responsibilities. This client needed Testlio’s fully-managed testing services, powered by a test management platform, complete client services, and a network of expert testers.

The client’s fully-managed services team included:

• An engagement team manager.
• An engagement manager to manage strategy and delivery and to serve as the POC.
• A testing manager responsible for test run success, freelancer tester coordination, and executing the overall plan.
• 25 expert testers in 13 countries using over 70 device/OS combinations

The client’s onboarding was fast, enabling testing to begin in days.

• Kick-off meeting to finalize requirements and conduct testing software training
• Project set up with application and environment access granted. Testlio builds testing strategy and testing plans with recommended test cases to review for client approval. Testlio manages issue tracking integration, configuration, and validation.
• Testing kickoff begins with using an automated suggestion engine to match freelancers in the correct countries and with the right skill for the engagement. The final builds are delivered and the first build submitted through the Testlio platform. On-demand testers go into rapid response mode.
• Same-day and overnight runs proceed rapidly with fixed issue verification, sanity checks, and other passes.
• The testing team reviews and approves issues with real-time sync to the client’s bug tracking system
• The testing team prepares an end report with insights into the cycle, top issues, and recommendations.
• Testlio and the client meet for a retrospective to compare results to benchmarks.

Outcomes

• The client earned a spot in the top 30 finance apps in the Apple App Store.
• The client was able to scale their testing without management responsibilities like hiring and training resources.
• Same-day and overnight testing turnaround provided the engineering team with faster coverage and enough time to fix issues in advance of their weekly release schedule.
• Full manual testing coverage on over 70 real mobile device/OS combinations.
• Localization testing in 12 countries with different languages.