QSR Happy Path Testing Case Study

The Company

Founded in the 1950s, the client is a quick-service restaurant chain made up of 3,000+ franchises in 40+ states. Over the years, the chain has maintained a quirky, retro feel and a wide variety of menu items while being a consistent innovator with its proprietary mobile app. Competitors include popular quick-service restaurant chains like In-N-Out Burger, Chick-fil-A, McDonald’s, and other major chains with drive-thru options.

Digital offering

Our client is a top-20 iOS app producer in the food and drink space. They offer a native mobile app with exclusive deals, menu information, custom ordering, pickup time scheduling, group orders, and integrations with loyalty programs. Their development team operates on two-week sprint and release cycles. Their culture of innovation leads them to experiment and pilot various technology like on-location AI-powered ordering and Alexa-enabled voice-activated advance ordering.

The Challenge

QSRs operate on thin margins within a highly competitive market focused on low prices and express service. Offering a consistent and pleasant end-to-end customer experience is essential. Until recently, providing fast-food customers with an easy-to-use ordering, rewards, and express pickup app was a novelty. Now, this type of technology is the point of entry to meet customer expectations.

But building these features and catching bugs before customers do is easier said than done, which led our client to search for the best software testing solution.

“Due to limited exposure to customer experience, device, and locations, we’ve allowed bugs to go into production, which have a significant impact on our user base and stores. We are looking to enhance our QA capabilities so that we can eliminate these issues.”

QSR Client

End-to-end app-driven customer experience

The client released a limited feature mobile app to a few remote locations and was weighing whether to launch it nationally. In addition to a seamless app experience, they wanted to evaluate the user experiences that extend beyond the app.

To feel confident about a scaled deployment, they first needed to answer questions by testing hundreds of potential device, payment, and order combinations.

• Real device testing
  • Is the mobile app easy to use?
  • Do loading times affect user experience?
  • Does the app function as expected?
• Payments and order management testing
  • Do all payment methods process?
  • Can in-app rewards be applied alongside other forms of payment and as stand-alone payments?
  • Can in-app rewards be earned, tracked, and used with ease?
• Happy path customer experience and usability testing
  • Do orders match selections made on the app?
  • Are check-out cashiers receptive and polite?
  • Was the order ready at the correct pick-up time?

The Solution

Testlio tackled various testing scenarios to identify high-priority bugs related to functionality, payments, and the end-to-end app-driven customer experience. Testlio combined the power of burstable, on-demand testers working within short testing windows to match development cycles through a networked testing approach.

Expert testers traveled to various restaurant locations and swarmed the client’s mobile app to find issues. They reported bugs using Testlio’s testing management platform and JIRA integration. For payments, they used 100’s of device/OS combinations and methods, including Google Pay, credit card, the client’s in-app credit, and Apple Pay.

Testers took videos, screenshots, and pictures for in-app bugs and in-person experiences so that the client could better understand and reproduce them.

Real device testing

A pool of expert testers conducted on-location (in-the-wild) testing comprising a blend of functional testing and structured exploratory testing. They used a mix of real devices, networks, and locations to ensure complete coverage, tracking and reporting insights on loading time, menu layout, ease of submission, and payments with different device/OS and payment method combinations.

Happy path usability testing

In addition to the mobile app and payments testing, Testlio’s expert testers conducted end-to-end usability testing to capture hiccups in the real-world user experience. They searched and customized orders, applied coupons, scheduled pickup times, submitted orders, and went to pick them up.

At the restaurant, testers tracked how fast food came out against the client’s processing time goals, checked the receipt, and verified the order for accuracy.

Outcomes

Using the Testlio test management platform, testers submitted actionable bug reports. The client reproduced and resolved the issues, gaining confidence that the new app features were ready to roll out nationally.

Testing outcomes

Testlio’s expert testers diagnosed the following high-priority issues in need of resolution before going into production or as part of a national roll-out:

• An inability to tip with gift cards
• Functional bugs like gift cards being charged twice on Android
• An inability to cancel orders in-app
• One location delivered orders 200% slower than the target order time

Business outcomes

• The new features supported a 2x growth in new users following testing
• The app update played a role in a 30% annual increase in mobile sales