Customer-Focused Testing Strategies That Don’t Break the Bank Arpita Goala , Content Marketing Manager March 30th, 2017 QA improves products by incorporating customer data from other business functions and channels. Consolidating learnings from departments like support and sales can be invaluable. It’s also true that testing the full customer journey — from marketing touchpoints to acquisition to sign up and product use — results in the best experience. But these methods require a lot of manpower and cash. The most accessible way to track user behavior Sometimes sharing customer feedback is easy. Small teams can discuss feedback during standups. Large corporations who’ve invested in big data can pass along reports from the data analytics department. More often, this is hard data for QA teams to get their hands on. If you don’t have immediate access to customer support conversations or acquisition data, you can still choose to focus testing on the customer. Commonly tracked user experience metrics can give QA teams enough insight into user behavior, while helping them prioritize testing and stay on budget, all without large organizational shifts that might not be possible in the short term. Customer usage data is more readily sharable, easier to access and easier to implement during testing than are customer feedback or complaints. Smoke tests based on critical use Smoke testing catches failures early on and determines whether a build is ready to move on to QA or whether its failures warrant extensive fixes first. This saves money and time by not wasting QA efforts on a unit or build that isn’t ready for detailed testing. Customer usage data can take the budget-saving strategy one step further by determining the areas most deserving of smoke tests. Rather than arbitrarily choosing which major functions of an application will be tested to determine stability, QA engineers can choose the functions that matter most to customers. The larger a platform, the more that usage is distributed. One team found that out of six core features, three of them experienced 93% of all customer use. Knowledge like that can be used to do more than set up smoke tests. It should be shared with every department to make customer-focused decisions. Prioritization of features prone to failure Error rates are one of the easiest UX metrics to track. You can learn which user functions result in an error most often, which have the highest individual error rate, and which nearly always result in success. Any feature or function with an unusually high error rate deserves extra attention, and tests should be planned accordingly. Also, users’ favorite features should be monitored closely for error rate changes and should be tested more in-depth when risks are identified. By keeping an eye out for what users are struggling with and prioritizing those areas, QA teams can have a big impact on improving a product. Device coverage for mobile apps or browser coverage for web apps The best strategy for mobile device coverage is to support the app experience for as many of your users as possible. Real customer data is a requisite for choosing devices and how to test them. With user data, you can identify your top iOS or Android devices and then determine their importance. The top devices are best for external manual testing and/or in-house manual testing, while outliers can be partially supported with simulations. This data can help you decide on: Devices to externally test on Devices to collect and test on in-house Devices to simulate With web apps, user data can help your team decide which browsers to support with in-depth testing. You can determine which browsers your users are running old versions of, and determine whether the number of users warrants full testing in outdated versions. What you find might surprise you: A financial application discovered the need to test on a gaming console’s browser. Decision-making during automation testing Automated software testing brings a lot to the table. It maximizes efficiency and creates more time for other quality-enhancing methods. The question is NOT, should we be automating? (Since the answer is already, YES!) Instead the question is, what do we automate? Due to the cost of writing and maintaining automated tests, the answer to that question isn’t, Everything. In addition to key factors like repeatability and risk, usage data can help you confidently choose what to automate. Critical features, popular features, those that users rely on every day…backing these up with continuous automated testing will lead to quality improvements that keep customers on board. Performance testing for real-life scenarios Understanding the top customer features can also help determine areas of focus for performance testing. How does the backend hold up under strain? During peak traffic hours, which queries are initiated the most? Top features should be prioritized for load and endurance testing to ensure that during high volume, they can keep up with customer demand. Usage data can also inform the level of strain to test for. What is the highest number of users the app will have at one time in a given month? How long does peak load last for? Handling performance testing requires a sandbox environment and performance testing tools, which can be costly to set up. Having clear parameters based on real customer use can not only keep this type of testing on budget, but it can also ensure the biggest payoff for the customer. The purpose of QA is to improve products. Keeping all efforts focused on the customer leads to improvements that are validated and highly useful. Better yet, customer-focused testing can actually maximize your QA budget and make testing more affordable by accurately narrowing your overall strategy. Some of the ways that QA can focus on the customer might involve extensive user journey testing and the analysis of qualitative data like survey responses and social media complaints. But not every input is so superfluous. User experience metrics can help QA departments focus on the customer and stay within budget and timeline constraints. Testlio helps enterprises deliver quality experiences to their customers. For custom testing strategies and full test management, get in touch with us.