Defects Happen, So What Can You Do About Them? Arpita Goala , Content Marketing Manager August 30th, 2024 The perfect defect-free app doesn’t exist… unless you shrink your codebase and deliver a product no one wants to use. Even then, there’s no guarantee it won’t have issues. For developers, zero-defect goals set unrealistic expectations that not only cause frustration but also lead to diminishing returns. Suppose you somehow manage to do the impossible and release software with no defects, the cost and time spent eliminating the last few defects are often disproportionate to the value they add. Even Experts Agree, Bugs are Unavoidable Industry leaders like David Heinemeier Hansson advocate for a balanced approach to handling bugs, acknowledging them as part of the process rather than exceptions. His blog “Software has bugs, this is normal” poignantly points out: “Once we accept that simple fact that software = bugs, we can progress to understand why fixing them may not even be that important a lot of the time. The absence of bugs is simply one parameter of success in software, but not even close to the most important one.” So, what is the most important parameter of success? Ensuring you deliver a valuable experience and prevent any critical bugs from going live. Still, there is a very real possibility that a few annoying pesky bugs make it into deployment. While they are unavoidable, how you respond can make or break the user experience. In this article, we asked industry experts and Testlio leaders for their go-to strategies for handling post-release defects. Oops, I Shipped a Release with Defects, What Do I Do? Here are a few recommended approaches for handling defects in a way that doesn’t add to your technical debt: Understand the Risks Often, issues go unnoticed because there is a lack of understanding of the associated risks, causing teams to not have the right mitigation strategies in place. “Before we get into the discovery and analysis, let’s talk about why issues get missed. For me, issue slippage is all about understanding the associated risk. Is the team aware of the risks and repercussions of faulty software, like unhappy customers, cart abandonment (revenue loss), integrations not working, etc? Because, in the end, all QA efforts are about protecting the software’s reputation, and it’s imperative your team understands what that means for your organization.” – Javier Mejia, Associate Director of Client Services, Testlio Issue Discovery Discovering issues is the first step in handling them. Here’s how to systematically uncover and categorize bugs that might otherwise slip through the cracks. “There are essentially three ways in which we find out or realize that an issue slipped through: user feedback via AppStore/PlayStore reviews or directly within the app, reports from clients or developers to QA managers, and detections by our QA team during subsequent testing runs. By integrating user inputs directly within the app, we ensure that we’re quickly aware of and can address bugs experienced by users.” – Alex Ilustre, Manager, Testing Delivery, Testlio Bug Analysis and Validation Once a defect is flagged, the next crucial step is rigorous analysis—ensuring that what we see is an actual bug worth fixing immediately. “Once an issue or defect is reported, it’s important to confirm if it’s an actual issue and not a misunderstood feature or user error. This involves manual testing across different platforms and devices to ensure the issue is consistently reproducible. For me and my team, it’s also important that we verify if the issue existed in older versions or if it was introduced in the latest build. We can explore the why once we know when the bug is introduced. Was it a lack of testing, incorrect instructions, complete oversight, or was the issue within the scope of testing, etc.?” – Alex Ilustre “Once we realize that issues have slipped, we can do a thorough investigation, find the root cause, fix and re-deploy them, using various tools. However, the important thing to then consider is, how do we prevent this from happening again. I think that this is the question that QA leaders should really be asking themselves and their teams, with the understanding that software will ALWAYS have bugs.” – Javier Mejia Stakeholder Communication Transparent communication with stakeholders is not just about conveying facts; it’s about fostering trust and clarity when unexpected issues arise. “Communicating effectively with stakeholders about discovered issues is critical. When my team uncovers an issue in a client’s product, we explain why an issue was missed and propose a remediation plan to prevent future occurrences. This level of openness is critical to maintaining trust and ensures our process is transparent, whether the stakeholders are external clients or internal team members.” – Alex Ilustre Internal Briefing and Accountability “Post-discovery, it’s important to sit down with your testing team to dissect what went wrong and how we can enhance our testing quality. This includes direct conversations with individuals responsible for the oversight to understand the reasons behind it and to foster an environment focused on collective improvement rather than individual blame. Ultimately, as the manager, I take full responsibility for the quality of our engagement, ensuring that our team remains aligned and accountable.” – Alex Ilustre Crisis Management Instead of waiting for defects to occur, plan for them. Developing robust crisis management plans allows you to act quickly and effectively when major issues arise. “Leaders need to recognize that software cannot be fully defect-free, and issues will always be part of it. While we do our best to avoid any defects, especially P1 and P0 bugs, leaking into production, it’s important to acknowledge that some issues may still slip through. So my advice is to consider designing crisis management action plans and protocols during the QA and test strategy creation phases.” – Ramy Loaiza, Principal Testing Manager, Testlio What should your plan include? According to Ramy, a great crisis management plan should ask the important questions like: What are the risks? What steps should we take to resolve an issue once it’s discovered? How do we prioritize it? Who are the stakeholders, and what information should we share with them? Which deliverables will be part of a potential post-mortem record? What test cases and suites should be implemented? How hotfix deployment and sign-off will be performed? Which remediation or damage control/containment protocols must be put in place for P0 issues? How will the post-mortem process and test strategy inclusion criteria guide further revisions and improvements? Collaboration is Key Without a unified approach, handling software defects can become disjointed. Fostering collaboration across departments strengthens your crisis response and defect management strategies. “Our crisis management plans involve key stakeholders across the development spectrum to ensure comprehensive incident management. While QA leads the initiative, input from product, development, and sometimes corporate communications and marketing teams ensure we take an integrated approach to defect management rather than a siloed one. Ultimately, these strategies should align with broader management goals, ensuring all testing and quality efforts are cohesive and reflect the organization’s overall objectives.” – Ramy Loaiza Preventing Issues from Reoccuring Preventing defects from recurring is as important as fixing them. Learning from your mistakes helps refine your approach to software testing. “So if we know that defects are in fact inevitable, how do you ensure that your QA activities protect brand reputation? I think that this starts with adopting a mindset of shift left testing, which is to test early and more often. This also means identifying your core flows and ensuring those are covered as much as possible. For my team, this includes not just re-testing fixed bugs but also testing them from multiple angles with varied data sets. Regular and thorough regression testing, ideally with the help of automation, helps reduce the reoccurrence of critical bugs, safeguarding our software’s reputation against potential damage.” – Javier Mejia Perfection Shouldn’t be the Goal As app complexity increases, striving for perfection is often counterproductive. Successful teams recognize that defects are an inherent part of the development process. The true measure of success isn’t the absence of bugs but the ability to manage and respond to them effectively. Effective defect management involves a combination of proactive strategies and responsive actions. Ultimately, the goal is to embrace a mindset that anticipates defects and ensures that your team is equipped to handle them adeptly. Because, at the end of the day, the role of a QA team is to deliver excellent, not perfect, user experiences. Ask us how we can help you deliver digital experiences your users will love.