Software Test Estimation & 6 Techniques Testlio June 13th, 2024 Software testing evolved from a simple debugging activity in the 1950s to becoming integral to software development with advanced testing tools and test estimation techniques. As a C-level executive or business developer, ensuring your teams provide accurate QA effort estimates is crucial. This precision influences the project outcome and bolsters your credibility with clients. Underestimating QA efforts can lead to potential underperformance and unclear requirements. Therefore, your QA teams must master software test estimation techniques to meet client expectations and consistently deliver on promises. In this article, we will explore various test estimation techniques you can leverage to deliver even complex projects backed by impeccable estimation efforts. What Is Test Estimation Software test estimation (STE) is a technique to calculate the time and resources required for project testing. This activity is usually performed by senior leaders. But why do you need it? Quality assurance (QA) or software testing is mandatory in SDLC to avoid critical issues. It ensures that your application works smoothly without any major problems. However, as apps become more complex, testing every possible critical use case can result in wasted time and resources without proper planning. To address that significant concern, testing leaders perform software test estimation. The goal is to avoid ambiguities and set clear expectations regarding the test phase so you can align your budget and timeline against these estimates. You may wonder what’s the benefit of software test estimation techniques when skilled QA engineers can provide reasonable testing costs and time estimates based on their experience. After all, they are the domain experts. Well, software estimation goes beyond simply measuring time and cost. It also addresses essential factors contributing to software development and affecting timelines and end results, such as resource replacement scenarios, change request management, bridging communication gaps, and preparing contingency plans for other unforeseen circumstances. Therefore, implementing appropriate software test estimation techniques not only promises sound project planning but also speaks volumes for the quality of testing services. What Is Software Test Estimation For? Other than providing estimates to complete software testing, STE can act as an infallible catalyst in streamlining software development and overall project planning in the following ways. 1. Resource Allocation Proper estimation allows for the efficient allocation of resources, such as the right number of testers with the required skill set, the necessary tools, and the testing environment. Overallocation and underutilization of resources can be avoided. 2. Time Management It helps set achievable milestones and deadlines with realistic scheduling and rational timelines. Unnecessary delays in the testing cycles can be prevented, as the estimation also ensures that testing phases are well-synchronized with development. 3. Budgeting STE enables precise cost forecasting for the testing phase, covering personnel, tools, and other resources. It also allows for a sound cost control system to be in place. 4. Risk Management Meticulous testing estimation helps identify and mitigate potential risks that could impact the testing process. Contingency plans can be created in advance to address challenges and unexpected issues using the correct estimation methods. 5. Stakeholder Communication Precise and accurate estimates help set realistic stakeholder expectations regarding testing efforts and timelines. It also establishes transparent reporting pertinent to testing activities so the stakeholders know the progress and any deviations from the plan. 6. Quality Assurance STE ensures thorough testing and prevents underestimation, which leads to a high-quality final product. 7. Project Planning and Control Project planning gets improved as critical paths and dependencies are identified earlier. The estimates can be used as a benchmark to track progress and make informed decisions, giving you more control over project completion and testing. In short, software test estimation ensures that testing activities are well-planned, resources are efficiently used, and the project is delivered successfully. Test Estimation Techniques The growing customer demand, increasing pressure from stakeholders, and advancements in technologies make software estimation a necessity to ensure project success and efficiency Let’s examine the most effective software testing estimation techniques being used by testing leaders worldwide. Work Breakdown Structure As the name implies, the Work Breakdown Structure (WBS) approach divides the project scope into smaller modules, which are subdivided into specific tasks/functionalities. Each QA team member is assigned a particular module or a task to assess and provide test estimates. After each module/task is approximated, the estimates are added to identify the total estimated time required for project testing. Pros of WBS Cons of WBS Smaller tasks are easier to estimate. Not applicable to smaller projects. It prevents overlap, eliminating the risk of multiple teams/testers simultaneously working on the same task. It is resource intensive, i.e., it involves all the stakeholders because knowledge is required from different project areas, especially when there is immense dependency between various modules. It enables quick identification of the responsible tester for any specific task. In case of change requests, the affected parts of the project are re-estimated; thus, the overall time duration is also re-calculated. 3-Point Software Estimation Test This software testing estimation technique considers three cases for each testing task. An estimate is calculated for each case. The average of all three values is computed using the three-point estimation formula to determine the nearly accurate estimate. This approach categorizes each task for agile teams as optimistic, most likely, or pessimistic estimates, reducing bias and providing a realistic view of potential outcomes. Here are the cases and their estimates respectively: Situation based cases Estimates What does it determine? Best Case Optimistic Estimate (O) Considers all the happy paths in testing where everything is ideal. Worst Case Pessimistic Estimate (P) Where nothing goes according to plan or requirements. Most Likely Most Likely Estimate (M) It’s a mixed situation that is neither ideal nor entirely wrong. Both positive and negative aspects of testing, along with risks and contingency plans, are considered. Three-point estimation formula: E = (O + 4M + P) / 6Where E is the total average expected duration. Example The upcoming release includes changes to the fields in the Salesforce forms used to collect leads’ data for the three user types. The QA team needs to perform ripple effect testing, as the values from the altered and new fields will affect the calculation in the system backend and thus be responsible for changed quotation prices for all three user roles. Also, the team needs to consider potential unforeseen challenges. Optimistic Estimate (O) Pessimistic Estimate (P) Most Likely Estimate (M) The QA team estimates only the positive scenarios and gives a ballpark figure of 1 week’s effort. The QA team foresees the inevitable challenges and roadblocks due to updates. They give an extended estimate of 4 weeks for testing under the worst conditions. The team adopts a rational approach and gives a realistic testing estimate of 2 weeks. Applying the three-point estimation formula E = (O + 4M + P) / 6 E = (1 + 4×2 + 4) / 6 E = 2.17 The expected duration of testing in this case is 2.17 weeks. Pros of 3-Point Estimation Cons of 3-Point Estimation Avoids optimistic and pessimistic biases, resulting in more realistic estimates. It is unsuitable for tight schedules as gathering and analyzing data for three different scenarios requires more time and effort. Encourages thorough analysis of potential risks and uncertainties, promoting a deeper understanding of the project. Expert testers must understand the project’s complexity and give accurate estimates for all three scenarios. Distribution in Percentage This is a very straightforward software testing estimation technique that evaluates and approximates each testing stage/activity in terms of percentage. It is based on historical data or industry benchmarks. Steps to Implement Distribution in Percentage Determining the testing stages and activities. Allocating a specific percentage to each testing activity to reflect the effort (in percentage) required. If, for example, your testing cycle has the following phases or activities, you can assign a particular percentage of effort to each. Test Plan: 15% Test Case Development: 15% Test Environment Setup: 5% Test Execution: 50% Test Cycle Closure: 15% Pros of Distribution in Percentage Cons of Distribution in Percentage It does not require deep expertise in estimation, so it is accessible for project managers and teams without extensive experience. It lacks precision and may lead to potential underestimation or overestimation as it might not account for unique project-specific factors. It is helpful in the early planning stages as it provides quick estimates. It uses broad percentages that are unsuitable for projects with unique or variable testing needs. It relies on historical data and industry standards, offering a reasonable basis for estimation. Depends on the availability and accuracy of historical data or industry benchmarks, which may not always be reliable or relevant. Tips While this software testing estimation technique offers a good starting point, it’s essential to be mindful of its limitations. Here are useful tips for using this technique effectively. You can consider adjusting the percentages to fit the specific context of your project better. You can combine this method with other test estimation techniques to achieve more accurate and tailored estimates. Wideband Delphi Method This is an iterative software testing estimation technique in which estimates are collected anonymously from domain experts. The estimates are then collated and shared across the panel so that each member can review their opinion in light of new concerns raised by others. These rounds of collecting revised estimates from experts continue until a unanimous consensus is reached. Several roles can be involved in this process, such as: Coordinator/Moderator: Responsible for collecting and organizing estimates. Project manager: Leads the project. Domain experts: These are requested to assess the project and share their insights along with estimates for testing. Steps to Implement Wideband Delphi Method You can use the following steps to apply this test estimation technique: Select a team of domain experts. For example, you will contact the SaaS testers and managers if the project is an ERP (SaaS) system. Request each team member to share their estimates anonymously to avoid any bias. Collect the estimates from each individual and prepare a summary. Share the estimates summary with the same team members for review. Repeat the steps 2 – 4 iteratively until the team narrows down to an accurate and similar estimation. Eventually, you will reach a final consensus estimate based on the collective judgment of the experts. Factors to consider for using the Delphi method To get a comprehensive perspective, select experts from diverse yet related fields. Ask the same questions from each expert to get a uniform answer. Share the perspectives and answers anonymously with the panel to avoid bias. Pros of Wideband Delphi Method Cons of Wideband Delphi Method Multiple rounds enhance the chances of reaching a more accurate final estimate. The iterative process can be lengthy, resource-intensive, and exhausting. Bias and the influence of dominant individuals are reduced due to anonymity. Collaboration and coordination among all participants can be challenging. Functional Point Analysis Functional Point Analysis (FPA) is a systematic software testing estimation technique. It measures the software functionality that will be delivered to the end user. It was introduced publically by Allan Albrecht of IBM in 1979. FPA categorizes the requirements under different function types, assigns a functional point value to each requirement based on complexity, and then summarizes all the points using the FPA formula. The final figure reflects the total number of man-hours required to deliver the specific requirements. Steps to Implement the FPA Test Estimation Technique 1. Identify and Classify Functions Functions are divided into five types: External Inputs (EI): Data entering the system (e.g., user inputs, data entry screens). External Outputs (EO): Data leaving the system (e.g., reports, output screens). External Inquiries (EQ): User queries requiring a response (e.g., search operations). Internal Logical Files (ILF): Internal databases the system maintains (e.g., tables, files). External Interface Files (EIF): Databases used by the system but maintained by other systems (e.g., shared data files). 2. Assign Complexity Weights Each function gets a weight based on its complexity (low, average, high). These weights are predefined according to industry standards. 3. Calculate Unadjusted Function Points (UFP) UFP = ∑(Number of each function type × Complexity weight) 4. Determine the Value Adjustment Factor (VAF) This factor adjusts the UFP based on 14 general system characteristics (GSCs) as listed below. Each characteristic is rated from 0 to 5. The sum of all the 14 GSCs ratings is labelled as Total Degree Influence (TDI). TDI is used in VAF calculation and it can have a value from 0 – 35. TDI = ∑GSCs ratings VAF=0.65+(0.01×TDI) 14 GSCs Data Communications Distributed Data Processing Performance Heavily Used Configuration Transaction Rate On-Line Data Entry End-User Efficiency On-Line Update Complex Processing Reusability Installation Ease Operational Ease Facilitate Change Multiple Sites 5. Calculate the Adjusted Function Points (AFP) AFP = UFP×VAF Example: Consider a project with the following functions and complexity weights: Function Type Number of functions Complexity Weight External Inputs (EI) 10 Low 3 External Outputs (EO) 5 Average 4 External Inquiries (EQ) 8 High 6 Internal Logical Files (ILF) 7 Average 7 External Interface Files (EIF) 4 High 10 Calculate UFP UFP = (10×3)+(5×4)+(8×6)+(7×7)+(4×10) UFP =30+20+48+49+40 UFP =187 Assuming the sum of GSCs ratings is 35 VAF = 0.65+(0.01×35) = 0.65+0.35 = 1.00 Calculate AFP AFP = 187×1.00= 187 187 man-hours are approximated to test the functionality of the project. FPA in Agile Teams FPA is not limited to traditional methodologies. It can be effectively integrated into agile workflows to address measurement challenges and improve predictability and productivity. It provides clear, quantifiable metrics and promotes thorough testing by accounting for all functionality. Adapting FPA for Agile Teams: Objective Measurement: FPA provides a standardized way to measure functionality, independent of technical implementation and development methods, making it suitable for agile teams. Enhancement Function Points (EFP): Measures functionality added, changed, or removed during sprints or releases, giving a clear project size. Automated Measurement: Tools for Automated Function Points (AFP) streamline the process, ensuring consistent and accurate estimations. Productivity Insights: FPA helps compare team performance, track progress, and manage scope, providing a common view across teams and projects. Pros of FPA Cons of FPA Accurately measures the scope of the project in terms of functionality. It can be challenging to implement as it requires detailed analysis and classification of functions. Its application is not bound to a specific technology or programming language. Assigning complexity weights and GSCs ratings can be subjective and affect the estimate’s accuracy. Agile Estimation Agile estimation is a critical technique that focuses on delivering high-quality software through iterative development and continuous feedback. Unlike traditional estimation methods, this approach embraces change and prioritizes flexibility and responsiveness to evolving project requirements. Key Agile Estimation Techniques Planning Poker: Planning Poker is a consensus-based estimation technique where team members use a deck of cards with numbers representing story points, which correspond to the effort required for a task. T-Shirt Sizing: T-shirt sizing is a relative estimation method in which tasks are sized using categories like XS, S, M, L, and XL, similar to clothing sizes. Bucket System: The bucket system is a structured estimation technique that organizes tasks into buckets of increasing size (e.g., 1, 2, 4, 8, 16 story points). Affinity Estimation: Affinity estimation groups similar tasks together and then assigns story points based on relative size within each group. Steps to Implement Agile Estimation Select the Estimation Technique: Choose an Agile estimation technique (e.g., Planning Poker, T-Shirt Sizing) that fits your team’s needs and project requirements. Prepare the Backlog: Ensure that the user stories or tasks are well-defined and understood by the team. Conduct Estimation Sessions: Facilitate estimation sessions in which the team discusses, estimates, and reaches a consensus on each task. Document Estimates: Record the estimates in the project management tool to keep track of the planned effort. Review and Adjust: Regularly review and adjust the estimates based on feedback and changes in project scope or requirements. Example Consider a scenario where you and your team estimate the effort required to implement a new login feature in a web application. You have decided to use the planning poker technique for this project. The steps you’d follow include: Story Discussion: The team discusses the new login feature, including requirements, potential challenges, and dependencies. Individual Estimation: Each team member selects a card representing their estimate (e.g., 3, 5, 8 story points). Reveal and Discussion: All estimates are revealed simultaneously. The estimates range from 3 to 8 story points. Discussion: The team discusses the reasons for the different estimates, considering potential risks and complexities. Re-estimation: After the discussion, team members re-estimate the task, converging on an estimate of 5 story points. Pros of Agile Estimation Cons of Agile Estimation Agile estimation adapts to changing requirements and provides the flexibility needed for iterative development. Techniques like Planning Poker can be time-consuming, especially for large backlogs. Promotes team collaboration and ensures that all perspectives are considered, leading to more accurate estimates. Estimates can be subjective and influenced by individual biases. Supports continuous improvement by incorporating feedback from previous iterations into future estimates. Teams new to Agile estimation may require time to learn and effectively apply the techniques. Provides transparency in estimation and planning, helping stakeholders understand the rationale behind estimates and project timelines. Effective coordination and communication among distributed team members can be challenging. Factors to Consider When Choosing a Test Estimation Technique Attaining the knowledge of software testing estimation techniques or software estimation tools does not guarantee you success in deducing correct estimates for your project. Not all testing estimation methods can be applied to all the projects. You need to identify which estimation technique best fits your project testing needs. It would be smart to consider the following factors when choosing a test estimation technique: Understand the project size and complexity by analyzing the requirements. Some techniques are suitable for smaller projects such as distribution in percentage. Consider the project’s timeline and resource constraints. Extensive estimation methods such as Functional Point Analysis may not be suitable for projects with tight schedules. Project budget or cost should be taken into account because testing can be more costly than expected for some projects. Identify risk factors that can impact the testing efforts. Evaluate your team’s skillset before selecting a test estimation technique. As we explored, techniques like FPA require specialized knowledge; therefore, choosing the testing technique that aligns with your team’s capabilities is essential. Determine the type of project development method. If it’s agile, your estimation technique should be flexible and adaptable to changes and updates. Figure out the accuracy and precision requirements. Some techniques, such as the distribution of percentage method, give rough estimates. Some projects require more accurate estimates. So, you should select the test estimation technique that complements your project requirements. Test Estimation Challenges Just like software development, challenges are inevitable in the testing estimation process. Here are some common ones you might encounter. Incomplete or uncertain requirements can lead to deviation from the real estimates. Frequently changing requirements can cause repetition and delays. Lack of cooperation and poor client communication can lead to false assumptions and misunderstandings. The project’s complexity can make it take time for the team to completely comprehend the scope and quality of the project/product to be delivered. Lack of historical data or relevant past project information can make it challenging to create reliable estimates for new types of projects. Technology upgrades and updates can introduce unknown variables that complicate the estimation process. The above-listed challenges may seem unrealistic to conquer. However, you can ensure success with the right combination of experts, detailed requirement analysis, effective use of historical data, stakeholder communication, and the application of appropriate software testing estimation techniques tailored to the project’s context and scope. Final Thoughts To conclude, software testing estimation (STE) cannot be overlooked today as it has emerged as the backbone of successful software development and smooth end-product delivery. Therefore, it is advisable to introduce yourself to the latest trends and ensure that upgraded testing techniques and tools are applied to your software projects. If you need assistance at any point, consider Testlio at your service. We can help you correctly approximate your testing efforts and create realistic timelines for your project. Schedule a call with an expert on our team today to learn how we can help you make QA efforts more efficient.