The ultimate guide to functional testing

Functional testing validates that the software is executing actions as it should. The software industry relies on functionality testing, performed by expert testers to ensure that all app segments and features are working as intended. Functional testing aims to identify functionality-based errors in hypothetical customer pathways so developers can fix the issues before releasing to production. 

If your web or mobile app can’t perform its core tasks, no amount of good UX, design, or speed will keep consumers satisfied. Imagine the first cell phone. What were its main functions? The ability to dial a number, accept and reject a call, and talk with another person remotely. Back in the days when there was a limited choice of cell phones, consumers had to make do no matter how poor the functionality. Now, the days of make-do are over. The market is saturated with countless smartphone models and applications from manufacturers around the world.

Product quality is now the true differentiator – it affects customers’ purchasing decisions and business profitability. No one wants to use an app that crashes or a website that can’t scroll down. Poor-quality software cost software teams over 2 trillion dollars in 2020.

In this guide, we’ll cover everything you need to know about functional testing, including definition, steps to test functionality, strategies, and use cases.

Table of Contents

• What is functional testing?
  • History of functional testing
  • Why we do functional testing
  • Types of functional testing
• How to do functional testing
• Functional testing strategies
• Examples and use cases

What is functional testing?

Functional testing is a section of software testing that validates the web or mobile software against pre-determined specifications and requirements. This testing ensures that software is performing as both developers and end-users expect. These requirements are translated into test cases for testers to execute and range from responsiveness and navigation to connectivity and payment funnels. Functional tests often use a black-box testing method where the code is unknown. The tester uses inputs/outputs to validate functionality.

The goal of functional testing is simple: make software, web, or mobile apps that work. Well, not so simple, after all. Consumers are not a simplified one-trick pony. They have access to countless device and OS combinations, and they are from hundreds of countries with thousands of data carriers and Wifi providers. They have computers, consoles, tablets, VR, cell phones, smart-TVs, smart-watches. 

Functional testers try to emulate scenarios and pathways that an end-user might take. Functionality is also about testing for non-functionality. What happens when the app goes into airplane mode? Or the video buffers on screen? 

Today, functional testing is seen as a first step in the software testing lifecycle, but when did it become a mainstay in software testing?

History of functional testing

Conceptual functional testing began around 1967 with the IBM white paper ‘Evaluation of the Functional Testing of Control Programs’ by William Elmendorf. This paper calls for a disciplined approach to software testing, including measuring, testing, and verifying functionality. Elmendorf published a manual to test software functionality, outlining individual functional requirements and testing features. 

Functional testing of control programs is undergoing a transition from a predominantly imprecise art to an increasingly precise science.

William Elmendorf

_{The first proposed functional test case plan by W. R. Elmendorf}

By 1968, functional testing was a central part of software development plans. During NATO’s Software Engineering Conference, QA for software production was a major topic. The conference produced a checklist for planning software system production, with a section on QA, and specifically asked: “is the product tested to ensure that it is the most useful for the customer in addition to matching functional specifications?”

Fast forward to 1998, and Stefano Mazzocchi published V1 of Apache Jmeter, a software tool to load test functional behavior and measure performance. In 2002, IBM published RobotJ, IBM’s rational functional tester.

With adequate tools and how-to guides, dev teams entered into the early 2000s focusing on software specification testing, bug catching, and preventative tests. 

Now, functional software testing is a robust segment of the software development lifecycle, with methodology on shifting right and left for testing. With the rise of automation, testers are using automated tools to test functions that remain constant, to avoid flaky tests.

Why do we do functional testing?

Functional testing validates that the software is executing as expected, which is a pivotal part of ensuring a quality product. Before or after an update or new release, devs can measure the quality of the web or mobile app version by the number of successfully passed test cases. 

But why else do we do functional testing? And why is it important?

Choice. Consumers have their choice in software options, and they’re willing to pay more for a better product that functions as they expect. Podium’s state of online reviews survey found that sixty-eight percent of consumers would be willing to pay more for the same product if guaranteed a better experience. Ninety-three percent of consumers look to online reviews to influence their purchases. Word of an app that struggles to function, or has bugs like gift cards being charged twice, can spread across review sites and destroy your product/brand reputation. Experience, functionality, and usability are not optional features – they make or break your business model.

The non-functionality of web and mobile apps can also have profound business implications that can fly under the radar. In one example, Testlio’s testers found that their client, a top-15 U.S. iOS shopping app, had a series of functionality failures in payment channels. Inside the app, varying bank account number formats prevented one of the world’s largest payment platforms from facilitating purchases. If your customers can’t pay, your product has a major time-sensitive problem. 

Types of functional testing

The umbrella term “functional testing” for software, systems, and apps is an amalgamation of different functionalities that you test for. Are buttons working? Passwords accepted? Video running at normal speed? These functionalities are tested through a variety of specific tests. The following are some of the most common:

Exploratory Testing

Exploratory testing can be an optimal way to test new features and for media-driven testing. Testing teams must learn about a product, its market, risks, and previous failures. This helps them uncover new tests. But explorations must be structured. And testers need to be highly skilled.

Scripted Testing

Scripted testing is manual testing, typically done by a junior tester who follows a step-by-step procedure written by a more senior tester. 

Regression Testing

You’re going to update code, features, and systems constantly. When you do, performing regression tests ensure that those changes do not adversely affect functionality.

Smoke Testing

This is your basic, initial scripted test that aims to find severe high-level flaws. Does the app run? Will the systems interface? Does the button do anything? Smoke testing is done for every build, and low-level tests every prominent feature and function. 

Unit Testing

Unit tests test a single function (the unit) and replace the rest of the system with other code called “mocks.” For example, a unit may be a function that calculates the sales tax on an item. The purpose of the unit tests is to ensure the code works from a functional perspective. Since the rest of the system is mocked out, unit tests can generally test every permutation possible for the function, including errors and exceptions. Unit tests run quickly; are usually written in the same language as the underlying code and are tightly coupled to that code. Unit testing uses a white box testing method.

Component testing

Done after unit testing, component testing takes each object/feature of the software and tests them separately. Component testing contains individual units, classified as a component, and tests that unit using a black box testing method. 

Component: [ Unit 1, Unit 1.1, Unit 1.2, Unit 2, Unit 3]

Sanity Testing

Sanity tests come after smoke tests. Sanity tests are narrow, deep, specific tests of a particular feature. Similar to a smoke test, sanity tests evaluate whether the software, program, or app meets claims of its performance. 

Smoke test > issue found > minor edits > sanity test

Regression testing requires QA teams to comb through the existing product with each new change. The work is time-consuming and there are tight windows of engineering downtime to conduct testing and deliver results without delaying development.

UI testing

User interface testing (UI functionality, specifically. UI and UX testing is another 3,000-word guide) primarily seeks to validate the function of two types of user interactions: inputs and visual elements. Can your web app accurately read a mouse click? Does a user keyboard integrate into your app and perform as expected? Are buttons/links/submission forms functional?

Integration Testing

This is the phase of software testing where software modules are combined, requiring group testing. Integration testing evaluates the compliance of software or apps with specific functional requirements. Integration tests segment components or modules and verify the functionality of each one individually and as part of the group. 

Acceptance testing

Acceptance testing includes beta testing, alpha testing, user acceptance testing and others to, quite frankly, see if the product or software is at an acceptable level. Individual tests target users, business goals, or beta versions of the software, and evaluate functionality.

System testing

System testing occurs when software is fully developed. At this stage, the software is fully integrated with additional hardware and software but isn’t yet released to the market. System testing plans perform many tests (including unit, load, regression) to evaluate and verify the functionality of the end-to-end product within a full computer system or software stack. 

How to do functional testing

A well-defined functional test plan starts by understanding what to test and what resources are available. Start with this post on creating any good test plan to familiarize yourself with general common terms, best practices, and steps. Then, let’s dive into these six steps for a functional testing strategy. 

Step 1: Set goals

Set parameters for what you define to be the product/app/websites:

• Specified requirements
• Business-critical features
• End-user expectations 
• Target end user
• Ideal functionality
• Ideal UX/UI

To test for functionality, you need to determine specifics. Does the term “working button” and a passed functionality test refer to the primary function? Or does it include usability and accessibility? Interface? Passing a use case test?

Step 2: Create test data

If you want to perform a functional test, you’re going to need data. Test data is the input given to the system that should have a specific outcome. You can use this test data to compare future runs. How do I get this data, you ask? You create test data by putting the software through specific conditions (i.e. wifi, device, network) and recording corresponding outputs (i.e. speed, time, expected actions).

Step 3: Create end-to-end scenarios

Think like an end-user and brainstorm common actions and pathways. Create a list of features, locations, device and O/S combinations, and languages, to test. Then, come up with some specific end-to-end scenario examples. For example, you might test a photo upload feature. When you add a photo, does it get added to the digital asset library? When you add a tag to that photo, does it get saved in the photo’s metadata? Does the picture you added to the tag appear in the digital asset library’s search results when you later search that tag?

Step 4: Design test cases

After you design your parameters that need to be verified and customer pathways, design test cases. Here’s an example: 

Payments testing test case

Description: A customer opens up the app, adds an item to her cart, checks out with an incorrect credit card number

Preconditions: The user has login credentials.

Steps:

1. Create a user profile with an email address

2. Add items to the cart

3. Checkout

4. Type in incorrect credit card information

Expected result: The user should receive an in-app notification of payment failure due to incorrect credit card information with the option to go back.

During this phase, lay out what appropriate feedback looks like and how you want testers to submit case results. Think about what entails a pass/fail in this case, and have specific criteria to evaluate during the next step.

Step 5: Run test cases

When you’ve gone through the steps, it’s time to execute the test cases and prepare to write the test case. 

Follow the actions on the test case to a T so that the run is easily repeatable.

_{Example test management platform run}

Step 6: Track and resolve

Perhaps the most valuable testing outcome for your dev team is the testing results. Results should offer traceable actions with visibility into the entire testing journey. Include test case details like date, parameters, devices, what devices are tested, costs, outcomes, and recommended actions. Write the test case inside of your test management tool. 

Functional testing strategies

Scripted vs. exploratory

There are two significant ways to run functional tests, scripted or exploratory. What’s the major difference? Well, for testers, freedom. 

Exploratory testing, the newer of the two strategies, investigates and discovers issues on the fly. Testers uncover issues as they navigate the app, mimicking user pathways and experiences. Exploratory testing emphasizes the individual tester’s experience to seek out and repair bugs – but you have to trust in the authority and brilliance of your testing team. Highly skilled testers must learn about a product, its market, risks, and previous failures. We hold that exploratory testing can be an optimal way to test new features.

Exploratory testing can be structured or unstructured. What? I thought you just said tests are either structured OR exploratory. Well, yes. And no. Structured exploratory testing puts parameters on the exploration and creates a very flexible use case that can change as testers dive into the app. Unstructured exploratory testing is even more fluid, where testers navigate around the app and use their domain knowledge for efficient exploratory testing and useful feedback.

“Exploratory testing gives testers the freedom to explore and try to break the app in a unique way. It can also catch edge cases you wouldnt normally catch in a regression run. If you’re doing regression back to back with a success rate over 95%, mix in some exploratory testing to dive into harder to catch areas.

Kirsten Johnson, Engagement Manager at Testlio

Scripted test execution is a method where senior QA engineers write out scripts for testers. Testers follow pre-determined steps, allowing QA testers to compare expected vs. actual results. A scripted test seeks to confirm that the software can perform the intended function. 

“I’d use functional testing to evaluate core features that need to be functional. If you’re testing a website search, regression testing should ensure that the link works, search bar works, and searches will pull up the right information,” said Johnson.

Here are some standard functional testing techniques:

Installation testing – for desktop or mobile application, testing proper installation

Boundary value analysis – testing of the boundaries of numerical inputs

Equivalence partitioning – grouping tests together to reduce overlap of similar functional tests

Error guessing – assessing where functional issues are most likely to be found and testing these more extensively than other areas

Unit testing – testing at the smallest level of the software to ensure that each unit/feature is executing properly

API testing – checks that internal and external APIs are functioning correctly, including data transfer and authorization

Regression testing – tests performed to verify that new software changes did not have adverse effects on existing functionality (most common automation technique)

Functional test case examples

Think of functional test cases like individual pieces of a functionality pie. You need to make sure the entire pie is delicious but you also need to divide it into slices and conquer. 

Functional test cases are individual tasks, pathways, or actions that testers need to test to ensure overall product functionality. Test cases aren’t just testing features, but diving into usability, connectivity, and scalability. Start with investigating a question, and create a test case scenario around that question. 

Here are 5 functional test case scenarios:

1. Does the app behave as intended when faced with external interruptions? (i.e., receiving SMS, minimized during an incoming phone call)
2. Can the device multitask as expected when the app is in use or running in the background?
3. Are social network options such as sharing, posting, or joining a stream working correctly?
4. Does the app support payment gateway transactions?
5. Are appropriate error messages received as necessary?

If you’re testing manually, utilize a test case management software that allows you write test cases and track metrics (like priority, pre-conditions, time estimates) in a singular place. Testers assigned to a specific case can record bug reports, add data, and explore previous runs, all within the test management platform. 

There are some fundamentals to consider when writing a test case. An individual test case should always include:

• Feature being tested
• Objectives
• Preconditions
• What if/failure scenarios
• Clear steps to complete the test
• Pass/fail metrics

To visualize theoretical test case questions, like those above, I’ll share a case study. When Testlio partnered with the sports network arm of a top-15 iOS publicly traded media company, they identified livestream bugs, faulty APIs, and complicated user flows in the beta version of their app.

By creating test case scenarios and executing them, Testlio’s testers were able to uncover:

• Faulty APIs that negatively impacted ad revenue and data strategy
• Non-responsive video stream buttons
• Testlio also conducted a particular type of exploratory testing to crawl the entire site manually that revealed various data issues from faulty API’s

With the help of Testlio, the client gained the confidence needed to make a significant feature update and release. Replicating Testlio’s efforts in-house would require a significant financial and time investment by their development team. 

Should you build an in-house team, or crowdsource your functional testing? To learn more about how to scale your functional testing, read about the economic considerations of crowdsourcing your QA (we call it networked testing) or growing QA in-house. 

Conclusion

Only 4 out of 100 unhappy customers will complain directly to a company — the other 96 will churn without providing feedback. Since it’s 6-7 times more expensive to acquire a new customer than keep an existing one, unlocking that silence is key. 

Consistently testing product functionality ensures that customers are able to use the product as intended, and that software errors are caught before reaching customers.

Functional testing verifies product features work according to predetermined requirements, identifying areas where performance doesn’t meet expected results. If your app isn’t meeting expectations, explore how Testlio can help you boost functionality through end-to-end functional QA testing services and tools.