In a study on mobile application usage and retention, 52 percent of users stated they were frustrated by overall app performance. Testing the performance and stability of your mobile application is critical to the user experience and, therefore, also important for protecting your revenue and reputation.
From on-the-go scenarios to various network conditions, QA teams should include mobile performance testing as part of your continuous testing pipeline.
Mobile application performance testing environment
Before your QA team begins a performance test, they must first understand the various operating systems, devices, and user markets being tested.
Emulators and real devices are best for running performance tests because development teams can:
- Put the system under high load using emulators
- Verify application behavior on real devices and networks
Mobile application testing strategy
Mobile app performance is heavily dependent on network and server performance. And in turn, your app needs to be able to recognize device and browser combinations in order to render properly. Not understanding how these all work together can lead to a reduced user experience.
Unfortunately, trying to test for every scenario can become overwhelming. As such, divide and conquer by actually splitting your performance tests into the following categories in order to build a more robust performance test strategy:
- Device performance
- Server performance
- Network performance
Device performance
Mobile devices are much more powerful than they were a few years ago. Even so, they have somewhat constrained resources. Users tend to dislike mobile applications that consume resources such as CPU, memory, and battery. For device performance, check for:
Application load time
How much time does it take for your app to load? Did you know over 25 percent of users abandon mobile apps after just three seconds of load delay?
Understand your app’s actual load capacity by testing it under various load conditions and build real-user flows to test on actual mobile devices. Keeping an eye on such issues is critical to ensuring that application response times match users’ expectations.
In addition to how long it takes your application to load, you should also consider other types of responses including serving up images or responding to the user’s input. QA teams should also understand why these response times vary.
Battery consumption
On constant use, some mobile apps consume a high amount of battery life. Excessive resource usage creates a burden on the processor.
Keep customers happy by ensuring your mobile application doesn’t eat away at the device’s battery life. Include mobile performance testing in your exploratory testing efforts when introducing new functionality and make performance testing part of your regression tests to ensure updates don’t adversely impact your application’s performance.
Memory consumption
Memory is one of the key resources of the mobile device. However, certain app functionalities increase memory consumption.
Especially important for Android app developers, memory consumption and leaks – to avoid annoying OutOfMemoryErrors – you should test to prevent your app from hogging precious mobile resources.
Hardware and software variations
We’ve already discussed why it’s imperative to test on real devices, and performance testing is no different.
It may sound like a no brainer, but in order to ensure your mobile app works across various hardware and software types, check apps on different devices.
Unsure which ones to prioritize? We’ve got you covered.
Usage with other apps
There’s no lone app that does what all users want. Hence why the average person uses nine mobile apps per day. This means at any given time, a user may have your app running with another.
As such, you should test for two scenarios:
- App running in parallel
- Another app running in the background
Check out Testlio’s full suite of QA testing capabilities including regression testing, exploratory testing, payments testing, and many more.
Server/API performance
Server/API level performance affects the speed of the app making the response time critical to an app’s performance. Response time is very critical, especially when applications are interacting with the server via API.
Data to and from server
How does your mobile application impact the back-end infrastructure supporting your app? Mobile applications should handle data efficiently as it’s sent to and from the server.
In certain apps, data is sent in a specified format that needs to be converted to a server readable format. In these instances, if the back-end server response time is slow, your app will be, too.
Additionally, data coming from legacy APIs may not be optimized for display or processing on a mobile device and can degrade the user’s experience.
API calls generated from the app
Test the API calls and the app behavior when a response is returned, regardless of the response.
In some cases, multiple API calls are made for the same functionality. These are redundant and slow your app’s performance. Overall, the number of calls from an app to the server should always be less – as low as you can make it.
Server downtime
When a server is down, the data must be saved on a backup server. You can do this by using failover database servers. This requires the backup server to be in continuous replication and synchronization with the main server.
Network performance
When creating a test strategy, start by defining the geographical location of your users as well as server and network types (3G, 4G, LTE, etc.). App behavior in low, slow, and no network conditions, as well as different bandwidth, latency, and packet loss scenarios, are all factors that shouldn’t be ignored.
For network performance, check for the following:
Network speed
Create different network conditions as well as network quality – everything from 2G, 3G, and 4G networks and Wi-Fi with best, average, and low speeds. QA testers should also understand how the app behaves when multiple networks are available and switching occurs from one network to the other.
Packets and packet loss
Collect network metrics on packets and bytes sent over the network as well as received. You should also test for packets dropped and total average delay.
In the case of complete packet loss, the app should be able to automatically resend the request or prompt a user accordingly.
Jitters
No, your app isn’t nervous… In this case, “jitters” is a delay in receiving information on the network. It’s merely a problem with connectionless networks or packet switch networks. But users should receive an appropriate notification when this happens.
Optimizing your mobile app performance
When conducting mobile performance tests, truly hone in on your scope and objective. Sometimes objectives may shift as tests are executed and mobile apps are adapted.
Contact Testlio today and ensure your mobile application has impeccable performance.