With an estimated 80% marketshare in 2015, Android is the most popular mobile operating system in the world. Mobile apps developers obviously have taken notice and the Google Play Store now boasts over 1,900,000 published apps.
Android OS offers tremendous opportunities for mobile apps developers. But Android application testing is known to be significantly more demanding than its iOS counterpart, mainly due to fragmentation.
What is fragmentation?
Because Android is open source, it is used by dozens of manufacturers who produce thousands of different mobile devices of all shapes, sizes, and hardware compositions. While this is great for consumers who benefit from a very large selection, the down side is that the differences in the hardware and software of these different devices can hinder the functionality of applications.
As such, an application may function very well on the latest Samsung Galaxy, but not an all on the HTC One M9 due to differences between the two devices. There is no alternative for developers but to test their apps on all statistically relevant devices in their target market.
Furthermore, unlike iOS where you can expect that the vast majority of users are using the latest version of the OS, Android updates are rolled out at different intervals depending on the device and the carrier. This causes users to be spread across a significant number of different platform versions.
As of January 2016, KitKat (4.04) is the most popular version of Android with a 36.1% share followed by Lollipop 5.0 at 16.9%. This means that 47% of Android users are using different OS versions.
If you multiply the number of devices by the number of statistically relevant platform versions, you can easily see how Android application testing can become very time consuming.
The key to Android application testing
Emulators and a number of tools can help you ensure the quality of your Android apps. But to guarantee that your users will be delighted by your app, manual testing on a good number of real devices is hugely beneficial. There are no shortcuts.
Here are some tips to ensure that you are tackling your Android application testing project efficiently:
- Test first on the oldest version of Android OS supported and the most recent version available
- User analytics data to identify the most popular devices used by visitors to your website or existing app and determine how many devices you would need to cover 80 or 90% of visitors
- Make sure to test at least the most popular device for each manufacturer with presence in your target market
- Don’t forget to change your device’s operating language
- Test your app using Wi-Fi, LTE, 3G and no internet connection at all
- Interrupt testing is very important! Remember that many events can interrupt your users’ experience
Unified testing criteria for Android applications
If you are unfamiliar with the App Quality Alliance, it is an initiative led by key players of the mobile industry – such as AT&T, LG, Motorola, Nokia, Samsung and many more. Their objective is to provide guidance and useful documentation to the mobile development community.
For many years, the Java Verified certification was the standard in mobile apps testing. Now that the landscape has changed, the App Quality Alliance has evolved is now setting the standard for Android apps development.
The Unified Testing Criteria for Android applications, initially released in March 2011, covers every aspect of Android applications: installation, interruptions, readability, performance and multiple display format handling.
The test cases contained within the Unified Testing Criteria are a great place to start when testing Android applications. In fact, we’ve used them for our own generic Android application test plan.
How can we help?
Buying a lot of Android devices is very costly. We know, we buy them. While we obviously cannot possess every single Android device in the world, we have multiple versions of the world’s most popular Android devices and we ensure to mix and match combinations of hardware and versions of Android OS to mimic real-world conditions.
But having the devices is only a part of how we can help you. Proper Android apps testing is an art that takes years to master.
Our team consists of highly experienced testers, some of them with over 10 years of experience in mobile apps testing. Testing Android apps since the very first Android device was released has made us aware of the sensitive areas of applications. We know where to look when it comes to finding bugs!
Contact us and we can talk about how to ensure that the users of your Android application are delighted with their experience.