Automated Accessibility testing for Android apps

Malware News
1

Image source: https://www.flickr.com/photos/lwr/285539676

Accessibility for apps implies that differently abled people can access the same information as people without the disability.

Accessibility analysis for your apps can unveil new opportunities to make your app more accessible and hence usable by wider audience. This can be done by manually scanning the app screens via Accessibility Scanner app. However, automating this process helps in covering all areas that we might miss during manual accessibility analysis. Firebase test lab comes to our rescue here and gives us a report of all accessibility bugs in our app without any manual intervention.

So, let’s start leveraging firebase test lab to make our android apps more accessible.

Step 1: Go to the firebase console here after login in your account.

Step 2: To start creating a new project, click on “Add project”. Then enter a project name.

Creating a Firebase project — step 1

You can choose to enable or disable Google analytics for the project.

Creating a firebase project — step 2

Your Firebase project is ready now.

Step 3: In the left navigation menu, go to “Test Lab” under the “Release & Monitor” tab.

Step 4: In the “Android Robo test” section, click on “Browse” and upload the apk or adb file.

Robo Test in Firebase Test Lab

This will run the robo test on the apk or adb.

Step 5: On the Robo test screen, we get to see the device on which the test was run and whether it passed or failed.

Robo test screenRobo test finished and passed

Click on device details to see the results of the test on that device.

Click on the “Accessibility” tab to view accessibility issues.

Robo test accessibility results

Here we see accessibility issues that are warnings, minor issues and some tips.

We also get to see classification of these issues based on four accessibility guidelines -

1- Issues related to touch targets: These issues arise when the clickable element in consideration has width or height less than “48dp”.

Example of Touch target issue

2- Issues related to low contrast: These issues arise when the element in consideration has a contrast ratio that doesn’t abide by contrast ratio guidelines given by W3C.

Example of Low Contrast issue

3- Issues related to content labelling: These issues arise when the element under consideration doesn’t have a content description or has content description same as other elements on the same screen(which makes it difficult for assistive technologies to distinguish elements).

Example of Content Labelling issue

4- Issues related to Implementation: These issues arise when the element under consideration has implementation issues listed here.

Example of Implementation Issue in EditText element

From the screenshots, we get to know the screens and the elements that cause violation of a particular accessibility guideline. On viewing the screenshot, we get to know the element id, the issue with the element and how that can be fixed.

How to start Another Robo test on a different device(s)?

Step 1: Click on “Test Lab” in the left navigation menu in Firebase console. On the screen that opens, click on “Run a test”.

Running a new robo test

Step 2: In step 1(configure test), upload the apk or adb file and in “Select dimensions” tab, choose a device.

Selecting a device from the list of available devices.

You can also filter the available devices list based on API Level or Form factor(Phone, Tablet, Wearable).

Filtering available devices based on API LevelFiltering available devices based on Form factor

After that we can select the device by clicking on the “+” button, We can also select various devices and click on “Continue” button.

Selecting a deviceSelecting multiple devices

How to make the Robo test consider the user journey(or screens encountered) for a particular user?

Step 1: Click on “Test Lab” in the left navigation menu in Firebase console. On the screen that opens, click on “Run a test”.

Step 2: In step 1(configure test), upload the apk or adb file and in “Select dimensions” tab, choose a device and click on additional options. In the “Test Account Credentials” section, enter username resource name and value and password resource name and value.

Configuring test.Entering expected values for username and password

This will help in capturing accessibility issues encountered for signed-in users(if most of your app screens work for signed-in users) or when the app is used by a particular user.

You can also switch to the “Robo” tab and get to understand whether the supplied credentials were used successfully by the Robo test crawler and the crawling path(screens traversed).

Crawl graph of Robo Test

Conclusion

Leveraging Firebase Test Lab for automating accessibility testing has proved to be an effective & reliable method for finding potential accessibility issues in Android apps. We can get to know all kinds of potential accessibility flaws in just a few minutes without even needing to open the app.

This process can be used with our app builds of any build variant(debug or release), which helps us identify and fix accessibility issues earlier in our development cycle and hence aids in making the app accessible sooner to a wider audience.

Firebase Test Lab uses Accessibility Scanner app(and Accessibility Test Framework) for computing this list of accessibility issues, hence is a reliable and cost-effective method to identify accessibility issues in our app. We can understand more about Firebase Test Lab Accessibility issue categories here.

At the end of this blog, I would encourage all of you to try leveraging Firebase Test Lab for accessibility analysis of your Android apps, so that you can identify and fix accessibility issues sooner and build better and inclusive apps.

References:
1. https://developer.android.com/guide/topics/ui/accessibility

2. https://firebase.google.com/docs/test-lab/android/analyzing-results#interpret_accessibility_results

Automated Accessibility testing for Android apps was originally published in Walmart Global Tech Blog on Medium, where people are continuing the conversation by highlighting and responding to this story.

Article Link: Automated Accessibility testing for Android apps | by Monika Kumar Jethani | Walmart Global Tech Blog | Sep, 2022 | Medium