Talking with TalkBack

I presented this topic at the GDG Sydney DevFest 2019, slides here.

Integrating with accessibility is simply the practice of making a product usable by as many people as possible. By improving usability, all user groups benefit and not just people with disabilities.
An inclusive product caters to the widest possible range of abilities within the broadest possible range of situations. This is not the same as designing for disabilities but rather for all users and making a positive impact to bring benefits to everyone.
In this post, let's take a look at how we can improve the accessibility of an app specifically for TalkBack.

What is TalkBack

TalkBack is a screen reader which helps blind and low vision users understand and interact with anything on their screens. It is part of the Android Accessibility Suite which is a collection of accessibility services that help you use your Android device eyes-free or with a switch device. It is comprised of Accessibility Menu, Select to Speak, Switch Access and TalkBack.

TalkBack features

• Screen reader
  • Allows users with visual impairments to interact with their Android device without seeing the screen.
• Navigate using linear navigation or explore by touch
  • Linear navigation: Swipe right or left to navigate through screen elements in a sequence. Double-tap anywhere to select.
  • Explore by touch: Drag your finger over the screen to hear what's under your finger. Double-tap anywhere to select.
• Common gestures
  • Has some common gestures to perform certain standard tasks
• Accessibility shortcut
  • To enable or disable the service quickly
• Display speech output
  • For developers mainly to debug the spoken feedback

Basic principles

Label content

Label informative content when no supporting text is available. By default, unlabelled content without any text will be announced as "Unlabelled" which can be frustrating when encountered.

Add the  android:contentDescription attribute specifying the description to be read for the associated content.

<ImageButton
    android:layout_width="wrap_content"
    android:layout_height="wrap_content"
    android:contentDescription="More information"
    android:src="@drawable/ic_info" />

Some best practices that have been commonly advised are:

1. Keep the content description concise and unique. Concise to better support other accessibility services(ex. BrailleBack) and unique to allow screen reader users to distinguish between different elements
2. Don't describe how to physically interact (e.g. click, tap, etc.) in the content description. AccessibilityService can take care of that as accessibility includes more than just TalkBack users like Switch Access or Voice Access users who don't interact with the screen at all
3. Have the most important information be announced first to enable users to decide whether to continue listening or skip ahead.
4. Decorative graphical elements should have their descriptions set to null like android:contentDescription = "@null" or set android:importantForAccessibility= "no". In case of a ViewGroup use android:importantForAccessibility= "noHideDescendants" to hide all the elements contained within the ViewGroup to be hidden from TalkBack.

Label editable content

Use android:hint to supplement an editable text field's usage. Do not use contentDescription with editable fields. Instead, use android:labelFor attribute with a view that should act as a content label for another view since hint goes away as a user types into a field.

<com.google.android.material.textfield.TextInputLayout
     android:layout_width="match_parent"
     android:layout_height="wrap_content"
     android:labelFor="@+id/age_edit_text"
     android:text="Age">

       <com.google.android.material.textfield.TextInputEditText
           android:id="@+id/age_edit_text"
           android:layout_width="match_parent"
           android:layout_height="wrap_content"
           android:hint="Numbers only"/>

  </com.google.android.material.textfield.TextInputLayout>

Structure content

By default, TalkBack announces each element individually however it may not be ideal to navigate by swiping a lot of elements when presenting information in the form of a natural group. When related items are placed together they usually end up within a container i.e a subclass of ViewGroup and we can present the content descriptions in a more digestible way to users of assistive technology.

To allow TalkBack to announce each inner element's content description as a concatenated string in a single announcement add android:focusable="true" (or android:screenReaderFocusable="true" for API 28 and above ) to the container and ensure each inner element has its focusable attribute set to false

Also, the container can have its own contentDescription set to announce a specified text which would make it easier to understand the grouped information presented.

Alternatively, with some additional nesting of the view hierarchy, the groups could also be divided into chunks of related information which may make it easier to navigate the layout as it scales to host larger amounts of information.

Intuitive navigation order

The default traversal order of screen readers is to start from top-left and follow a "Z" shape till bottom-right. It might be required in certain situations to change this default navigation order to better convey the logical order in which the items are connected. From API 22 onwards android:accessibilityTraversalAfter and android:accessibilityTraversalBefore can be used to change the traversal order.

Announce dynamically updating content

When you need to relay information that updates dynamically without the ability to be able to navigate and focus on an element then we can use the accessibilityLiveRegion API. This could be in situations where data is being updated without navigation changes or any content shown on a view that is of importance to a user to be announced immediately as a result of some user action.

There are three possible options for accessibilityLiveRegion: none(default) | polite | assertive

none: no live region updates, this is the default.

polite: TalkBack won't interrupt whatever is being currently announced but will announce the updated content after it has finished through all its current announcement events.

assertive: TalkBack will allow the newly updated content to interrupt any in-progress announcements.

Try to avoid using assertive as it interrupts all current announcements and the experience could be jarring to a user.

TalkBack friendly solutions to common patterns

Network requests and responses

For network or any asynchronous operation, we may need to keep the user notified about the current status. In this case, there might not be any form of UI element to show this but perhaps a progress indicator and there might be no live regions available.

By simply calling  view.announceForAccessibility("results updated") or window.decorView.announceForAccessibility("results updated")  from an Activity TalkBack will announce the text string provided.

Modify default spoken text (API 21+)

TalkBack can figure out the associated accessibility role and its associated information for native elements. For example, a Button is associated with a click action that announces Double tap to activate  when it is clickable. This can be modified to provide a more tailored experience to users.

There are APIs to add, remove and replace accessibility actions available from the View class and the ViewCompat helper class can be used for ease.

The following code replaces the default click action of a button for TalkBack to instead announce a different click action text and also to perform the actual action as well.

ViewCompat.replaceAccessibilityAction(
    button,
    AccessibilityNodeInfoCompat.AccessibilityActionCompat.ACTION_CLICK,
    "place order"
) { view, arguments -> 
	placeOrder()
	true 
}

Gesture actions

TalkBack on Android supports a host of different gesture actions built-in to provide control over how content is navigated, announced and perform complex actions that aren't just tap or a double-tap related actions.

We can easily add our own actions to the natively provided actions as well. These can be accessed by swiping up and right to show a local context menu that contains the actions available.

ViewCompat.addAccessibilityAction(
	itemView, 
	"delete"
) { view, arguments ->
	itemView.delete()
	true
}

Other considerations when integrating with TalkBack

• Numbers and abbreviations in content descriptions might require to be space-separated.
  • TalkBack isn't smart enough to announce it correctly at times and this also applies to other screen readers. The content description should represent how the text should be eventually announced to a user. For example, a card number is best read out digit by digit rather than in thousands.
• Try to avoid having another set of flows just for users of assistive technology.
  • This adds additional effort from a maintainability and testing standpoint, however, some experiences may require dedicated efforts in this area so choose wisely.
• Extend from lowest in hierarchy when extending for custom views.
  • The Button class extends TextView which extends View, if you are creating a custom view that would act similar to a button then extend from the Button class itself since that would provide you with all the button-like accessibility offerings such as click handling and the various state announcements.
• Do not create an inconsistent user experience.
  • With so many apps available try to follow the existing patterns established by Android as much as possible so users do not have to learn a different pattern just for a particular app. Help users leverage their existing knowledge as they use your app.

Testing

• Lint rules!
  • Use and pay attention to Lint warnings for common accessibility issues.
• Accessibility Scanner
  • Accessibility Scanner is a tool that suggests accessibility improvements for Android apps. Use the Accessibility Scanner app to analyze your screens and see suggestions to improve the accessibility of your app.
• User testing
  • The best way to test the accessibility of an app is to test with real users.
• Espresso and Robolectric
  • Automated tools and testing have their own limitations and should be used in addition to manual user testing.

Conclusion

Making an app accessible for users of assistive technology requires some planning and effort upfront but thankfully most of the work is already built into the framework itself and the final product extends its reach to an even wider group of people.

Additional resources

Build more accessible apps