Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Android] Keyboard accessibility improvements #24359

Closed
wants to merge 6 commits into from

Conversation

sweggersen
Copy link
Contributor

@sweggersen sweggersen commented Apr 8, 2019

Summary

In order to meet our accessibility requirements we need to have full support for keyboard navigation. The Touchable components works with press/tap with a finger, but doesn't respond to 'enter' when using a keyboard. Navigation works fine. This PR adds an onClick listener to touchable views that have the onPress prop defined.

Changelog

[Android] [Added] - Add View.OnClickListener to Touchable components when onPress is defined

Test Plan

Add any Touchable component with an onPress callback. Use a keyboard and navigate to the touchable component and press the 'enter' key. Expected behavior is that the onPress callback is called.

@acoates-ms

Sam Mathias Weggersen
Microsoft Corp.

@facebook-github-bot facebook-github-bot added CLA Signed p: Microsoft Partner labels Apr 8, 2019
Copy link

@analysis-bot analysis-bot left a comment

Code analysis results:

  • eslint found some issues. Run yarn lint --fix to automatically fix problems.
  • flow found some issues.

Libraries/Components/View/ViewPropTypes.js Outdated Show resolved Hide resolved
Libraries/Components/View/ViewPropTypes.js Outdated Show resolved Hide resolved
@cpojer
Copy link
Contributor

@cpojer cpojer commented Apr 8, 2019

Thanks so much for the PR! Seems like our bot is unhappy, do you mind fixing it up?

Copy link
Contributor

@blavalla blavalla left a comment

To note up front, I am not very familiar with React Native's touchable elements, so I may be way off here, but I'm not sure that I follow along as to why we need to define a new "clickable" prop to make these work correctly.

It seems like anything that has an onClick or onPress listener should already be setting the View's focusable and clickable attributes to true. If that isn't happening already, should we not just fix that rather than adding a new prop?

@cpojer - I'm not sure who is most familiar with the ReactNative side of things here, but from an accessibility standpoint setting clickable/focusable on the View is helpful. I don't see why we wouldn't just want to always do this for any view that is clickable/touchable though.

@cpojer
Copy link
Contributor

@cpojer cpojer commented Apr 11, 2019

Thanks @blavalla!

@sweggersen what do you think about making this a default?

@sweggersen
Copy link
Contributor Author

@sweggersen sweggersen commented Apr 11, 2019

@blavalla @cpojer Thanks for great feedback. Yeah, I think that it makes sense to have it on by default.

Let me get back to you.

@cpojer
Copy link
Contributor

@cpojer cpojer commented Apr 11, 2019

Awesome, would you mind making those changes?

Copy link

@analysis-bot analysis-bot left a comment

Code analysis results:

  • eslint found some issues. Run yarn lint --fix to automatically fix problems.

Libraries/Components/View/ViewPropTypes.js Outdated Show resolved Hide resolved
cpojer
cpojer approved these changes Apr 17, 2019
Copy link
Contributor

@cpojer cpojer left a comment

Thank you :)

Copy link
Contributor

@facebook-github-bot facebook-github-bot left a comment

@cpojer is landing this pull request. If you are a Facebook employee, you can view this diff on Phabricator.

@react-native-bot
Copy link
Collaborator

@react-native-bot react-native-bot commented Apr 17, 2019

This pull request was successfully merged by @sweggersen in 01bcde3.

When will my fix make it into a release? | Upcoming Releases

@react-native-bot react-native-bot added the Merged label Apr 17, 2019
*
* @platform android
*/
clickable?: boolean,
Copy link
Contributor

@necolas necolas May 24, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I want to share some details about some of the work happening inside Facebook and how it relates to this change.

  • Supporting multiple input types is a problem for all platforms.
  • The web platform has designed APIs for this purpose.
  • The concept of "click" on the web is essentially legacy, and touch events still trigger onClick.
  • React Native for Web supports multiple input types but relies on many more events than onClick (i.e., onMouseDown, onKeyPress, etc.).
  • Internally we're experimenting with a new React event system. On the web that relies on Pointer Events and would eschew the need to use onClick. The general idea is to move away from input-specific event types and instead use onPress, etc., with the event object containing information about the pointer type that was used.
  • Internally we're experimenting with web implementations of View and Text that shed all the event props found in React Native (in favour of the new event system) and introduce a focusable prop to mark a View as being able to receive (or not) user-initiated focus by any input type (all Views are programmatically focusable). In this way you could set focusable={false} and retain all the mouse/touch functionality but hide the View from inputs like keyboards that first need to focus an element before it can be interacted with. That's an important pattern on the web where you might want to make something like an avatar interactive to pointers, but hidden from keyboards if the functionality is redundant, i.e., if there's also an interactive user name that performs the same action.

Given all that I'd suggest we do something like rename clickable to focusable and decouple it from onClick. If onClick here is meant to be used for mouse as well as keyboard, that's probably a good enough prop for now to unblock other input types on Android. We'll hopefully address this properly in RN in the future with the new event system.

Copy link
Contributor Author

@sweggersen sweggersen May 28, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the write up!

About the decouple of focusable from onClick, what do you suggest?

The Android default onClick listener works for all input types 'keyboard', 'touch', 'mouse', which is what we need support for. Before this change, only 'touch' and 'mouse' events would work.

Copy link
Contributor

@blavalla blavalla May 28, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Naming this "focusable" could definitely lead to some confusion here due to Android's already confusing separate concepts of keyboard focusability and accessibility focusability.

Setting "focusable=true" on a standard Android view will set it to be both keyboard and accessibility focusable, but if this is property is meant to only trigger keyboard focusability, we should probably be more explicit in its naming.

Copy link
Contributor

@necolas necolas May 28, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

About the decouple of focusable from onClick, what do you suggest?

I was thinking that the clickable/focusable feature is something we need independent of how events are handled. So in the future we might not have event handlers on View but we'll still need to mark views as being able to receive user-initiated focus following a mouse, touch, or keyboard event.

Setting "focusable=true" on a standard Android view will set it to be both keyboard and accessibility focusable, but if this is property is meant to only trigger keyboard focusability, we should probably be more explicit in its naming.

What is "accessibility focusable" on Android? The way we're using this on web is to allow the element to receive focus following any user action from any pointer device. It happens to also be required to support interactions on a view (with seems to overlap with this PR for Android). There might be a better name though…interactive? IIRC accessible is also equivalent to Android's native focusable!

Copy link
Contributor

@blavalla blavalla May 28, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To answer what "accessibility focusable" is requires a bit of a history lesson, so bear with me.

In the early days of Android many devices had trackballs or d-pads as an alternate non-touch way to interact with the device. To the system, these keys worked much like external keyboards. To allow your UI to be interactive you had to make sure it was focusable in "non touch mode", and the best way to do this was to set focusable="true". You would generally want all interactive elements to be focusable in this way, but wouldn't want non-interactive elements to be focusable, since it would be annoying to have to press keys to get past all of the non-interactive content.

Then later on Google decided to add a real accessibility system to Android, and built the Talkback screen reader as well as other accessibility services like Switch Access and Select to Speak. Some of these services (like Talkback) can only really be useful if you can focus on non-interactive content, like plain text. Otherwise, this content could be totally hidden from a user with a vision impairment. This meant that they couldn't simply use the same focus system used for D-Pad's and keyboards, and they had to come up with a way that you could have an element be focusable for accessibility, but not focusable for keyboard users. This is what I mean when I say "accessibility focusable".

Unfortunately, they didn't simply add a property like accessibilityFocusable and instead derived whether or not something would be accessibility focusable by looking at the combination of multiple different properties (click handlers, the focusable property, if the view was a leaf node, if it had focusable ancestors, etc.). It wasn't until Android Pie that they cleaned this up and added setScreenReaderFocusable to the View class.

Copy link
Contributor Author

@sweggersen sweggersen Jun 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, I think it makes sense to call it focusable. Who should do the work? Do we just update this PR?

Copy link
Contributor

@cpojer cpojer Jun 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please send a new PR that changes everything. We can do it all in a single commit :)

Copy link
Contributor

@necolas necolas Jun 5, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you'd like to write the patch that would be great! A new PR is best since this one has already been merged.

Copy link
Contributor

@cpojer cpojer Jun 11, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @sweggersen: just wanted to ping you and ask whether you could send a PR and rename this to focusable?

Copy link
Contributor Author

@sweggersen sweggersen Jun 12, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's on my list. I'll do it this week.

@necolas
Copy link
Contributor

@necolas necolas commented May 24, 2019

I left a comment on the diff sharing some context about how we're looking to solve similar use cases for the web. It might be worth adjusting this change. Let me know your thoughts

facebook-github-bot pushed a commit that referenced this issue Jun 21, 2019
Summary:
This is a follow up PR to #24359. There's a good thread in the mentioned PR for more background for why I'm doing this change. Essentially `focusable` makes more sense since it is about whether a view can receive user-initiated focus from a pointer or keyboard.
Pull Request resolved: #25274

Differential Revision: D15873739

Pulled By: cpojer

fbshipit-source-id: 0f526bb99ecdc68131dfc10200a5d44c2ef75b33
M-i-k-e-l pushed a commit to M-i-k-e-l/react-native that referenced this issue Mar 10, 2020
Summary:
This is a follow up PR to facebook#24359. There's a good thread in the mentioned PR for more background for why I'm doing this change. Essentially `focusable` makes more sense since it is about whether a view can receive user-initiated focus from a pointer or keyboard.
Pull Request resolved: facebook#25274

Differential Revision: D15873739

Pulled By: cpojer

fbshipit-source-id: 0f526bb99ecdc68131dfc10200a5d44c2ef75b33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
API: Keyboard CLA Signed Merged p: Microsoft Partner Platform: Android
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

8 participants