New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

APG guidance for touch devices #60

Closed
LJWatson opened this Issue May 25, 2015 · 12 comments

Comments

Projects
None yet
8 participants
@LJWatson
Contributor

LJWatson commented May 25, 2015

We need to address the problem of design pattern accessibility on touch screen devices. Touch ATs intercept all touch gestures. Only click/blur/focus are handled reliably - and they're not sufficient to support interaction with complex widgets like trees.

We should look to include guidance that surfaces these shortcomings, and indicate which of the design patterns will/won't work on touch devices with ATs enabled.

@patrickhlauke

This comment has been minimized.

Show comment
Hide comment
@patrickhlauke

patrickhlauke May 25, 2015

Member

For context (as the above is paraphrasing my recent ranting on twitter), this relates to further thinking around this bug/comment I filed https://lists.w3.org/Archives/Public/public-pfwg-comments/2015AprJun/0005.html / https://www.w3.org/Bugs/Public/show_bug.cgi?id=28694

Member

patrickhlauke commented May 25, 2015

For context (as the above is paraphrasing my recent ranting on twitter), this relates to further thinking around this bug/comment I filed https://lists.w3.org/Archives/Public/public-pfwg-comments/2015AprJun/0005.html / https://www.w3.org/Bugs/Public/show_bug.cgi?id=28694

@stevefaulkner

This comment has been minimized.

Show comment
Hide comment
@stevefaulkner

stevefaulkner May 25, 2015

Contributor

Have set up a google spreadsheet to record compatibility issues with design patterns
https://docs.google.com/spreadsheets/d/1gN9oRZPdrJxLDNtT6nVO4fn7E7sn1061L9Xl3__slZ4/edit?usp=sharing

Contributor

stevefaulkner commented May 25, 2015

Have set up a google spreadsheet to record compatibility issues with design patterns
https://docs.google.com/spreadsheets/d/1gN9oRZPdrJxLDNtT6nVO4fn7E7sn1061L9Xl3__slZ4/edit?usp=sharing

@patrickhlauke

This comment has been minimized.

Show comment
Hide comment
@patrickhlauke

patrickhlauke May 25, 2015

Member

A simple, concrete example: take an implementation of a "faked" slider like http://hanshillen.github.io/jqtest/#goto_slider - to my knowledge, this implementation follows the advice provided in http://www.w3.org/WAI/PF/aria-practices/#slider which defines the expected keyboard behavior (use of left/right arrow, home/end, page up/down, tab to get in/out of the slider widget itself). Now, as with many other patterns, this: a) assumes that it's working fine for mouse users (so nothing special is defined/mentioned about how a mouse user would need to interacts with it) and b) defines behavior for keyboard users, which the author is expected to include by hooking into keydown/keypress events, check for those defined expected keystrokes, and trigger relevant JS in response.

This simple slider is unusuable for users with touchscreen interfaces and AT. Simply taking the example of iOS/VoiceOver, a user is able to set focus to the slider "knob". VoiceOver announces (because of the role="slider") that it's "adjustable" and that a user can "swipe up or down with one finger to adjust the value". However, doing this has no effect - as the actual functionality that fakes this is attached via keydown events listening for arrow/home/end/etc, and swiping up/down does NOT trigger these (VoiceOver does not fire fake keyboard events that would rattle those event handlers, and I believe it shouldn't either - though it may have to come to this if no better solution is available). The naive solution here would be to also layer touch/gesture events on top of the keyboard listener events, but this will not work because VO will intercept touch events/gestures (similar to role="document" on desktop/keyboard/AT scenarios, and there's no sensible way to force touch/AT like VO to go into role="application" mode). The only way, if there WERE touch/gesture listeners, for a user would be to perform an override/pass-through gesture in VO (double-tap and hold, then swipe) to pass actual swipe gestures to the underlying page/JS, bypassing VO...but this, in my mind, is the philosophical equivalent of saying "keyboard users can just use mouse keys to interact with this" - it's a crutch, not a solution (and not all platforms have this sort of pass-through gesture...for instance, I don't believe Android/Talkback offers anything like this).

In short, even with such a simple widget as a slider, following the current advice/design pattern (which tacitly assumes a false dychotomy of mouse/keyboard only) results in a construct that simply will not work for touch/AT users.

Member

patrickhlauke commented May 25, 2015

A simple, concrete example: take an implementation of a "faked" slider like http://hanshillen.github.io/jqtest/#goto_slider - to my knowledge, this implementation follows the advice provided in http://www.w3.org/WAI/PF/aria-practices/#slider which defines the expected keyboard behavior (use of left/right arrow, home/end, page up/down, tab to get in/out of the slider widget itself). Now, as with many other patterns, this: a) assumes that it's working fine for mouse users (so nothing special is defined/mentioned about how a mouse user would need to interacts with it) and b) defines behavior for keyboard users, which the author is expected to include by hooking into keydown/keypress events, check for those defined expected keystrokes, and trigger relevant JS in response.

This simple slider is unusuable for users with touchscreen interfaces and AT. Simply taking the example of iOS/VoiceOver, a user is able to set focus to the slider "knob". VoiceOver announces (because of the role="slider") that it's "adjustable" and that a user can "swipe up or down with one finger to adjust the value". However, doing this has no effect - as the actual functionality that fakes this is attached via keydown events listening for arrow/home/end/etc, and swiping up/down does NOT trigger these (VoiceOver does not fire fake keyboard events that would rattle those event handlers, and I believe it shouldn't either - though it may have to come to this if no better solution is available). The naive solution here would be to also layer touch/gesture events on top of the keyboard listener events, but this will not work because VO will intercept touch events/gestures (similar to role="document" on desktop/keyboard/AT scenarios, and there's no sensible way to force touch/AT like VO to go into role="application" mode). The only way, if there WERE touch/gesture listeners, for a user would be to perform an override/pass-through gesture in VO (double-tap and hold, then swipe) to pass actual swipe gestures to the underlying page/JS, bypassing VO...but this, in my mind, is the philosophical equivalent of saying "keyboard users can just use mouse keys to interact with this" - it's a crutch, not a solution (and not all platforms have this sort of pass-through gesture...for instance, I don't believe Android/Talkback offers anything like this).

In short, even with such a simple widget as a slider, following the current advice/design pattern (which tacitly assumes a false dychotomy of mouse/keyboard only) results in a construct that simply will not work for touch/AT users.

@patrickhlauke

This comment has been minimized.

Show comment
Hide comment
@patrickhlauke

patrickhlauke May 25, 2015

Member

Another example of a construct that follows ARIA design patterns and fails for touch/AT: http://hanshillen.github.io/jqtest/#goto_menubar which follows http://www.w3.org/WAI/PF/aria-practices/#menu - specifically, if a menu item has a submenu, the pattern currently expects

If the menu item with focus has a submenu, pressing Enter, Space, or the right arrow key opens the submenu and puts focus on the first submenu item.

With iOS/VO, I can open the first level of each menu, set focus to any item, but if the item also has a submenu, i cannot trigger it (in Hans' implementation), as this will NOT send Enter/Space/right arrow, but simply a click event

Member

patrickhlauke commented May 25, 2015

Another example of a construct that follows ARIA design patterns and fails for touch/AT: http://hanshillen.github.io/jqtest/#goto_menubar which follows http://www.w3.org/WAI/PF/aria-practices/#menu - specifically, if a menu item has a submenu, the pattern currently expects

If the menu item with focus has a submenu, pressing Enter, Space, or the right arrow key opens the submenu and puts focus on the first submenu item.

With iOS/VO, I can open the first level of each menu, set focus to any item, but if the item also has a submenu, i cannot trigger it (in Hans' implementation), as this will NOT send Enter/Space/right arrow, but simply a click event

@patrickhlauke

This comment has been minimized.

Show comment
Hide comment
@patrickhlauke

patrickhlauke May 25, 2015

Member

A third one, for good measure: http://hanshillen.github.io/jqtest/#goto_tree which implements the TreeView http://www.w3.org/WAI/PF/aria-practices/#TreeView

With iOS/VO, I can set focus to individual visible nodes (and also hidden nodes, but this seems a shortcoming in Hans' implementation, rather than intention), but I cannot open/expand a tree branch, as I cannot fire

Right arrow key expands a closed node, moves to the first child of an open node, or does nothing on an end node.

I can only fire a click event at the actual focused node, which only triggers what I assume is the mouse-based equivalent coded for

Enter key performs the default action on end nodes.

Member

patrickhlauke commented May 25, 2015

A third one, for good measure: http://hanshillen.github.io/jqtest/#goto_tree which implements the TreeView http://www.w3.org/WAI/PF/aria-practices/#TreeView

With iOS/VO, I can set focus to individual visible nodes (and also hidden nodes, but this seems a shortcoming in Hans' implementation, rather than intention), but I cannot open/expand a tree branch, as I cannot fire

Right arrow key expands a closed node, moves to the first child of an open node, or does nothing on an end node.

I can only fire a click event at the actual focused node, which only triggers what I assume is the mouse-based equivalent coded for

Enter key performs the default action on end nodes.

@patrickhlauke

This comment has been minimized.

Show comment
Hide comment
@patrickhlauke

patrickhlauke May 25, 2015

Member

Additionally, when I originally said that for touch/AT situation we can only currently rely on focus/blur/click ... the situation is actually even murkier, as some OS/Browser combos don't send focus/blur when you'd expect it. see http://patrickhlauke.github.io/touch/tests/results/#mobile-tablet-touchscreen-assistive-technology-events

Member

patrickhlauke commented May 25, 2015

Additionally, when I originally said that for touch/AT situation we can only currently rely on focus/blur/click ... the situation is actually even murkier, as some OS/Browser combos don't send focus/blur when you'd expect it. see http://patrickhlauke.github.io/touch/tests/results/#mobile-tablet-touchscreen-assistive-technology-events

@patrickhlauke

This comment has been minimized.

Show comment
Hide comment
@patrickhlauke

patrickhlauke May 26, 2015

Member

Quick additional note, from testing: iOS devices with a paired bluetooth keyboard do not fire key events (whether VO is enabled or not) unless focus is in a text input/textarea (so patterns reliant on key events will fail even for "keyboard" users on iOS); Android with a paired keyboard does act mostly like a traditional "desktop" device with keyboard - no keys are intercepted, fired as normal (in standard Browser or Chrome; some browsers, like Firefox/Android, do provide additional shortcut keys when running Talkback, and these are intercepted)

Member

patrickhlauke commented May 26, 2015

Quick additional note, from testing: iOS devices with a paired bluetooth keyboard do not fire key events (whether VO is enabled or not) unless focus is in a text input/textarea (so patterns reliant on key events will fail even for "keyboard" users on iOS); Android with a paired keyboard does act mostly like a traditional "desktop" device with keyboard - no keys are intercepted, fired as normal (in standard Browser or Chrome; some browsers, like Firefox/Android, do provide additional shortcut keys when running Talkback, and these are intercepted)

@mraccess77

This comment has been minimized.

Show comment
Hide comment
@mraccess77

mraccess77 Jun 10, 2015

In my experience custom ARIA combo boxes are also problematic with mobile devices such as iOS and Android. Listboxes faired better in my testing. My presentation at CSUN indicated the same challenges with sliders, menus, and trees as discussed above. Spinboxes also were not supported.

mraccess77 commented Jun 10, 2015

In my experience custom ARIA combo boxes are also problematic with mobile devices such as iOS and Android. Listboxes faired better in my testing. My presentation at CSUN indicated the same challenges with sliders, menus, and trees as discussed above. Spinboxes also were not supported.

@DavidMacDonald

This comment has been minimized.

Show comment
Hide comment
@DavidMacDonald

DavidMacDonald Jun 15, 2015

I'm totally on board with this discussion and we'll be looking at it carefully with the Mobile Task Force

DavidMacDonald commented Jun 15, 2015

I'm totally on board with this discussion and we'll be looking at it carefully with the Mobile Task Force

@jnurthen jnurthen added the APG label Jul 8, 2015

@hshepherd

This comment has been minimized.

Show comment
Hide comment
@hshepherd

hshepherd Jan 25, 2016

It seems like interactions with role="slider" elements are still an issue in iOS Voiceover even when using the recommended aria attributes (aria-valuemin, aria-valuemax, aria-valuenow). Are there any plans to revisit this?

hshepherd commented Jan 25, 2016

It seems like interactions with role="slider" elements are still an issue in iOS Voiceover even when using the recommended aria attributes (aria-valuemin, aria-valuemax, aria-valuenow). Are there any plans to revisit this?

@patrickhlauke

This comment has been minimized.

Show comment
Hide comment
@patrickhlauke

patrickhlauke Jan 25, 2016

Member

interactions with ARIA menus that have submenus are still an issue in iOS/VO too. generally, over the last few months, i've actually started advising clients NOT to implement certain ARIA patterns as they fall apart on mobile.

Member

patrickhlauke commented Jan 25, 2016

interactions with ARIA menus that have submenus are still an issue in iOS/VO too. generally, over the last few months, i've actually started advising clients NOT to implement certain ARIA patterns as they fall apart on mobile.

@MichielBijl

This comment has been minimized.

Show comment
Hide comment
@MichielBijl

MichielBijl Apr 29, 2016

Contributor

This issue was moved to w3c/aria-practices#8

Contributor

MichielBijl commented Apr 29, 2016

This issue was moved to w3c/aria-practices#8

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment