Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

APG guidance for touch devices #60

Closed
LJWatson opened this issue May 25, 2015 · 14 comments
Closed

APG guidance for touch devices #60

LJWatson opened this issue May 25, 2015 · 14 comments

Comments

@LJWatson
Copy link

We need to address the problem of design pattern accessibility on touch screen devices. Touch ATs intercept all touch gestures. Only click/blur/focus are handled reliably - and they're not sufficient to support interaction with complex widgets like trees.

We should look to include guidance that surfaces these shortcomings, and indicate which of the design patterns will/won't work on touch devices with ATs enabled.

@patrickhlauke
Copy link
Member

For context (as the above is paraphrasing my recent ranting on twitter), this relates to further thinking around this bug/comment I filed https://lists.w3.org/Archives/Public/public-pfwg-comments/2015AprJun/0005.html / https://www.w3.org/Bugs/Public/show_bug.cgi?id=28694

@stevefaulkner
Copy link
Contributor

Have set up a google spreadsheet to record compatibility issues with design patterns
https://docs.google.com/spreadsheets/d/1gN9oRZPdrJxLDNtT6nVO4fn7E7sn1061L9Xl3__slZ4/edit?usp=sharing

@patrickhlauke
Copy link
Member

A simple, concrete example: take an implementation of a "faked" slider like http://hanshillen.github.io/jqtest/#goto_slider - to my knowledge, this implementation follows the advice provided in http://www.w3.org/WAI/PF/aria-practices/#slider which defines the expected keyboard behavior (use of left/right arrow, home/end, page up/down, tab to get in/out of the slider widget itself). Now, as with many other patterns, this: a) assumes that it's working fine for mouse users (so nothing special is defined/mentioned about how a mouse user would need to interacts with it) and b) defines behavior for keyboard users, which the author is expected to include by hooking into keydown/keypress events, check for those defined expected keystrokes, and trigger relevant JS in response.

This simple slider is unusuable for users with touchscreen interfaces and AT. Simply taking the example of iOS/VoiceOver, a user is able to set focus to the slider "knob". VoiceOver announces (because of the role="slider") that it's "adjustable" and that a user can "swipe up or down with one finger to adjust the value". However, doing this has no effect - as the actual functionality that fakes this is attached via keydown events listening for arrow/home/end/etc, and swiping up/down does NOT trigger these (VoiceOver does not fire fake keyboard events that would rattle those event handlers, and I believe it shouldn't either - though it may have to come to this if no better solution is available). The naive solution here would be to also layer touch/gesture events on top of the keyboard listener events, but this will not work because VO will intercept touch events/gestures (similar to role="document" on desktop/keyboard/AT scenarios, and there's no sensible way to force touch/AT like VO to go into role="application" mode). The only way, if there WERE touch/gesture listeners, for a user would be to perform an override/pass-through gesture in VO (double-tap and hold, then swipe) to pass actual swipe gestures to the underlying page/JS, bypassing VO...but this, in my mind, is the philosophical equivalent of saying "keyboard users can just use mouse keys to interact with this" - it's a crutch, not a solution (and not all platforms have this sort of pass-through gesture...for instance, I don't believe Android/Talkback offers anything like this).

In short, even with such a simple widget as a slider, following the current advice/design pattern (which tacitly assumes a false dychotomy of mouse/keyboard only) results in a construct that simply will not work for touch/AT users.

@patrickhlauke
Copy link
Member

Another example of a construct that follows ARIA design patterns and fails for touch/AT: http://hanshillen.github.io/jqtest/#goto_menubar which follows http://www.w3.org/WAI/PF/aria-practices/#menu - specifically, if a menu item has a submenu, the pattern currently expects

If the menu item with focus has a submenu, pressing Enter, Space, or the right arrow key opens the submenu and puts focus on the first submenu item.

With iOS/VO, I can open the first level of each menu, set focus to any item, but if the item also has a submenu, i cannot trigger it (in Hans' implementation), as this will NOT send Enter/Space/right arrow, but simply a click event

@patrickhlauke
Copy link
Member

A third one, for good measure: http://hanshillen.github.io/jqtest/#goto_tree which implements the TreeView http://www.w3.org/WAI/PF/aria-practices/#TreeView

With iOS/VO, I can set focus to individual visible nodes (and also hidden nodes, but this seems a shortcoming in Hans' implementation, rather than intention), but I cannot open/expand a tree branch, as I cannot fire

Right arrow key expands a closed node, moves to the first child of an open node, or does nothing on an end node.

I can only fire a click event at the actual focused node, which only triggers what I assume is the mouse-based equivalent coded for

Enter key performs the default action on end nodes.

@patrickhlauke
Copy link
Member

Additionally, when I originally said that for touch/AT situation we can only currently rely on focus/blur/click ... the situation is actually even murkier, as some OS/Browser combos don't send focus/blur when you'd expect it. see http://patrickhlauke.github.io/touch/tests/results/#mobile-tablet-touchscreen-assistive-technology-events

@patrickhlauke
Copy link
Member

Quick additional note, from testing: iOS devices with a paired bluetooth keyboard do not fire key events (whether VO is enabled or not) unless focus is in a text input/textarea (so patterns reliant on key events will fail even for "keyboard" users on iOS); Android with a paired keyboard does act mostly like a traditional "desktop" device with keyboard - no keys are intercepted, fired as normal (in standard Browser or Chrome; some browsers, like Firefox/Android, do provide additional shortcut keys when running Talkback, and these are intercepted)

@mraccess77
Copy link

In my experience custom ARIA combo boxes are also problematic with mobile devices such as iOS and Android. Listboxes faired better in my testing. My presentation at CSUN indicated the same challenges with sliders, menus, and trees as discussed above. Spinboxes also were not supported.

@DavidMacDonald
Copy link

I'm totally on board with this discussion and we'll be looking at it carefully with the Mobile Task Force

@jnurthen jnurthen added the APG label Jul 8, 2015
@hshepherd
Copy link

It seems like interactions with role="slider" elements are still an issue in iOS Voiceover even when using the recommended aria attributes (aria-valuemin, aria-valuemax, aria-valuenow). Are there any plans to revisit this?

@patrickhlauke
Copy link
Member

interactions with ARIA menus that have submenus are still an issue in iOS/VO too. generally, over the last few months, i've actually started advising clients NOT to implement certain ARIA patterns as they fall apart on mobile.

@ZoeBijl
Copy link

ZoeBijl commented Apr 29, 2016

This issue was moved to w3c/aria-practices#8

@ZoeBijl ZoeBijl closed this as completed Apr 29, 2016
@intimatehetero

This comment was marked as spam.

@patrickhlauke

This comment has been hidden.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

9 participants