canvas accessibility

[stevefaulkner]

This area needs further exploration. Using fallback content (the canvas DOM sub-tree) should allow screen readers such as VoiceOver to interact with the content. We've seen mixed results with the iOS devices we've tested.

On touch devices for interactive UI you will need to route events on the canvas to the appropriate interactive elements in the sub DOM.

May be helpful: http://www.paciellogroup.com/blog/2015/02/html5-canvas-sub-dom/

[stevefaulkner]

an issue with use of -webkit-overflow-scrolling:touch was mentioned by @mjohnston http://www.paciellogroup.com/blog/2015/02/flipboard-react-canvas-accessibility/#comment-2143 any test html files available?

[mjohnston]

Yes - will drop a test case in here later today. Another issue I've seen with VoiceOver is that the focus ring is not drawn correctly for canvas sub-DOM. No matter the size or position of an element inside the canvas, the focus ring is drawn at the very top of the screen. Has this been your experience as well?

[mjohnston]

See: https://gist.github.com/mjohnston/023bb3d89aa7a810d462

[stevefaulkner]

thanks for the test file, will look into it.

No matter the size or position of an element inside the canvas, the focus ring is drawn at the very top of the screen. Has this been your experience as well?

yes, as drawFocusIfNeeded() is not implemented cross browser yet. Try http://codepen.io/stevef/pen/WbMGgE in Firefox to see drawFocusIfNeeded() working, also works in chrome behind flag

[stevefaulkner]

I created a pen of the test file you provided http://codepen.io/stevef/pen/jEZMXY

[mjohnston]

Scribbling some notes down here just for reference.

Canvas accessibility is something we've been looking into since React Canvas was just an idea. Many approaches have been tried. In particular, the canvas sub-DOM approach seems like the most practical solution. But with this approach you lose the ability to position focusable elements (see Steve's previous comment about drawFocusIfNeeded). This breaks the common use case of dragging 1 finger across the screen to discover UI elements. We wanted a better solution.

The approach we now use on flipboard.com is to have an invisible DOM overlaid on top of the drawing canvas. To be clear this is NOT a fallback DOM with different content. This DOM represents the same UI elements (buttons, links, text, images) as what is drawn inside the canvas.

It sounds like it may be hard to keep this DOM in sync with the canvas UI. But the React Canvas layout system keeps a tree of layers corresponding to the currently displayed elements on screen. This tree can easily be converted into DOM, and React handles diffing the DOM when the layer tree is updated.

It's not as easy to generalize this approach to every use case, however, which is why it has not made it into React Canvas core. But we're working to make that happen. In particular, scrolling lists of content present a major challenge.

The WKWebView in iOS 8 changes the behavior for navigating scrollable content with VoiceOver. Prior to WKWebView, a <div> with overflow: auto + -webkit-overflow-scrolling: touch was completely accessible with VoiceOver - meaning a 3-finger swipe up or down would scroll the content one page at a time. With WKWebView, the 3-finger swipe gesture is broken though the list can still be advanced one item at a time. This greatly hinders the usability of scrollable content.

It is possible to get the pre-WKWebView 3-finger swipe behavior but only if the entire window is scrollable. This presents many challenges though, while maintaining the fidelity of the UI for non-VoiceOver users. Add to this the fact that many apps have not opted in to WKWebView on iOS 8 (eg. Chrome).

We've come up with an approach on flipboard.com that allows the main scrolling timeline to be scrolled either via an inline scroll container or window scrolling, without affecting the UI for sighted users. I won't go into much detail here as we're still working to refine this implementation.

Note: this issue is not exclusive to canvas rendering. It affects any site using inline scrollable elements. See twitter.com on iPhone as an example.

Android is an entirely different story, though in many ways a much better one. Since Lollipop, the TalkBack feature has improved significantly and it's now arguably on par or better than VoiceOver on iOS. Vision-impaired users are able to interact with content in almost the same way as fully-sighted users using two fingers instead of one for scrolling and pinching. Also, the one finger drag across the screen behaves the same way as VoiceOver.

There is still much room to improve on the accessibility front for both Android and iOS. But I thought it would be helpful to jot down the state of things as they stand.