Slider value only spoken when keyboard focused

[zepumph]

I noticed this in the sonification wrapper and on master. When using the keyboard or mouse to move a slider that is highlighted with keyboard focus, the Screen Reader will state the change, but if you use the mouse on the other slider when the first slider still has focus on it, then it will not alert. It will also not alert at all when just using the mouse with no highlight anywhere.

[zepumph]

Witnessed on FF and Chrome with NVDA.

[jessegreenberg]

Thanks @zepumph, this is actually a known problem with our current parallel DOM implementation. The screen reader requires CSS bounds to handle mouse and touch interaction. Since the accessible DOM is hidden, it doesn't know that we are touching the slider. This is the exact same reason that our sims are not accessible on mobile VoiceOver.

There is hope that someday we can support this kind of interaction on these platforms. But it would take some investigation on how to do it.

I would say that it is up to @emily-phet and @terracoda for whether we need a workaround for sliders. We could potentially look into an alternative solution so that it always comes through (again, would likely mean not using <input type="range">, trading it for something custom). But we haven't expected all interaction descriptions to still come through when using a mouse/finger for now. @emily-phet @terracoda could you please comment?

[zepumph]

Is there a way that we could fix this internally? Perhaps in accessibleInputListener, we are able to transfer over (to the PDOM) mouse events correctly similar to how we do that for keyboard events? Maybe I don't understand accessibility enough, and that is unrealistic or not thought through.

[jessegreenberg]

That is a really interesting idea, I hadn't thought of that. Scenery's input listener would need to create a mock mouse/touch event whose target would the the PDOM element. Is that possible? Not sure if scenery can do that, and I don't know if DOM allows fake mouse events.

[jessegreenberg]

Maybe something like this? https://developer.mozilla.org/samples/domref/dispatchEvent.html

[jessegreenberg]

@zepumph, this isn't working yet, but I am wondering about something like this? https://jsfiddle.net/mk63j417/5/

In HSlider drag handler, we would create an event and dispatch it to the accessible DOM element. "click" event might be why it isn't working, maybe we can use another event. But when I test with NVDA, i still don't hear the aria-valuetext.

[jessegreenberg]

https://developer.mozilla.org/en-US/docs/Web/API/MouseEvent/initMouseEvent I see a big 👎 :( maybe there is a better approach. But I think we should hear from @emily-phet before spending more time as this will be an investigation.

[terracoda]

@jessegreenberg and @zepumph this is a very interesting issue. When I first imagined how the PDOM would be implemented, I thought it would be directly on top of the visual sim, but in reality it is off screen, off to the side.

I agree, there is no easy answer at the moment.

[zepumph]

I looked into this a bit more. @jessegreenberg it seems like var document.createEvent( "MouseEvents" ).initMouseEvent is deprecated. I'll keep poking around.

[samreid]

Since the accessible DOM is hidden, it doesn't know that we are touching the slider.

When scenery starts processing inputs on the 2nd slider, can we just set the focus to the second slider programmatically?

[zepumph]

I would think so, we can always manually set focusHighlight to a node. see Accessibility.setFocusHighlight().

[samreid]

Even better to set focus to the DOM element with domElement.focus() then the highlight should move automatically and then the new slider is set up to receive events. https://developer.mozilla.org/en-US/docs/Web/API/HTMLElement/focus

[zepumph]

That sounds reasonable. I would want @jessegreenberg's opinion about bypassing the accessibility mixin there. Potentially they could do the same thing?

[jessegreenberg]

When scenery starts processing inputs on the 2nd slider, can we just set the focus to the second slider programmatically?

Ill give it a shot! We would make the focus highlight invisible while interacting with the slider with mouse/keyboard so it doesn't interfere for all interaction. The question is whether or not focus is enough to read the aria-valuetext or if the screen reader requires a keydown or input event to read the new value.

[zepumph]

We would make the focus highlight invisible while interacting with the slider with mouse/keyboard so it doesn't interfere for all interaction.

I guess we probably want two types of keyboard focus. The normal way and a 'silent' way.

The question is whether or not focus is enough to read the aria-valuetext or if the screen reader requires a keydown or input event to read the new value.

Too true. It also seems like we want to make sure that this doesn't mess up the current sound of descriptions.

[jessegreenberg]

When focusing on the down input event, the slider works great on VoiceOver! I am hearing the aria-valuetext when I change the value.

[jessegreenberg]

In order for it to work with NVDA, we need to focus the slider on drag to hear the updated value (makes sense because aria-valuetext is always read on focus). This seems like a reasonable workaround.

[zepumph]

I wonder how important this problem is. I tried to tackle it today and found that you can't drag a thumb with the mouse when that slider is focused. @jessegreenberg you were right though, it sounded nice on NVDA to focus on drag when using the other slider
.

The reason I don't think that we should spend too much time on this is because the focusHighlight pops up. I'm not sure how to handle that, and I'm not sure that this is a priority. Unassigning for now.

[emily-phet]

@jessegreenberg and I discussed, and think this discussion should be moved to a11y-research repo.

[jessegreenberg]

This issue was moved to a11y-research, referenced in phetsims/a11y-research#69. Closing this issue since it will be discussed generally for all sims there.