Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Screen reader: orca stays in focus mode when using the show hover command #95032

Closed
jvesouza opened this issue Apr 12, 2020 · 19 comments
Closed
Assignees
Labels
accessibility Keyboard, mouse, ARIA, vision, screen readers (non-specific) issues bug Issue identified by VS Code Team member as probable bug linux Issues with VS Code on Linux verified Verification succeeded
Milestone

Comments

@jvesouza
Copy link

Issue Type: Bug

  1. Launch orca.
  2. Create a new js file in vscode and type the following code:
/** this is a test */
function test() { }
  1. Place your cursor over the word test on the second line.
  2. Press ctrl+k ctrl +i.
  3. Try to browse using up and down.

Orca doesn't read anything.

This appears to be because the orca remains in focus mode. If we switch to browse mode by pressing orca_key + a, we can read the displayed text using the arrows.
CC @isidorn
VS Code version: Code - Insiders 1.45.0-insider (44c5185, 2020-04-10T18:15:17.029Z)
OS version: Linux x64 5.6.3-arch1-1

@isidorn
Copy link
Contributor

isidorn commented Apr 13, 2020

Reading the conent of the hover works nicely on my macOS with Voice Over and also on Linux with Orca. VS Code passes focus to the hover and screen readers read the content nicely.
I can not browse through the conent via up and down as the editor hvoer gets hidden as soon as a key is pressed.

@jvesouza do you get the conent of the hover read when you press ctrl+k ctrl+i?
@joanmarie when does orca auto switch to browse mode? Since that looks like a missing piece here (I guess Orca than steals keybinding from VS Code and the hover would not get closed)

@isidorn isidorn added accessibility Keyboard, mouse, ARIA, vision, screen readers (non-specific) issues under-discussion Issue is under discussion for relevance, priority, approach labels Apr 13, 2020
@microsoft microsoft deleted a comment from vscodebot bot Apr 13, 2020
isidorn added a commit that referenced this issue Apr 13, 2020
@isidorn
Copy link
Contributor

isidorn commented Apr 13, 2020

I have pushed a commit that all VS Code hovers start using a role tooltip.
In the aria descriptoin I see it is recommended that we use that in combination with aria-describedby but we currently pass the focus to the hover widget which I think should also be fine.

@jvesouza you could try it out with Tuesday's vscode insiders
@jonamarie let me know what you think
fyi @alexdima

@jvesouza
Copy link
Author

jvesouza commented Apr 13, 2020

@jvesouza do you get the conent of the hover read when you press ctrl+k ctrl+i?

Yes, and I need to press the escape key to return to the editor.
For me it would be interesting if we could browse the displayed text using the arrows.

@isidorn
Copy link
Contributor

isidorn commented Apr 13, 2020

@jvesouza ok thanks for claryfing. My fix with the roles might potential fix this, but let's wait and see what @joanmarie says

@joanmarie
Copy link

@joanmarie when does orca auto switch to browse mode? Since that looks like a missing piece here

Orca's intent is to remain in focus mode when in a descendant of an element with role='application'. The belief being that within a web app, the author is expected manage keystrokes. Orca has no idea what you (the app developer) intend the user interaction to be within your custom app/widget. So if memory serves me, Orca wouldn't switch to browse mode automatically within VSCode.

Users can always toggle into browse mode manually when the authors have not provided adequate navigation. If you want this to happen automatically, perhaps it would make sense to move the single application role to the relevant children in which VSCode navigation should take precedence.

(I guess Orca than steals keybinding from VS Code and the hover would not get closed)

Orca doesn't steal Escape, Return/Enter, or Tab even in browse mode.

I've not yet tried your changes. But tooltips are typically not navigable or focusable. Mind you, if VSCode gave focus to a tooltip and that tooltip were not a descendant of an element with the application role, Orca might switch to browse mode. Mind you I might have to make a change or two in Orca for browse mode to work in a tooltip because, again, tooltips in (at least) native apps were never meant to be keyboard navigable.

If the tooltip is so large and/or detailed as to warrant navigation inside of it, I wonder if some other role might make sense.

All of this said, let me update my build and give it a spin (in a little while).

@isidorn
Copy link
Contributor

isidorn commented Apr 13, 2020

@joanmarie ok. Also note that we are having some discussions if VS Code should use the role='application' or not here #94847

@jvesouza
Copy link
Author

I have pushed a commit that all VS Code hovers start using a role tooltip.
In the aria descriptoin I see it is recommended that we use that in combination with aria-describedby but we currently pass the focus to the hover widget which I think should also be fine.

I did a test but it continues with the same behavior, it does not automatically switch to browse mode.
In addition, after I press ctrl + k ctrl + i and manually switch to browse mode, I cannot read what was displayed using the arrows.

@isidorn
Copy link
Contributor

isidorn commented Apr 14, 2020

@jvesouza thanks for trying it out.
I have changed that the role application is set on an html element on level lower than before (I think this will not fix it, but might help).

@joanmarie let me know if you have advice, thanks!

@joanmarie
Copy link

@isidorn:

  1. Orca currently doesn't think tooltips should be navigable. Tooltips are not expected to contain so much text that they need to be navigated within. They are transitory in nature. They can be there unexpectedly (user can't see where the mouse pointer is, but it just so happens to be over an object with a tooltip and thus the tooltip is now showing). And they can appear in unexpected places (See item 3). For all of these reasons, I don't think they are something users should be able to accidentally wander into in browse mode. So Orca is actively trying to prevent that. Stopping that prevention is not something I think is a good idea. (Though I have a candidate compromise/idea at the end of this comment.)

  2. The code editor is a textarea which makes it a focus mode widget independent of whether and where the application role is used. Just because a tooltip is showing doesn't mean Orca should update focus and switch to browse mode. Who's to say that's what the user wants?

  3. While I've not yet looked, there may be an issue depending on where the thing winds up in the accessibility tree. If it gets tacked on to the very end, even if Orca allowed it to be accessed via browse mode, the user might have to arrow a lot to go find it. Browse mode presents everything in the current object first. So if you press Down Arrow, first Orca would:

    1. proceed through the entire rest of the text in the file.
    2. Look in the accessibility tree for the next object with something presentable and let the user progress through all of that
    3. Rinse and repeat until finally the tooltip is located.

    This would be the case regardless of the role.

What about.... 🤔

If, when the user used the keyboard commands in screen reader optimized mode to cause the popup to appear, VSCode gave the popped-up object focus? While I don't like the idea of being able to browse into and out of a tooltip, I would happily modify Orca to treat a focused tooltip as being browse mode container. When the user pressed Escape, VSCode would then return the user to the textarea (it would need to claim focus).

Thoughts?

@webczat
Copy link

webczat commented Apr 15, 2020

note that there is a shortcut ctrl+k ctrl+i and I believe it is intended to focus on tooltip? so in that case it leaves the editor and should be in browse mode shouldn't it?

@joanmarie
Copy link

note that there is a shortcut ctrl+k ctrl+i and I believe it is intended to focus on tooltip? so in that case it leaves the editor and should be in browse mode shouldn't it?

Looks like there is a focus event indeed. Lemme dig some more.

@joanmarie
Copy link

Ok, I just updated my build. The thing which claims focus:

  • Is a div
  • Lacks any ARIA role
  • Appears to a descendant of an element with the application role

The third thing seems to be why Orca isn't switching to browse mode.

I can update Orca to ignore that third thing if the div which claims focus has the tooltip role. Which from the discussion above, I thought it would.

@isidorn: The focus event I'm getting is coming from the monaco-editor-hover. The tooltip role is applied to its child monaco-editor-hover-content. Could you make it so that the object which claims focus has the tooltip role?

@isidorn
Copy link
Contributor

isidorn commented Apr 17, 2020

@joanmarie sorry for the slow response, this somehow fell through my fingers.
I have changed that now the role tooltip is on the monaco-editor-hover element.
So I believe we should be good. Let me know if you try it out so I go ahead and potentially close this one. Thanks!

@isidorn isidorn added bug Issue identified by VS Code Team member as probable bug and removed under-discussion Issue is under discussion for relevance, priority, approach labels Apr 17, 2020
@isidorn isidorn added this to the April 2020 milestone Apr 17, 2020
@joanmarie
Copy link

@isidorn No worries. I have to confess I'm sometimes losing track of which VSCode issues still require my input.

So the change you made does make it possible for me to cause Orca to automatically switch to browse mode for the tooltip. Neat! I've not yet pushed that change to Orca master because I noticed something as a side effect: Right now, it is possible for the user to accidentally arrow out of the toolitp and then wind up in an unexpected place. So I'm still working on this.

Related aside: Any chance we could have a label for this project like assigned-to-orca (or anything I could search on to quickly find what I still need to do for you/vscode)?

@isidorn isidorn added the linux Issues with VS Code on Linux label Apr 17, 2020
@isidorn
Copy link
Contributor

isidorn commented Apr 17, 2020

Ok then closing this.
@joanmarie We can introduce a label of course. But how about the following query https://github.com/microsoft/vscode/issues?q=is%3Aopen+label%3Aaccessibility+label%3Alinux

It is label accessibility and label linux. From now I am will be more carefull to add label linux when it is orca specific. What do you think?

@isidorn isidorn closed this as completed Apr 17, 2020
@joanmarie
Copy link

It is label accessibility and label linux. From now I am will be more carefull to add label linux when it is orca specific. What do you think?

Works for me! Thanks!!

@joanmarie
Copy link

@jvesouza @webczat. I believe I have this working in Orca master now.

Some basic usage notes:

  • When VSCode's tooltip claims focus (requires @isidorn's change made earlier today), Orca will announce the tooltip and switch to browse mode.
  • If you arrow outside of the tooltip, Orca will return to focus mode -- unless you are in sticky browse mode.
  • Orca will announce when you are leaving the tooltip. This was done because (IMHO) wandering into and out of tooltips in an application can be confusing if you're not told this is what is happening. Right now, the tooltip messages are being treated the same as those for panels. So if you find the tooltip leaving message bothersome, you should be able to disable it via Orca's speech preferences for VSCode via panel announcement.
  • If you wander outside of the tooltip accidentally, and don't have sticky browse mode enabled, you can just toggle browse mode back on and then arrow immediately back to it.

Please let me know if this works for you (again, once @isidorn's change wind up in an insiders build) and what you think. Thanks!

@jvesouza
Copy link
Author

For me it is working exactly as I expected. It was great, thanks.

@isidorn
Copy link
Contributor

isidorn commented Apr 20, 2020

@jvesouza @joanmarie thanks a lot!
Adding verified label.

@isidorn isidorn added the verified Verification succeeded label Apr 20, 2020
@github-actions github-actions bot locked and limited conversation to collaborators Jun 1, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
accessibility Keyboard, mouse, ARIA, vision, screen readers (non-specific) issues bug Issue identified by VS Code Team member as probable bug linux Issues with VS Code on Linux verified Verification succeeded
Projects
None yet
Development

No branches or pull requests

4 participants