Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

NVDA doesn't toggle browse mode off when focus is moved to an interactive list #13221

Closed
kloots opened this issue Jan 8, 2022 · 28 comments · Fixed by #13357
Closed

NVDA doesn't toggle browse mode off when focus is moved to an interactive list #13221

kloots opened this issue Jan 8, 2022 · 28 comments · Fixed by #13357
Milestone

Comments

@kloots
Copy link
Sponsor

kloots commented Jan 8, 2022

I believe I have discovered a regression in NVDA. @jcsteh can correct me if I am misremembering, but I recall NVDA used to toggle browse mode off if focus was moved to an HTML list made interactive with tabindex and keyboard shortcuts implemented in JavaScript. Reason I remember this: while I was working at Twitter I made the timeline an interactive list so that customers could use keyboard shortcuts provided by Twitter to navigate their home timeline. And I recall working with @jcsteh on ensuring this worked in NVDA. Interestingly, the same code example I provided in this bug report works fine in JAWS in that it toggles browse mode off when focus lands on an ARIA listitem instance.

Steps to reproduce:

  1. Start NVDA
  2. Open https://codepen.io/toddlr/pen/mdBjoJV
  3. Use tab-key navigation to move focus to the first ARIA listitem in the ARIA list

Actual behavior:

NVDA doesn't toggle browse mode off

Expected behavior:

NVDA should toggle browse mode off so the up/down arrow keys can be used to navigate (move focus) between listitems in the list

System configuration

NVDA installed/portable/running from source:

NVDA version: 2021.3.1

Windows version: Windows 10

Name and version of other software in use when reproducing the issue:

Other information about your system:

Other questions

Does the issue still occur after restarting your computer?

Yes

Have you tried any other versions of NVDA? If so, please report their behaviors.

No

If NVDA add-ons are disabled, is your problem still occurring?

I'm not using any add-ons

Does the issue still occur after you run the COM Registration Fixing Tool in NVDA's tools menu?

Yes

@jcsteh
Copy link
Contributor

jcsteh commented Jan 8, 2022 via email

@kloots
Copy link
Sponsor Author

kloots commented Jan 11, 2022

@jcsteh I know lists are not considered keyboard navigable widgets. That said, I spent the weekend thinking about it and I don't yet have a clear understanding of the practical risks to the user. Especially, if the heuristics used were:

  • The list itself has a tabindex (either 0 or -1)
  • Each listitem has an aria-label or aria-labelledby
  • The list uses aria-activedescendant or the roving tabindex pattern to manage focus

If those were the signals in HTML and either the list or listitem has focus, what would the practical downside be of toggling browse mode off?

@jcsteh
Copy link
Contributor

jcsteh commented Jan 12, 2022 via email

@michaelDCurran
Copy link
Member

@kloots Can you please clarify why you can't use role="option" and role="listbox"? I'm sure there is a good reason, but it would be good if it were clearly specified on this issue.
Is it because you know that a screen reader will not render the entire list in browse mode if you do?
I'm not too sure I'm that happy specifically implementing an exception for a single use case here. Also @jcsteh has pointed out a valid concern, and we don't really know how much that is used in the wild.
Of course, the screen reader user always has the option of toggling browse mode off, or at least quicknav keys, if they want to use your specific shortcut keys here.

@kloots
Copy link
Sponsor Author

kloots commented Feb 14, 2022

@michaelDCurran the short answer is role="option" will result in each message being rendered as an atomic element and therefore its content will no longer be accessible in browse mode. (It'll have the same negative impact for users of VoiceOver on Mac OS). And browse mode support is important considering how many Slack messages have rich and/or interactive content.

I've taken the time to write up a doc detailing all the nuances of this problem for us at Slack including:

  • Our design goals/requirements
  • The limitations of the existing standards
  • The limitations of AT implementations

I hope it makes clear how nearly every existing option has a downside and how the changes I am proposing in this ticket would help.

@MarcoZehe
Copy link
Contributor

I support this proposal. Another example is the downloads manager in MS Edge. Press CTRL+J from any web page, and you land in the exact same pattern. You can then not press Enter on an item until you turn off browse mode. These list items, too, have interactive child elements. JAWS switches to forms mode seamlessly here, making it much more intuitive for beginners and less tech-savvy people. NVDA is unnecessarily complicated and unintuitive here.

@kloots
Copy link
Sponsor Author

kloots commented Feb 15, 2022

Thanks for the vote of confidence @MarcoZehe. Nearly every single list in Slack is a list composed of list items with interactive child elements. So having the screen reader support this pattern would be massively helpful.

@jcsteh
Copy link
Contributor

jcsteh commented Feb 15, 2022

the short answer is role="option" will result in each message being rendered as an atomic element and therefore its content will no longer be accessible in browse mode.

Is that actually true, though? Consider this:

data:text/html,<button>before</button><div role="listbox"><div role="option" tabindex="0"><div role="document" tabindex="0">Some message

The document is still included in the a11y tree and it's still possible to tab into it. NVDA doesn't switch to browse mode here, which seems like a different bug that would have to be fixed in order for that to work.

The challenge is that you can equally argue that it's not appropriate for listitem to be used for interactive content and that option is more appropriate for that reason. Either way, there are bugs to fix, but as documented in the spec, it's reasonable to argue that option is more appropriate.

@jcsteh
Copy link
Contributor

jcsteh commented Feb 15, 2022

FWIW, I'm not saying we shouldn't push for a change to the spec if that proves to be necessary, but it seems to me that there are semantic inconsistencies/problems/bugs either way.

@kloots
Copy link
Sponsor Author

kloots commented Feb 15, 2022

@jcsteh I didn't consider embedding an ARIA document inside an option. I assumed @michaelDCurran was suggesting we just use option. That said, if NVDA isn't switching to browse mode with an ARIA document inside an option it could be the same bug as 12896. So, no matter what Slack is in a holding pattern here.

@kloots
Copy link
Sponsor Author

kloots commented Feb 15, 2022

@jcsteh and @michaelDCurran I just created a new example to examine the possibility of the suggestion of using the ARIA listbox pattern. Unfortunately, this doesn't work at all on the Mac as all children of each ARIA option are presented as text — including the interactive children. Funny enough, I now recall attempting this years ago while I was working at Twitter and gave up on this solution for the same reason.

@michaelDCurran
Copy link
Member

I would much prefer to go with option/document as @jcsteh suggests, but also commit to solving #12896.
That really seems the right approach which keeps us as close to standards as possible.

@kloots
Copy link
Sponsor Author

kloots commented Feb 15, 2022

@michaelDCurran going with option/document will not work for us due to lack of compatibility on the Mac (per this comment) and changes to VoiceOver requiring an uphill battle as well as upgrades to Mac OS.

@kloots
Copy link
Sponsor Author

kloots commented Feb 15, 2022

Regarding the standards argument: the heuristics I suggested are based on standards for keyboard navigation. And per my points in that comment, I'm only suggesting the mode switch happens when DOM focus lands on a listitem made interactive via that pattern. It seemed like a pretty strong signal for switching browse mode off to me.

@jcsteh
Copy link
Contributor

jcsteh commented Feb 15, 2022 via email

@kloots
Copy link
Sponsor Author

kloots commented Feb 15, 2022

To be honest, I don't understand why what I'm suggesting is being considered a spec violation. Seems more like a gray area to me. And even if it is a spec violation, I also don't yet understand the realistic downsides to what I am suggesting given the specific heuristics I've outlined in this comment.

Anyway, I've done my very best to outline my reasoning in the doc I wrote. Just today we had another customer complain about the verbosity of the "application" announcements within Slack. Personally, I think the benefits of my suggestion outweigh the spec violation. If NVDA decides to hold the line here, I understand. If that's the case then we'd definitely need issue 12896 fixed so we can switch to a single-column ARIA grid as that is our only other practical option. I just don't know if actual users are going to like that.

@michaelDCurran
Copy link
Member

@kloots Based on the thorough and healthy discussion here, I am happy enough to go with the listItem approach.
We will switch to focus mode if a listItem gets focus, but only if the list itself is also focusable. I.e. the list has at least tabindex of -1.
thank you for providing the extra information and reasoning here.

@kloots
Copy link
Sponsor Author

kloots commented Feb 15, 2022

@michaelDCurran wow! Thank you so much for hearing me out. This is (hopefully) the end of a very long quest for me that started back when I was at Twitter in 2014.

@MarcoZehe
Copy link
Contributor

Just to add to the data:

@jcsteh wrote:

The document is still included in the a11y tree and it's still possible to tab into it. NVDA doesn't switch to browse mode here, which seems like a different bug that would have to be fixed in order for that to work.

Yeah, but like on the Mac, with JAWS and a Chromium-based browser, the contents in your example is not accessible. So besides VoiceOver on the Mac, there's another hard-to-change contender here where the listbox/option approach simply wouldn't work.

@geoffshang
Copy link

geoffshang commented Feb 15, 2022 via email

michaelDCurran pushed a commit that referenced this issue Feb 20, 2022
…re focusable and gain focus (#13357)

Fixes #13221

Summary of the issue:
In the Microsoft Edge Downloads manager (CTRL+J), as well as in the channel and DM outputs of the Slack app, HTML lists and list items are used to display the items. These are made focusable via either aria-activedescendant, or the rowing tabIndex pattern. The reason is that for compatibility, these can be used to host rich content, such as whole sub documents or controls. Until now, NVDA would not automatically switch to focus mode when focus lands on such a focusable list item.

Description of how this pull request fixes the issue:
This PR adds a rule for enabling passthrough mode if a list item that is read-only, has state focused, and at the same time its paarent is of role list and is focusable.
@nvaccessAuto nvaccessAuto added this to the 2022.1 milestone Feb 20, 2022
@NeilWix
Copy link

NeilWix commented Aug 28, 2022

I think I encountered a related bug/issue: when tabbing to a button that resides inside a listitem role, focus mode is not triggered automatically.
Here is a demo https://codepen.io/WW3/pen/KKoENOW?editors=1010
Steps to reproduce:

  1. Navigate to the above codepen
  2. Switch to debug view
  3. Open NVDA (I am running 2022.2.2 on win 10)
  4. Tab through the buttons, pressing enter or space doesn't trigger the handlers
    BTW, on Jaws focus mode is auto triggered when tabbing to these buttons

@jcsteh
Copy link
Contributor

jcsteh commented Aug 28, 2022

I think I encountered a related bug/issue: when tabbing to a button that resides inside a listitem role, focus mode is not triggered automatically.

That is expected behaviour. Links and buttons are controls that do not require focus mode, so focus mode is not enabled for them.

@NeilWix
Copy link

NeilWix commented Aug 29, 2022

I think I encountered a related bug/issue: when tabbing to a button that resides inside a listitem role, focus mode is not triggered automatically.

That is expected behaviour. Links and buttons are controls that do not require focus mode, so focus mode is not enabled for them.

Got it, but that's exactly the point in the shared demo. Users tab to the [html] buttons, hit space or enter and nothing happens. Only when the mode is toggled (NVDA+space) the buttons can evoke the attached event handler.

@jcsteh
Copy link
Contributor

jcsteh commented Aug 29, 2022

When in browse mode, space/enter send click events, not key events. I'm guessing this is the source of the problem. Among other things, this is an unfortunate necessity due to the number of broken controls on the web. Even putting that aside (and assuming I'm correct about the cause), note that you'd have the same problem if a user tried to use this with a touch screen screen reader.

@NeilWix
Copy link

NeilWix commented Aug 29, 2022

When in browse mode, space/enter send click events, not key events. I'm guessing this is the source of the problem. Among other things, this is an unfortunate necessity due to the number of broken controls on the web. Even putting that aside (and assuming I'm correct about the cause), note that you'd have the same problem if a user tried to use this with a touch screen screen reader.

In this particular case, it's a drag and drop component which has dedicated events for pointing devices (touch and mouse). Hence, the keyboard events are needed to support keyboard users. While the code works for sighted keyboard users, it doesn't work for NVDA users (NP for Jaws and VO users).

@jcsteh
Copy link
Contributor

jcsteh commented Aug 29, 2022

It's interesting that JAWS switches to forms mode for these buttons. My understanding was that it had similar rules for buttons and links to NVDA, but apparently not in this case. Either way, I think this should be split into a separate issue for further discussion/diagnosis.

@NeilWix
Copy link

NeilWix commented Aug 29, 2022

Sure, thanks

@alexstine
Copy link

I am having a similar issue on the above PR I am testing. It seems that NVDA switches to edit mode if it comes in contact with the list but if a button happens to exist, it forces itself back to browse mode. This is a problem because then the keyboard navigation commands to navigate the list are not sent to the application and browse mode allows me to skip to the next element on the page instead of staying within the list. It is painful that there is not better docs on screen reader interactions. I expect that if I use the correct ARIA roles, even if an element is a button, it will not switch modes. The same issue exists with multiple other ARIA roles such as the treegrid role. If I navigate to an options menu in a treegrid, NVDA automatically switches to browse mode which messes up the ability for my arrow keys to be sent to the application. The only work around I can imagine is using invalid HTML to fake a button so the screen reader quits switching modes. There has got to be a better solution and fighting to get this right cross-platform is tiring.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
8 participants