Accessibility - Keyboard Navigation using a Screen Reader

[m158261]

These issues have been discovered while trying to navigate in Jupyterlab using keyboard navigation with a screen reader active. The screen reader used is Microsoft Narrator.

Problem

Issue no. 1 - New Launcher Icon label not descriptive

• When navigating to the new launcher button the screen reader announces "graphic". This should be solved using aria-labels.

Resolved - using caps lock + right arrow key solves this issue

Issue no. 2 - Unhelpful description when opening new launcher

• When opening a new launcher the screen reader announces "launcher close launcher". This is not descriptive of the action being done.

Issue no. 3 - Arrow key navigation between launcher tabs not as expected

• When screen reader is active, pressing right arrow keys steps through the launcher title (each individual letter) rather than moving to the next launcher tab. The launcher doesn't close when pressing enter on the close launcher button.

Resolved by this PR - #15347

Issue no. 4 - Cannot expand toggle arrows in Terminal/Debugger/Extension Manager sidebar widgets

• When navigating in sidebar widgets the toggle arrows are not expandable using keyboard keys.

Resolved - Pressing Ctrl + Enter opens the sub-menus.

Issue no. 5 - Unable to open menu items using enter/arrow keys

• Menu items in the top menu bar are not expandable using enter/space/arrow keys.

Resolved - Pressing Ctrl + Enter opens the sub-menus.

Issue no. 6 - Unable to select all cells using Ctrl + A in notebook

• When navigating cells in a notebook, Ctrl + A does not select all cells. It selects all elements in the document.

Issue no. 7 - Move cell up/down in notebook not working with keyboard commands

• This functionality is flaky when using a screen reader. It is difficult to focus on the cell to move it up/down.

Issue no. 8 - Navigation to cell buttons difficult

• It is difficult to access the buttons in the right of a cell using keyboard navigation. It is not intuitive, it can be currently accessed via arrow keys but you have to exit the cell, tab to switch kernel then use arrow keys.

Issue no. 9 - Pressing A in notebook does not insert a new cell above

• Pressing A does not insert a new cell when using the screen reader. The screen reader announces "unavailable".

Issue no. 10 - Pressing B in notebook does not insert a new cell below

• Pressing B does not insert a new cell when using the screen reader. The screen reader announces "unavailable".

Issue no. 11 - Pressing C/V does not copy/paste a cell in a notebook

• Pressing C/V does not copy and paste a cell when using the screen reader. The screen reader announces "unavailable" for V and "no next" for C.

Issue no. 12 - Pressing D+D in notebook does not delete a cell

• Pressing D+D doesn't delete a cell. It navigates you to the status bar.

Issue no. 13 - Pressing Y/M doesn't switch between code and markdown in a cell

• Pressing Y/M doesn't toggle between code and markdown. The screen reader announces "unavailable".

Issue no. 14 - Unable to open files/folders in file browser using keyboard

• When navigating through the file structure in file browser you cannot open folders or files using the keyboard. The screen reader announces "No primary action".

Additional context

This testing was done on a demo branch with improved accessibility features that can be found here. Please note you need jupyterlab and lumino to replicate this environment.

https://github.com/t03857785/jupyterlab/tree/demo
https://github.com/t03857785/lumino/tree/demo

[welcome]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! 🤗

If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively.

You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! 👋

Welcome to the Jupyter community! 🎉

[tonyfast]

this issue is great! thanks @m158261. any chance you could make bullets into checklist items so it will be easier to break this mega issue into smaller ones?

[m158261]

this issue is great! thanks @m158261. any chance you could make bullets into checklist items so it will be easier to break this mega issue into smaller ones?

Thanks @tonyfast, I've changed it to checkboxes.

[tonyfast]

Issue no. 9 - Pressing A in notebook does not insert a new cell above

on Orca, A takes me to the next clickable, if there is no next click then i hear "Unavailable". i have to hit the Orca modifier key and A to add a cell above. so i expect A shortcuts to be affected with the screen reader active. are things better if you use the Narrator modifier with A, Caps Lock + A or Insert + A?

[e218736]

Issue no. 9 - Pressing A in notebook does not insert a new cell above

on Orca, A takes me to the next clickable, if there is no next click then i hear "Unavailable". i have to hit the Orca modifier key and A to add a cell above. so i expect A shortcuts to be affected with the screen reader active. are things better if you use the Narrator modifier with A, Caps Lock + A or Insert + A?

@tonyfast using Narrator modifier with A still doesn't add a new cell above. Some of the other issues above have been edited as modifier combinations have the correct Functionality. Do you know if there is a combination to solve Issue no. 14 - Unable to open files/folders in file browser using keyboard? as using modifiers + Enter is not opening either

[e218736]

@tonyfast
For the remaining issues, a user must know to turn scan mode off manually (Narrator + Space) to enable functionality of keyboard shortcuts.
Should a user have to do this? If this is the accepted functionality should they be notified somehow that scan mode should be disabled?
Or when navigating through cells/files is it possible that scan mode can be automatically turned off?

In addition, screen reader does not read all punctuation for certain keyboard short cuts, this can be manually changed by pressing Narrator + Alt + Plus sign (+) or Minus sign (-). Again, is this accepted functionality or should all keyboard shortcut punctuation be read out to the user? possibly with the addition of aria labels

[tonyfast]

i actually misspoke. with orca, modifier+A toggles browse mode and focus mode. so screen reader users cannot effectively use the common A or B shortcuts. B and Modifier+B on orca traverse button elements.

For the remaining issues, a user must know to turn scan mode off manually (Narrator + Space) to enable functionality of keyboard shortcuts. Should a user have to do this?

i don't think they should have to do this. we definitely make sure they buttons are configurable, and maybe consider a default that would work against screen reader conflicts.

If this is the accepted functionality should they be notified somehow that scan mode should be disabled? Or when navigating through cells/files is it possible that scan mode can be automatically turned off?

it is hard to speak for all the screen reader literacies we may encounter, but currently adding a cell above and below is not specifically easy for screen reader users that rely on keyboard navigation.

In addition, screen reader does not read all punctuation for certain keyboard short cuts, this can be manually changed by pressing Narrator + Alt + Plus sign (+) or Minus sign (-). Again, is this accepted functionality or should all keyboard shortcut punctuation be read out to the user? possibly with the addition of aria labels

right now, keyboard shortcuts are announced through the title property i think? either way, we are not using aria-keyshortcuts that provides a standard for announcing keyboard actions to the screen reader. i don't have a reference example for notebooks, but there are others out there. that said, splitting title into aria-label and aria-keyshortcuts properties will be better at announcing specific key combinations properly. for + you may need to use an escape sequence like ' maybe there an escape for minus you could use.

https://developer.mozilla.org/en-US/docs/Web/Accessibility/ARIA/Attributes/aria-keyshortcuts#property_value_rules

[e218736]

@tonyfast
Hi Tony, thanks for the help on this. I will make a separate issue for keyboard shortcut punctuation and start working on adding aria-labels/keyshortcuts for them.

As for interaction with cells, I'll keep looking into Narrators scan mode. For the moment it looks as though disabling scan mode gives the best user experience for keyboard navigation and could look at alerting the user with guidance on using tools effectively

[tonyfast]

For the moment it looks as though disabling scan mode gives the best user experience for keyboard navigation and could look at alerting the user with guidance on using tools effectively

yea i think this a course of action. it will be hard to change the default keyboard options with so much history. i do think the best course of action we have is helping users become better screen reader users. this is consistent with the history of computational notebooks as a literacy too. it does mean that providing that experience would require a super excellent assistive tech experience. there is some work to do before we can say that.

[tonyfast]

we'll want to keep an eye on https://www.guidepup.dev/ for testing these features in playwright.