Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🗣️ Improve screen reader support for reactive output changes #2768

Open
3 tasks
fonsp opened this issue Jan 3, 2024 · 0 comments
Open
3 tasks

🗣️ Improve screen reader support for reactive output changes #2768

fonsp opened this issue Jan 3, 2024 · 0 comments
Labels
accessibility Accessibility improvements of the frontend, like screen reader support and internationalization frontend Concerning the HTML editor

Comments

@fonsp
Copy link
Owner

fonsp commented Jan 3, 2024

PR #2757 makes a first step to get Pluto closer to being usable keyboard-only with a screen reader.

When you edit and run code, it will start running, and when finished, the result is returned by updating the cell's output. Because this is asynchronous, this information is currently not surfaced with a screen reader. The solution is to use an "ARIA live region": you can declare a DOM element to be "live", meaning that any update that happens to it should be announced with speech.

This PR makes the <pluto-output> a live area.

  • The "polite" level is used: changes only get announced when the user is idle, not directly interrupting other things.
  • A label is added. If a cell has an assignee xyz = , it reads "Result of variable xyz:", otherwise "Result of unlabeled cell:".
  • The <pluto-output> is marked as "atomic": screen readers should always announce its full contents when anything changes. E.g. if the output changes from [1,2] to [1,2,3], don't just announce 3. It also means that the label will be read on any change.
  • The aria-relevant setting is all: additions, deletions, modifications.

Here is a screen recording (no audio but subtitles). Notice that the third cell runs reactively, and the new result gets announced.

Schermopname.2023-12-18.om.15.06.49.mov

Recorded on Chrome with MacOS voice-over enabled (default settings, dutch).

TODO:

  • What is the best way to label cells in the announced text? Because cells run reactively, in "random" order, it's important that users can keep a mental model of what is happening where. Maybe we generate short memorable labels? Maybe screen reader users will be encouraged to label every cell themselves? (But then you would have a hard time editing notebooks written by people who don't use a screen reader.) Maybe we can use GPT to generate labels?
  • I noticed that even with the aria-live="polite" setting, some announcements still interrupt others. E.g. in the recording above, I sometimes saw that the tiny delay between the two cells caused the first announcement to be interrupted. We could use the aria-busy setting to deliver the multiple updates in a batch (I think), but we also don't want to hold all updates until all cells completed running.
  • What if a cell is running for a long time? It's currently not possible to tell if you are waiting for running cells or not.
@fonsp fonsp added frontend Concerning the HTML editor accessibility Accessibility improvements of the frontend, like screen reader support and internationalization labels Jan 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accessibility Accessibility improvements of the frontend, like screen reader support and internationalization frontend Concerning the HTML editor
Projects
None yet
Development

No branches or pull requests

1 participant