Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🗣️ Screen reader support for reactive output changes #2757

Merged
merged 4 commits into from
Jan 3, 2024

Conversation

fonsp
Copy link
Owner

@fonsp fonsp commented Dec 18, 2023

This PR makes a step to get Pluto closer to being usable keyboard-only with a screen reader.

When you edit and run code, it will start running, and when finished, the result is returned by updating the cell's output. Because this is asynchronous, this information is currently not surfaced with a screen reader. The solution is to use an "ARIA live region": you can declare a DOM element to be "live", meaning that any update that happens to it should be announced with speech.

This PR makes the <pluto-output> a live area.

  • The "polite" level is used: changes only get announced when the user is idle, not directly interrupting other things.
  • A label is added. If a cell has an assignee xyz = , it reads "Result of variable xyz:", otherwise "Result of unlabeled cell:".
  • The <pluto-output> is marked as "atomic": screen readers should always announce its full contents when anything changes. E.g. if the output changes from [1,2] to [1,2,3], don't just announce 3. It also means that the label will be read on any change.
  • The aria-relevant setting is all: additions, deletions, modifications.

Here is a screen recording (no audio but subtitles). Notice that the third cell runs reactively, and the new result gets announced.

Schermopname.2023-12-18.om.15.06.49.mov

Recorded on Chrome with MacOS voice-over enabled (default settings, dutch).

TODO:

  • What is the best way to label cells in the announced text? Because cells run reactively, in "random" order, it's important that users can keep a mental model of what is happening where. Maybe we generate short memorable labels? Maybe screen reader users will be encouraged to label every cell themselves? (But then you would have a hard time editing notebooks written by people who don't use a screen reader.) Maybe we can use GPT to generate labels?
  • I noticed that even with the aria-live="polite" setting, some announcements still interrupt others. E.g. in the recording above, I sometimes saw that the tiny delay between the two cells caused the first announcement to be interrupted. We could use the aria-busy setting to deliver the multiple updates in a batch (I think), but we also don't want to hold all updates until all cells completed running.
  • What if a cell is running for a long time? It's currently not possible to tell if you are waiting for running cells or not.

@fonsp fonsp added frontend Concerning the HTML editor accessibility Accessibility improvements of the frontend, like screen reader support and internationalization labels Dec 18, 2023
Copy link
Contributor

Try this Pull Request!

Open Julia and type:

julia> import Pkg
julia> Pkg.activate(temp=true)
julia> Pkg.add(url="https://github.com/fonsp/Pluto.jl", rev="screen-reader-reactive-outputs")
julia> using Pluto

@fonsp fonsp marked this pull request as ready for review January 3, 2024 23:09
@fonsp fonsp merged commit be51e4b into main Jan 3, 2024
3 checks passed
@fonsp fonsp deleted the screen-reader-reactive-outputs branch January 3, 2024 23:10
@fonsp
Copy link
Owner Author

fonsp commented Jan 3, 2024

Let's work on these TODOs in the future :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
accessibility Accessibility improvements of the frontend, like screen reader support and internationalization frontend Concerning the HTML editor
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

1 participant