Allow Mac accessibility for apps other than VoiceOver

[ivmirx]

All native Mac app UIs are accessible by default, including web pages in Safari.

When it comes to Electron apps, the accessibility is disabled and it's not possible to turn it on without:

• turning on undocumented and buggy parameter AXEnhancedUserInterface (it breaks window positioning and animations)
• turning on VoiceOver itself

I suggest to remove [self voiceOverEnabled] check and add an alternative attribute for enabling accessibility or another way to set it on.

[kevinsawicki]

Thanks for reporting this, what would be the best way to reproduce this issue? Is there an accessibility tool other than VoiceOver that you are using that isn't working well?

[ivmirx]

@kevinsawicki I'm currently developing a time tracker that detects how much time users spend in apps, including different Slack teams, so I hit this limitation when Slack moved from accessible Safari-based wrapper to Electron. I found a workaround (by using window names) but there are other accessibility-heavy tools like Hammerspoon and different macOS window managers.

In case of Chromium-based browsers users can always enable accessibility at chrome://accessibility/ or other apps can force AXEnhancedUserInterface parameter (though it's a problematic approach). So it would be great to have some switch for Electron too.

Thank you!

[kevinsawicki]

@ivmirx so if we exposed an API to allow apps to turn accessibility mode on/off explicitly, would this fix your time tracking app use case for Slack? How do you envision apps would use it directly? Thanks.

[ivmirx]

@kevinsawicki yes, thank you! I use workaround already but it will be a great improvement to match native apps.

Any app with accessibility permission can use extern AXError AXUIElementSetAttributeValue (AXUIElementRef element, CFStringRef attribute, CFTypeRef value); and set attribute on any AXUIElementRef of another app, including the app itself. As far as I understand, you can declare a custom attribute but I've never tried it.

Allowing Apple's AXEnhancedUserInterface without the check for VoiceOver would be great too.

[ivmirx]

@kevinsawicki I recently had a thought about the possible issue with Electron's approach but I'm not sure if it's correct because I've never worked on actual tools for users with disabilities.

Electron apps enable accessibility when they detect VoiceOver's attribute but I think there is a group of users with motor disabilities who operate their Macs through accessibility API but don't use VoiceOver because they have good vision.

With the raise of popularity of Electron they will be cut off from many apps made with it. Probably something like a default menu bar option can solve this.

Again, it's a guess, it would be great to reach somebody who's an expert on Mac accessibility tools.

[anaisbetts]

I'm actually super not into other apps being able to reach into Slack and nerf its performance, because people won't make the connection that "I enabled this app => Slack is slower", they'll just think Slack is slow and blame us.

Instead of abusing accessibility APIs for things that aren't accessibility, we should make an API to actually get the information directly without sending thousands of thousands of accessibility events, 99.9% of which are ignored

[ivmirx]

@paulcbetts it's not only about Slack but Electron apps on macOS in general. The platform standard is being accessible by default and other cross-platform frameworks like Qt often disregard it.

I doubt that any popular app that uses accessibility sends "thousands of thousands" events. Tools for users with disabilities navigate the accessibility hierarchy following their commands, other tools like Spectacle usually get or set window frame. Also pre-Electron Slack was fully accessible (used Safari-based web view probably?).

[anaisbetts]

I doubt that any popular app that uses accessibility sends "thousands of thousands" events

afaik any time the DOM changes and the contents are on-screen you'll have to send an a11y event to notify the reader (even if this doesn't result in text being read aloud). There's a reason that Chrome doesn't leave this running all the time

[stuartbrussell-intuit]

We use a WebDriver tool for automation scripts that uses accessibility on macOS. It's called AppiumForMac. It works for all elements of our native apps, and the menu bar for our Electron app. It would be great to have a non-VoiceOver enabler for Electron window contents.

[programmerq]

I am writing this comment using the Mac OS dictation feature.

When using Mac native apps such as safari and text edit, I can do things like say "select previous word" or "replace with ".

If I am using an electron application, I cannot have this full control. This makes it much much more difficult interact with the computer. I use many electron based applications. I would be very happy if there is a way as an end-user to be able to enable any extra accessibility features that would allow this extra functionality.

Another significant feature that I use with dictation is the "show numbers" feature. When using an electron app, the only elements that have numbers assigned are the maximize, minimize, Close buttons. For example, if I open slack in Safari, and then give the show numbers command, almost every clickable element has a number bubble that appears on the screen, and I can speak a number to select it. When I open slack in electron, none of these elements have a number of bubble. Opening links is all but impossible.

Slack is probably the electron application I use the most, but the same problem extends to other electron based applications.

[jkleinsc]

This issue was resolved via #10305.