New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow Mac accessibility for apps other than VoiceOver #7206

Closed
ivmirx opened this Issue Sep 14, 2016 · 11 comments

Comments

Projects
None yet
6 participants
@ivmirx
Contributor

ivmirx commented Sep 14, 2016

All native Mac app UIs are accessible by default, including web pages in Safari.

When it comes to Electron apps, the accessibility is disabled and it's not possible to turn it on without:

  • turning on undocumented and buggy parameter AXEnhancedUserInterface (it breaks window positioning and animations)
  • turning on VoiceOver itself

I suggest to remove [self voiceOverEnabled] check and add an alternative attribute for enabling accessibility or another way to set it on.

@kevinsawicki

This comment has been minimized.

Contributor

kevinsawicki commented Sep 16, 2016

Thanks for reporting this, what would be the best way to reproduce this issue? Is there an accessibility tool other than VoiceOver that you are using that isn't working well?

@ivmirx

This comment has been minimized.

Contributor

ivmirx commented Sep 18, 2016

@kevinsawicki I'm currently developing a time tracker that detects how much time users spend in apps, including different Slack teams, so I hit this limitation when Slack moved from accessible Safari-based wrapper to Electron. I found a workaround (by using window names) but there are other accessibility-heavy tools like Hammerspoon and different macOS window managers.

In case of Chromium-based browsers users can always enable accessibility at chrome://accessibility/ or other apps can force AXEnhancedUserInterface parameter (though it's a problematic approach). So it would be great to have some switch for Electron too.

Thank you!

@kevinsawicki

This comment has been minimized.

Contributor

kevinsawicki commented Nov 1, 2016

@ivmirx so if we exposed an API to allow apps to turn accessibility mode on/off explicitly, would this fix your time tracking app use case for Slack? How do you envision apps would use it directly? Thanks.

@ivmirx

This comment has been minimized.

Contributor

ivmirx commented Nov 12, 2016

@kevinsawicki yes, thank you! I use workaround already but it will be a great improvement to match native apps.

Any app with accessibility permission can use extern AXError AXUIElementSetAttributeValue (AXUIElementRef element, CFStringRef attribute, CFTypeRef value); and set attribute on any AXUIElementRef of another app, including the app itself. As far as I understand, you can declare a custom attribute but I've never tried it.

Allowing Apple's AXEnhancedUserInterface without the check for VoiceOver would be great too.

@ivmirx

This comment has been minimized.

Contributor

ivmirx commented Dec 17, 2016

@kevinsawicki I recently had a thought about the possible issue with Electron's approach but I'm not sure if it's correct because I've never worked on actual tools for users with disabilities.

Electron apps enable accessibility when they detect VoiceOver's attribute but I think there is a group of users with motor disabilities who operate their Macs through accessibility API but don't use VoiceOver because they have good vision.

With the raise of popularity of Electron they will be cut off from many apps made with it. Probably something like a default menu bar option can solve this.

Again, it's a guess, it would be great to reach somebody who's an expert on Mac accessibility tools.

@paulcbetts

This comment has been minimized.

Contributor

paulcbetts commented Dec 18, 2016

I'm actually super not into other apps being able to reach into Slack and nerf its performance, because people won't make the connection that "I enabled this app => Slack is slower", they'll just think Slack is slow and blame us.

Instead of abusing accessibility APIs for things that aren't accessibility, we should make an API to actually get the information directly without sending thousands of thousands of accessibility events, 99.9% of which are ignored

@ivmirx

This comment has been minimized.

Contributor

ivmirx commented Dec 18, 2016

@paulcbetts it's not only about Slack but Electron apps on macOS in general. The platform standard is being accessible by default and other cross-platform frameworks like Qt often disregard it.

I doubt that any popular app that uses accessibility sends "thousands of thousands" events. Tools for users with disabilities navigate the accessibility hierarchy following their commands, other tools like Spectacle usually get or set window frame. Also pre-Electron Slack was fully accessible (used Safari-based web view probably?).

@paulcbetts

This comment has been minimized.

Contributor

paulcbetts commented Dec 19, 2016

I doubt that any popular app that uses accessibility sends "thousands of thousands" events

afaik any time the DOM changes and the contents are on-screen you'll have to send an a11y event to notify the reader (even if this doesn't result in text being read aloud). There's a reason that Chrome doesn't leave this running all the time

@stuartbrussell-intuit

This comment has been minimized.

stuartbrussell-intuit commented Apr 6, 2017

We use a WebDriver tool for automation scripts that uses accessibility on macOS. It's called AppiumForMac. It works for all elements of our native apps, and the menu bar for our Electron app. It would be great to have a non-VoiceOver enabler for Electron window contents.

@programmerq

This comment has been minimized.

programmerq commented May 5, 2017

I am writing this comment using the Mac OS dictation feature.

When using Mac native apps such as safari and text edit, I can do things like say "select previous word" or "replace with ".

If I am using an electron application, I cannot have this full control. This makes it much much more difficult interact with the computer. I use many electron based applications. I would be very happy if there is a way as an end-user to be able to enable any extra accessibility features that would allow this extra functionality.

Another significant feature that I use with dictation is the "show numbers" feature. When using an electron app, the only elements that have numbers assigned are the maximize, minimize, Close buttons. For example, if I open slack in Safari, and then give the show numbers command, almost every clickable element has a number bubble that appears on the screen, and I can speak a number to select it. When I open slack in electron, none of these elements have a number of bubble. Opening links is all but impossible.

Slack is probably the electron application I use the most, but the same problem extends to other electron based applications.

ivmirx added a commit to ivmirx/electron that referenced this issue Aug 19, 2017

Special attribute for macOS accessibility electron#7206
In the linked issue we were discussing that Electron apps are inaccessible unless VoiceOver is enabled. While it's a working solution for users with vision impairment, all other users and apps that require accessibility can't interact with Electron-based software because they don't keep VoiceOver running.

I suggest adding `AXManualAccessibility` for programmatically enabling it in Electron apps. The reason for a new attribute is that `AXEnhancedUserInterface` is already reserved by VoiceOver.

Adding this attribute will allow both Electron developers and 3rd party developers to enable and disable accessibility from their code by calling `accessibilitySetValue:forAttribute:` on the application.

It will be also possible to create a small utility app to switch accessibility in Electron-based apps until there's a native UI solution (like the accessibility settings page in Chrome).
@jkleinsc

This comment has been minimized.

Contributor

jkleinsc commented Sep 23, 2017

This issue was resolved via #10305.

@jkleinsc jkleinsc closed this Sep 23, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment