Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

accessibility dictation: first round #9

Merged
merged 20 commits into from Mar 22, 2022
Merged

accessibility dictation: first round #9

merged 20 commits into from Mar 22, 2022

Conversation

phillco
Copy link
Owner

@phillco phillco commented Mar 20, 2022

No rush on reviewing this! I figured it would be good to get your thoughts on this first batch of changes, before I go on to push more stuff. I'm probably going to focus on cursorless-on-JetBrains for the rest of the weekend so I'm not blocked on this review.

In this change:

  • using AX to acquire context for dictation formatters
  • APIs for overriding in specific apps, a sample override for Messages.app
  • the ability to automatically enable AX for Electron
  • basic debugging of accessibility using hissing

My thought is the experimental branch can be a sort of middle ground between main and our feature branches. Ambitious beta testers could use it and it also gives us a place to merge our development changes together and test them together. We can promote it to main afterwards.

@phillco phillco requested a review from nriley as a code owner March 20, 2022 16:07
nriley added a commit to nriley/talon_community that referenced this pull request Mar 20, 2022
nriley added a commit to nriley/talon_community that referenced this pull request Mar 20, 2022
dictation/debugging.py Outdated Show resolved Hide resolved
dictation/debugging.py Outdated Show resolved Hide resolved
dictation/dictation_context.py Outdated Show resolved Hide resolved
dictation/dictation_context.py Outdated Show resolved Hide resolved
dictation/dictation_context.py Outdated Show resolved Hide resolved
dictation/dictation_context.py Outdated Show resolved Hide resolved
dictation/dictation_context.py Outdated Show resolved Hide resolved
dictation/dictation_context.py Outdated Show resolved Hide resolved
dictation/electron.py Outdated Show resolved Hide resolved
@phillco phillco requested a review from nriley March 22, 2022 02:07
@phillco
Copy link
Owner Author

phillco commented Mar 22, 2022

Thanks, made revisions

@phillco
Copy link
Owner Author

phillco commented Mar 22, 2022

Done

Copy link
Collaborator

@nriley nriley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM aside from a few tiny nits.


Sometimes the accessibility context reported by the application is wrong, but fixable in predictable ways (this is most common in Electron apps). This method can be overwritten in those applications to do so.

TODO(pcohen): it's a it strange to have both this and dictation_current_element;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Anything after a ":" gets stripped in actions.list() output (empirically) so consider moving into a comment.


try:
app.element.AXManualAccessibility = True
except UIErr as e:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can just use except UIErr: here.

@phillco phillco requested a review from nriley March 22, 2022 03:24
@nriley nriley merged commit 5644cba into experimental Mar 22, 2022
@nriley nriley deleted the dictation branch March 22, 2022 04:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants