Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Basic implementation of UIA navigation functions #11412

Merged
merged 17 commits into from
Mar 30, 2023

Conversation

FalseLobster
Copy link
Contributor

@FalseLobster FalseLobster commented Mar 22, 2023

Description

Adds a AutomationProvider for non-Root elements and enables the UIA client to navigate among our Fabric-based Component's UIA providers. This adds a very basic implementation for a few RN a11y props (accessibilityRole, name, and testId) to make the tree a little more interesting and unblock basic automated-test scenarios; however, since that was not the focus of this PR, work remains to be done here (e.g. UIA property change notifications, separating Control/Content/Raw trees). PR sent out now to keep the change small since there's a lot of work to be done, we'll keep iterating :)

Type of Change

  • New feature (non-breaking change which adds functionality)
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
    • A few of the accessibilityRole mappings do not match the Paper implementation (radiogroup, alert), preferring instead the mappings specified by the ARIA Core AAM

Why

Right now, the only accessible element in the UIA tree is the RootView. This makes the rest of the tree accessible for AT users (albeit with very limited functionality).

What

  • Added new CompositionDynamicAutomationProvider
    • Currently assumes Components derive from View props so that all our ComponentViews can be represented in the tree, but this will need to be corrected for Text/Image types
  • Implemented Navigate on the Dynamic and Root provider
  • Root provider now no longer expects HWND as an argument, it is optionally injected by the HWNDHost
  • Since AutomationProviders will likely always require a back pointer to the ComponentView, I added enable_shared_from_this to the CompositionBaseComponentView, moved ctors to private, and added nodiscard factory functions
    • I'm using ReactTaggedView for the back pointers, and I updated them to use weak pointers instead
  • Added support for GetRuntimeId to distinguish fragments, using react tag to future proof against recycled views

Screenshots

a2560345-53b2-4aec-893a-2a86e717e517

Testing

Manually verified a few of the JS samples have reasonable a11y trees in Accessibility Insights

Microsoft Reviewers: Open in CodeFlow

@FalseLobster FalseLobster requested review from a team as code owners March 22, 2023 22:55
long GetControlType(const std::string &role) noexcept {
if (role == "adjustable") {
return UIA_SliderControlTypeId;
} else if (role == "alert" || role == "group" || role == "search" || role == "timer" || role.empty()) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"alert" returned Text before

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I mentioned in the description that some of the ways Paper is mapping roles to UIA control types doesn't agree with the ARIA Core AAM. Given Meta seems to be moving more toward ARIA standards, I thought I'd fix this now, but I suppose we can discuss this more and I'll leave the original mapping in place for now.

@FalseLobster FalseLobster added the AutoMerge Causes a PR to be automatically merged once all requirements are passed (label drives bot activity) label Mar 28, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
AutoMerge Causes a PR to be automatically merged once all requirements are passed (label drives bot activity)
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

3 participants