Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support UIA ::SetFocus and basic property change notifications #11674

Merged
merged 13 commits into from
Jun 7, 2023

Conversation

FalseLobster
Copy link
Contributor

@FalseLobster FalseLobster commented May 30, 2023

Description

Improves the navigation and focus management experience for screen readers and other assistive technologies (ATs) and creates a pattern for property invalidation for UIA-related RN props.

Type of Change

  • Bug fix (non-breaking change which fixes an issue)
  • New feature (non-breaking change which adds functionality)

Why

Improve the ability of ATs to navigate RNW Fabric and manage focus.

What

  • Added support for IRawElementProviderFragment::SetFocus
  • Added support for IRawElementProviderAdviseEvents
  • Added support for RN's accesibilityState.disabled
  • Added UIA property changes and events for focusable, HasKeyboardFocus, and enabled
  • Fixed a playground crash when ATs try to get the RootProvider before the HwndHost is initialized
  • Added a few helper classes and methods to hopefully make code easier to understand and reuse

Testing

I performed some testing using Narrator, NVDA, and Accessibility Insights to confirm that AT navigation is improved and property and event changes happen as expected.

Microsoft Reviewers: Open in CodeFlow

@FalseLobster FalseLobster added the Area: Fabric Support Facebook Fabric label May 30, 2023
@FalseLobster FalseLobster requested a review from a team as a code owner May 30, 2023 22:40
Comment on lines 7 to 8
#define IN
#define OUT
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this for? SAFEARRAY?
Most of the code is using std::vector, so where's the SAFEARRAY dependency being forced?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yea, a lot of the UIA APIs use SAFEARRAY for certain parameters.

#pragma warning(disable : 4229)
#define IN
#define OUT
#include <atlsafe.h>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do not use ATLSafe.h

Copy link
Member

@asklar asklar May 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this PR adds SAFEARRAY support to WIL which we should be using instead:
microsoft/wil#145

could you perhaps work with @chrisglein 's team to see that PR through and use it here?

Copy link
Member

@asklar asklar May 31, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+@carlos-zamora who filed the original wil issue. Can you help? :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice, I wasn't aware of the WIL API. @asklar I've been pulled off this work for now. Would you be OK with me just using the SAFEARRAY directly and opening an issue to move this to the WIL stuff?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@jonthysell we can interface manually (which I think is what @FalseLobster is proposing) but it's going to be a lot more code than necessary and hard to maintain/reason over. WIL is header only and designed to simplify calling win32 apis so your code becomes a lot more concise. Ideally we can move to that sooner than later.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@FalseLobster I'm okay with taking things as you've written them and opening an issue to replace ATL with WIL and assign it to me. There's probably other places the (Fabric) codebase could benefit from access to WIL helpers to deal with all this old Win32 stuff.

@asklar Yes, WIL is header only, but it's also "yet another re-wrapping of standard C++" to add to this project. Like do we really need every implementation of smart pointers that have ever been made? We have stl, boost, Mso, cppwinrt, now wil.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the alternative being used here is to use ATL. We should not be using ATL.
WIL provides convenient wrappers for this specific use case. If STL, Boost, MSO or CppWinRT had RAII wrappers for these types, great. I don't believe they do though.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@asklar , @jonthysell , could we avoid dependency on the new library?
SAFEARRAY are not so difficult to use directly.
I would expect that C++/WinRT has all the required wrappers for them.

C++/WinRT is also header-only library, and we know the overall impact on the developer experience.
I would stay away from the header-only libraries since they put a huge pressure on the compiler: such headers must be often compiled again and again for each .cpp that slows down compilation no matter how powerful your computer is.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

C++/WinRT is for working with WinRT, but these are not WinRT apis, so C++/WinRT doesn't help in this case. Feel free to use SAFEARRAY directly.

Copy link
Member

@asklar asklar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please do not use ATLSafe.h
Please also remove it from here:

@microsoft-github-policy-service microsoft-github-policy-service bot added the Needs: Author Feedback The issue/PR needs activity from its author (label drives bot activity) label May 31, 2023
return rootProvider->WasPropertyAdvised(propId);
}

void UpdateUiaProperty(winrt::IInspectable provider, PROPERTYID propId, bool oldValue, bool newValue) noexcept {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Was this the function you were talking about maybe needing to be templatized?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea, I was considering templatizing this but I was fumbling a little with the SFINAE, and maybe we can just add it later when there are more overloads needed. Having two functions isn't too bad for now.

@microsoft-github-policy-service microsoft-github-policy-service bot removed the Needs: Author Feedback The issue/PR needs activity from its author (label drives bot activity) label Jun 1, 2023
#pragma warning(disable : 4229)
#define IN
#define OUT
#include <atlsafe.h>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@FalseLobster I'm okay with taking things as you've written them and opening an issue to replace ATL with WIL and assign it to me. There's probably other places the (Fabric) codebase could benefit from access to WIL helpers to deal with all this old Win32 stuff.

@asklar Yes, WIL is header only, but it's also "yet another re-wrapping of standard C++" to add to this project. Like do we really need every implementation of smart pointers that have ever been made? We have stl, boost, Mso, cppwinrt, now wil.

@FalseLobster
Copy link
Contributor Author

@asklar can I get a sign off?

@FalseLobster FalseLobster merged commit 72e480d into microsoft:main Jun 7, 2023
44 checks passed
This was referenced Apr 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Area: Fabric Support Facebook Fabric New Architecture Broad category for issues that apply to the RN "new" architecture of Turbo Modules + Fabric
Projects
Status: Done
Development

Successfully merging this pull request may close these issues.

None yet

7 participants