Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UIA objects: recognize more UIA dialogs via more class names and IsDialog property in Windows 10 Redstone 5 #8473

Merged
merged 10 commits into from
Jul 19, 2018

Conversation

josephsl
Copy link
Collaborator

@josephsl josephsl commented Jul 1, 2018

Link to issue number:

Resolves #8405

Summary of the issue:

NVDA does not recognize dialogs in various Windows 10 functions and apps.

Description of how this pull request fixes the issue:

Added two checks:

  1. Added more dialog class names, and moved existing class names to a new list in UIA handler along with new ones.
  2. In Windows 10 RS5, tried using IsDialog property (30174).

The new routine will consult IsDialog property first, and if that doesn't work, find out if the class name is one of the predefined dialog class names. one exception to this is Popup class, which isn't a window element yet it is a dialog (recognized as such in Redstone 5).

Testing performed:

Tested via Windows 10 App Essentials add-on, as well as on Windows 10 Version 1803, 1709, and Insider Preview builds.

Known issues with pull request:

None

Change log entry:

Bug fixes (possibly): NVDA will recognize more dialogs in Windows 10 and other apps.

A note about pull request content

Because IsDialog property is included in Redstone 5/IUIAutomation6 interface, the constant defined in UIA handler will be gone once IUIA6 support comes to NVDA later in 2018.

…ws 7, 8.x, 10, as well as apps. Re nvaccess#8405.

Windows 10 and various apps added more dilaog class names. Rather than using a static list as part of overlay class chooser in UIA object, this list, along iwth new ones, are now available in UIA handler.
Also, added constant for UIA_IsDialogPropertyID in UIA handler, to be removed once IUIAutomation6 interface support comes to NVDA, as this property ID will be included in Redstone 5.
…roperty in Windows 10 RS5. Re nvaccess#8405.

Instead of looking up only three class names, look up more via:
* Additional class names found in Windows 10 and various apps.
* In RS5, IsDialog property.
This allows dialogs such as app uninstlal dialog and many others to be recognized and their contents read by NVDA.
@@ -789,7 +789,15 @@ def findOverlayClasses(self,clsList):
elif UIAControlType==UIAHandler.UIA_ListItemControlTypeId:
clsList.append(ListItem)
# #5942: In Windows 10 build 14332 and later, Microsoft rewrote various dialog code including that of User Account Control.
if self.UIAIsWindowElement and UIAClassName in ("#32770","NUIDialog", "Credential Dialog Xaml Host"):
# #8405: there are more dialogs scattered throughout Windows 10 and various apps.
# Apart from "popup" dialog seen when uninstaling apps and such, they are window elements (popup dialog isn't).
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instaling > installing

try:
isDialog = self._getUIACacheablePropertyValue(UIAHandler.UIA_IsDialogPropertyId)
except COMError:
isDialog = False
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Instead of setting this to False, you could consider setting isDialog to the expression in the if clause. like:

isDialog = (self.UIAIsWindowElement and UIAClassName in UIAHandler.UIADialogClassNames) or (not self.UIAIsWindowElement and UIAClassName == "Popup"):

To me, this is a bit more readable. Or are there cases where isDialog returns False, and yet we should treat something as a dialog? In that case, the "not isdialog" check still seems redundant. I think we should always respect isDialog if possible.

@josephsl
Copy link
Collaborator Author

josephsl commented Jul 3, 2018

Hi,

I think you're right. Note that there might be something that IsDialog UIA property says yes but it should not be treated as a dialog, in which case we can consult another list for this.

Thanks.

Reviewed by @LeonarddeR (Babbage): simplify dialog flag via assigning this flag from exception hyandler segment rahter than setting it to False when COM error is thrown. This then allows IsDialog flag to be set whether or not exceptoin has occured.
@josephsl
Copy link
Collaborator Author

Hi,

Tagging @michaelDCurran: I have combined IsDialog with IUIAutomation6 (build 17692). I hope I addressed changes requested by @LeonarddeR, and feel free to use things fomr that new interface that could benefit NVDA.

Thanks.

@michaelDCurran
Copy link
Member

I have removed Popup as a fallback dialog class as that name is too generic. At least all the other ones are clearly dialogs from the name and all depend on being a window element. As this code will only be used for pre rs5 I don't want it to be over complicated.

@josephsl
Copy link
Collaborator Author

josephsl commented Jul 19, 2018 via email

@michaelDCurran
Copy link
Member

@LeonarddeR: you have a pending review before I can merge this.

Copy link
Collaborator

@LeonarddeR LeonarddeR left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we make sure that the UIA interfaces we will use with this pr are stable enough (i.e. is it possible that Microsoft will change them before RS5 final is there)?

@josephsl
Copy link
Collaborator Author

josephsl commented Jul 19, 2018 via email

@michaelDCurran
Copy link
Member

Based on internal discussions with Microsoft, I agree with Joseph.

@michaelDCurran michaelDCurran merged commit bff9b40 into nvaccess:master Jul 19, 2018
@nvaccessAuto nvaccessAuto added this to the 2018.3 milestone Jul 19, 2018
@josephsl josephsl deleted the i8405uiaDialogs branch July 30, 2018 20:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Windows 10: recognize several common dialog classes from XAML and other controls
4 participants