-
-
Notifications
You must be signed in to change notification settings - Fork 627
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
MS Word with UIA: use custom extensions to support line number, layout column number and section number #13283
Conversation
…here is not text column count. Such as for Microsoft Word with UIA.
…ept docElement as an argument, rather than fetching the enclosing element of the textRange. Saves a call and also reflects Micrsoft's own test code more closely.
See test results for failed build of commit 61e47fbed5 |
This comment has been minimized.
This comment has been minimized.
Co-authored-by: Łukasz Golonka <lukasz.golonka@mailbox.org>
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
…to its own Python function to allow unit tests to continue to pass. Unit tests do not require this library.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There are a few build warnings introduced, these may be outside your control, but it would be good if they were silenced:
"C:\work\repo\nvda\PR\include\microsoft-ui-uiautomation\src\uiAutomation\microsoft.ui.uiautomation\Microsoft.UI.UIAutomation.vcxproj" (Build target) (1) ->
(PrepareForBuild target) ->
C:\Program Files (x86)\Microsoft Visual Studio\2019\Community\MSBuild\Microsoft\VC\v160\Microsoft.CppBuild.targets(499,5): warning MSB8004: Output Directory does not end with a trailing slash. This build instance will add the slash as it is required to allow proper evaluation of the Output Directory. [C:\work\repo\nvda\PR\include\microsoft-ui-uiautomation\src\uiAutomation\microsoft.ui.uiautomation\Microsoft.UI.UIAutomation.vcxproj]
nvdaHelper/microsoft-ui-uiautomation/Microsoft.UI.UIAutomation.dll.manifest
Show resolved
Hide resolved
…VT_HEADER_DEPRECATION_WARNING` by converting the const char* from exception::what to wchar_t using multibyteToWideChar as recommended by the compiler warning, rather than using wstring_convert deprecated in c++17.
…ather than a template.
…g UiaArray to a local vector, and also ensure it is not copied.
…{outDir} passed to msbuild to suppress warnings amout the missing slash.
@feerrenrut I believe I have addressed all your points. |
…to its own utility function. also ensure that the buffer that receives the wide chars is initialized to 0s and is large enough to allow for a null-terminator.
…tly what is produced.
Fix caps and typos. Co-authored-by: Reef Turner <feerrenrut@users.noreply.github.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'll leave it up to you about this, I think the C++ standard library approach is less error prone and sets a good example.
… intermediate vector.
See test results for failed build of commit bf6ff0d7ba |
The test failure looks unrelated. I have kicked off the build again. |
…t column number and section number (#13283) Supersedes pr #13149 Summary of the issue: The following error is in the log when viewing annotations in the NVDA elements list: ``` ERROR - queueHandler.flushQueue (19:47:20.811) - MainThread (9476): Error in func ElementsListDialog.initElementType Traceback (most recent call last): File "queueHandler.pyc", line 55, in flushQueue File "browseMode.pyc", line 1054, in initElementType File "browseMode.pyc", line 1074, in filter File "NVDAObjects\UIA\wordDocument.pyc", line 139, in label File "NVDAObjects\UIA\wordDocument.pyc", line 95, in getCommentInfoFromPosition AttributeError: 'WordBrowseModeDocument' object has no attribute '_UIACustomAnnotationTypes' ``` This is because getCommentInfoFromPosition tries to fetch custom annotation types off its obj property, which it incorrectly expects to be a UIA NVDAObject. But in this case it is a TreeInterceptor. PR #13149 was opened to address this by specifically creating a UIA NVDAObject rather than getting it from the position TextInfo. However, Further investigation has found that it is actually not necessary to even support custom annotation types (draft comment, resolved comment) in getCommentInfoFromPosition as: 1. Quicknav / Elements List iteration was never updated to specifically jump to / list those custom comment types. See CommentUIATextInfoQuickNavItem.wantedAttribValues 2. Although the IUIAutomationTextRange implementation in MS Word does return these custom comment types in GetAttributeValue when given UIA_AnnotationTypesAttributeId, the elements returned with UIA_AnnotationObjectsAttributeId only ever return the standard Comment annotation type via their UIA_AnnotationAnnotationTypeId property. In fact for draft comments, no annotation objects are returned at all. In short, supporting custom comment annotation types in getCommentInfoFromPosition was not necessary in the first place. Description of how this pull request fixes the issue: Remove support for custom comment annotation types from getCommentInfoFromPosition. It again only supports the standard comment annotation type.
…Word via UIA, using UIA remote ops (#13387) This reintroduces the commits in pr #13283 which had been reverted in pr #13350. However it also addresses build issues. Summary of the issue: PR #13283 introduced support for reporting line/column/section numbers in Microsoft word via UI Automation, by using the UI Automation Remote Operations Library. However, this PR had to be reverted in #13350 as 1. UIA would not initialize properly in binary builds due to a missing manifest file. (Improper initialization of UIA during NVDA start #13347) 2. NVDA failed to build on Visual Studio 2022. Specifically when building the Remote Ops library with msbuild, midl would fail with an error about a system environment variable being missing. 3. NVDA could no longer be built on Windows 7. (Impossible to build NVDA after introduction of UIA remote library #13346) Description of how this pull request fixes the issue: This PR reintroduces all of the original changes, but: • setup.py now includes *.manifest files in the lib directory. • NVDA again now requires Visual Studio 2019. However, builds will not fail if a newer version of Visual Studio (E.g. 2022) is installed along side 2019. this is managed by setting MSVC_VERSION in scons before it looks for Visual Studio, so that it specifically selects VS 2019 (VC 14.2). Although building on Windows 7 could not be fixed, the readme now notes that only building on Windows 10 and higher is supported.
Link to issue number:
None.
Summary of the issue:
When supporting Microsoft word via UI automation, up to now there was no way to report line numbers, layout column numbers or section numbers. This was because these concepts do not exist in the UI Automation standard.
However, Microsoft recently added custom patterns to UI automation, as a part of its Custom Extensions work, which included custom properties and custom annotations.
Microsoft word has now added its own custom pattern which allows fetching information from a text range, such as line number, layout column number and section number.
Description of how this pull request fixes the issue:
Using the microsoft/microsoft-ui-uiautomation Remote Operations library, fetch line number, layout column number and section number in Microsoft word.
Note that the Remote Operations library is only supported in Windows 11 and above, and the custom pattern in Microsoft Word is only available as of build 16.0.14725.
A part from adding support for these specific attributes, this PR also acts as a first commit to add integrations with the UI Automation Remote Operations library. Now that we are building and including this library, there is much more code that can eventually be remoted for a massive speedup.
Testing strategy:
Known issues with pull request:
None known.
Change log entries:
New features
Changes
Bug fixes
For Developers
Code Review Checklist: