Use UIA in MS Word by default on MS Word version 16.0.15000 or higher #13437

michaelDCurran · 2022-03-06T23:58:23Z

Link to issue number:

Summary of the issue:

Microsoft Word 2016 exposes a rich UI Automation implementation. For some time now, users have been able to optionally turn this on with an advanced setting. NVDA's support for MS Word via UIA has major performance advantages over the older object model support, so NVDA should use the UIA support by default where available. However, as the UIA implementation improved throughout Office 2016's lifetime, we should only enable our support for recent builds of Office 2016, specifically for build 15000 and higher, and when only on Windows 11.
This was previously tried in pr #12770 but reverted in pr #12989
The main argument for reverting was the lack of math support, and ability to report line column and section numbers.
These have all been since addressed.

Description of how this pull request fixes the issue:

NVDA now uses UI Automation to access Microsoft Word document controls by default when on Windows 11, for Microsoft Word version 16.0.15000 and higher.
The Use UI Automation to access Microsoft Word document controls when available checkbox has been replaced with a combo box with the following values:

Default (where suitable)
Only where necessary: where the Microsoft Word object model is not available at all
Where suitable: Windows 11 / Microsoft Word version 16.0.15000 or higher, or where the Microsoft Word object model is unavailable
Always: where ever UI automation is available in Microsoft word (no matter how complete).
If the older checkbox was previously checked, the setting will be set to Always.

Testing strategy:

With MS Word 16.0.15000 on Windows 11:

With the setting set to Only where necessary: confirm that NvDA is not using UIA to access Microsoft Word by focusing on a Word document and viewing NVDA dev info in the log.
With the setting set to Where Suitable, ensure that NVDA is using UIA with Microsoft Word by focusing a Microsoft Word document and viewing the dev info in the NVDA log.
With the setting set to Always, ensure that NVDA is using UIA with Microsoft Word by focusing a Microsoft Word document and viewing the dev info in the NVDA log.
Make sure that with the old checkbox checked, and then running the new NvDA, that the profile upgrade steps run and that the new setting is set to Always.

Known issues with pull request:

None known.

Change log entries:

New features

For Microsoft Word 16.0.15000 and higher on Windows 11, NVDA will use UI Automation to access the document by default, providing a significant performance improvement over the old Object model access.
Changes
Bug fixes
For Developers

Code Review Checklist:

Pull Request description:
- description is up to date
- change log entries
Testing:
- Unit tests
- System (end to end) tests
- Manual testing
API is compatible with existing add-ons.
Documentation:
- User Documentation
- Developer / Technical Documentation
- Context sensitive help for GUI changes
UX of all users considered:
- Speech
- Braille
- Low Vision
- Different web browsers
- Localization in other languages / culture than English

Replace the `Use UI Automation to access Microsoft Word document controls when available` checkbox with a combo box allowing to change from Default (where suitable), Only where necessary, Where suitable, and Always. If the older checbox was already checked, the setting becomes `Always`.

CyrilleB79 · 2022-03-07T00:07:40Z

Typo in change log (in case you copy/paste):
Mcirosoft -> Microsoft

seanbudd

General approach looks good to me. I'm awaiting confirmation that #13006 does not block this PR, as it is not reproducible in Word 16.0.15000 or higher.

source/gui/settingsDialogs.py

source/UIAHandler/__init__.py

seanbudd · 2022-03-07T08:04:40Z

source/config/profileUpgradeSteps.py

+	except KeyError:
+		useInMSWord = False
+	if useInMSWord:
+		profile['UIA']['allowInMSWord'] = 3  # ALWAYS


Could we use the enum here?

Suggested change

profile['UIA']['allowInMSWord'] = 3 # ALWAYS

profile['UIA']['allowInMSWord'] = AllowUiaInMSWord.ALWAYS

While I agree using an enum here makes this much more readable the suggestion cannot be applied as is:

We have to use .value on an enum member when adding its value to config.

More importantly importing UIAHandler in config (that is much earier in NVDA's startup process than it is done now) would almost certainly prevent NVDA from starting. This suggest to me that the enum in question should be placed in config rather than in UIAHandler and UIAHandler can just reuse it from there.

I'll move both AllowInMSWord and AllowInChromium enums to configspec.py and rename them to reflect they are for UIA.

source/config/profileUpgradeSteps.py

codeofdusk · 2022-03-07T08:41:50Z

I'm awaiting confirmation that #13006 does not block this PR, as it is not reproducible in Word 16.0.15000 or higher.

I recommend not blocking for #13006 and deprecating setting columns/rows via bookmarks as UIA provides first column/row as headers automatically.

josephsl · 2022-03-07T09:05:33Z

Hi,

I think #13006 will not block this PR because:

Testing it with Microsoft 365 beta 16.0.15018 shows that it can still be reproduced.
More importantly, NVDA+Shift+C/R isn't even a part of UIA Word document scripts list as evidenced by input help being silent about these commands when performed from Word with UIA enabled.

I think Bill raises an important point: if we are going to use UIA from Microsoft 365 version 2203 or later, then we might as well take advantage of what UIA says provided that we can communicate issues with Microsoft Word team. This obviously requires comparing with Narrator and other AT's which can provide another data point to consider when refining NVDA's support for Office via UIA.

Thanks.

cary-rowen · 2022-03-07T09:58:22Z

Hi,
Yes, I agree with Joseph's point, another reason is that if the row and column headers are set in non-uia mode, they are still valid in UIA mode.
Thanks

Neurrone · 2022-03-07T10:48:06Z

Would #12998 block this? Tables are painful to navigate with UIA enabled.

michaelDCurran · 2022-03-08T00:44:56Z

We should certainly try and address #12998, but I don't think it should block this. For the reason that #12998 does not cause information to be lost or greatly delayed. It is just that there is extra redundant information spoken at the end.

Co-authored-by: Sean Budd <sean@nvaccess.org>

Qchristensen

User guide looks good. Are there places where a Microsoft Word control is used outside these two examples? "This applies to documents in Microsoft word itself, plus messages in Microsoft Outlook" implies it is likely to be found ONLY in those two situations, though I'm sure there might be other places it is used. (Not blocking in any case, that's just a comment / question).

… config and make use of AllowUiaInMSWord in profileUpgradeSteps.

seanbudd

Thanks @michaelDCurran

AppVeyorBot · 2022-03-08T02:02:40Z

Unable to install NVDA prior to tests.
Build (for testing PR): https://ci.appveyor.com/api/buildjobs/2axjcy1l4848tp65/artifacts/output/nvda_snapshot_pr13437-24900,7dd32d1f.exe

See test results for failed build of commit 7dd32d1f79

LeonarddeR · 2022-03-08T06:44:22Z

May be change the changelog to reflect that the mail editor/reading in Outlook is also affected by this change?

LeonarddeR · 2022-03-08T07:08:34Z

    Actually, when testing this, I noticed that NVDA is very obnoxious in logging `MathPlayer 4 not available` from mathPres.ensureInit. Could we avoid calling `mathPres.ensureInit()` in WordDocumentTextInfo.getTextWithFields? I assume checking for mathPres.speechProvider or mathPres.brailleProvider should be enough as long as we ensure init of math presentation somewhere earlier, preferably when initialising NVDA as everything else.

michaelDCurran · 2022-03-08T07:28:35Z

@LeonarddeR If we only call ensureInit on NvDA startup, the user would have to restart NvDA if later installing mathPlayer.
I think we should deal with this separately. The issue is not new to this pr.

LeonarddeR · 2022-03-08T07:31:22Z

I'm fine with that, as long as it is addressed pre 2022.1 release.

michaelDCurran · 2022-03-08T07:34:01Z

Opening an issue with an example log would be a great help if you could please. As I think there needs to be some further thought / discussion of exactly how we can address it.

CyrilleB79 · 2022-03-10T13:20:05Z

Hello

FYI there is an other issue regarding Word with UIA reported on NVDA mailing list; it's here.
I have not tested myself, but @Qchristensen did.

Cyrille

Qchristensen · 2022-03-11T01:47:38Z

That issue has been reported as #13458 - but I don't think it is a blocker here. The majority of users likely don't have colour reporting enabled, and reading tables using the table navigation commands do work as expected.

Using UIA by default is also different to disabling NOT using UIA - For some time there have been some users for whom UIA provides a better experience in Word. Similarly, for some time yet, there will be some users for whom disabling UIA is the better experience. The key is to change over at a point where the UIA experience will be better for the majority of users. Those who need to, can then turn the feature off, just as currently, those who need it, can turn the feature on.

In any case, it is, of course, important to capture those issues like #13458, so they can be addressed.

LittleStar-VIP · 2022-03-11T13:39:38Z

Hello, based on my further testing, the issue is not limited to report of color #13459
But the UIA issue with reporting of color is really a trouble to me.
In my daoly work, sighted staffs require me to hightlight certain text in certain color, so I really need to enable UIA to edit a word file.
Hmm, disabling the UIA can fix the problem temporarily, but I am unable to check if which text is hightlighted.

…13437).

…13563) Fix-up of #13437. Summary of the issue: With #13437, the option "Use UI Automation to access Microsoft Word document controls" in Advanced settings panel has become a combo-box; it was a checkbox before. In the same time, the accelerator key (mapped to W) has been removed for this option: "&" removed from the label. When testing Word with and without UIA, it was handy to focus the Advanced panel, check the first checkbox and press alt+W to jump directly to this option. Now tabbing between the options is required. Description of how this pull request fixes the issue: Just add the "&" in the label to restore the accelerator key for this option.

…13437). (nvaccess#13563) Fix-up of nvaccess#13437. Summary of the issue: With nvaccess#13437, the option "Use UI Automation to access Microsoft Word document controls" in Advanced settings panel has become a combo-box; it was a checkbox before. In the same time, the accelerator key (mapped to W) has been removed for this option: "&" removed from the label. When testing Word with and without UIA, it was handy to focus the Advanced panel, check the first checkbox and press alt+W to jump directly to this option. Now tabbing between the options is required. Description of how this pull request fixes the issue: Just add the "&" in the label to restore the accelerator key for this option.

…13563) (#13590) PR #13563 restores an accelerator key that has been removed by #13437 during NVDA 2022.1 release cycle. Unfortunately, PR #13563 has been opened after translation freeze and thus could not integrate 2022.1. However in #13563, I have written: I would have wished to open this PR against beta to fix the issue immediately before the stable release. However, we are already in the translation freeze, thus this is not possible to change a translatable string, unfortunately. If a new translation freeze should occur, please reconsider merging this PR in beta. According to this thread, an extension translation freeze is in preparation with new translatable strings. Thus this allow to integrate this fix for 2022.1 considering that the accelerator key was removed during 2022.1 dev cycle. Description of how this pull request fixes the issue: Cherry-pick the commit corresponding to #13563, i.e. c446688.

LittleStar-VIP · 2022-05-18T11:23:42Z

Not sure if this temporary fix is possible.
nvda can offer an input gesture so that user can temporary switch the uia support for word or excel on or off

codeofdusk · 2022-05-18T20:26:21Z

@LittleStar-VIP This is possible by creating a config profile with UIA deactivated, then binding the profile's activation to an input gesture. Note that, as the active accessibility API for a window is chosen on focus, it is necessary to switch away from (Alt+Tab) then back to Word for the change to take effect.

LittleStar-VIP · 2022-05-19T00:52:14Z

@codeofdusk
Understood, thanks for your instructions.

michaelDCurran requested review from a team as code owners March 6, 2022 23:58

michaelDCurran requested review from Qchristensen and seanbudd March 6, 2022 23:58

michaelDCurran added this to the 2022.1 milestone Mar 6, 2022

Fix spelling mistake in user guide.

aa2d70f

michaelDCurran added 2 commits March 7, 2022 10:25

Linting

a7d459c

UIAHandler.shouldUseUIAInMSWord: fix indenting and clarify log message.

4a88be2

cary-rowen mentioned this pull request Mar 7, 2022

Cannot set column & row headers in Word using UIA #13006

Closed

seanbudd reviewed Mar 7, 2022

View reviewed changes

Apply suggestions from code review

8724680

Co-authored-by: Sean Budd <sean@nvaccess.org>

Qchristensen approved these changes Mar 8, 2022

View reviewed changes

Move AllowUiaInChromium and AllowUiaInMSWord enums from UIAHandler to…

3b51d1c

… config and make use of AllowUiaInMSWord in profileUpgradeSteps.

seanbudd approved these changes Mar 8, 2022

View reviewed changes

Fix typo

e556911

seanbudd assigned michaelDCurran Mar 8, 2022

michaelDCurran added 2 commits March 8, 2022 16:33

Merge branch 'master' into wordWithUIAByDefault

e955abc

Update what's new

6274221

Clarify What's new entry

3253080

LeonarddeR approved these changes Mar 8, 2022

View reviewed changes

michaelDCurran merged commit 0d9a280 into master Mar 8, 2022

michaelDCurran deleted the wordWithUIAByDefault branch March 8, 2022 07:29

Neurrone mentioned this pull request Mar 8, 2022

Tab Navigation in Tables Announces Selection Changes in Word When Using UIA #12998

Open

CyrilleB79 added a commit to CyrilleB79/nvda that referenced this pull request Mar 30, 2022

Advanced settings panel: restore accelerator key (fix-up of nvaccess#…

ac07396

…13437).

CyrilleB79 mentioned this pull request Mar 30, 2022

Advanced settings panel: restore accelerator key #13563

Merged

5 tasks

CyrilleB79 mentioned this pull request Apr 7, 2022

[Cherry-pick to beta] Advanced settings panel: restore accelerator key #13590

Merged

5 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use UIA in MS Word by default on MS Word version 16.0.15000 or higher #13437

Use UIA in MS Word by default on MS Word version 16.0.15000 or higher #13437

michaelDCurran commented Mar 6, 2022 •

edited

Loading

CyrilleB79 commented Mar 7, 2022

seanbudd left a comment

seanbudd Mar 7, 2022

lukaszgo1 Mar 7, 2022

michaelDCurran Mar 8, 2022

codeofdusk commented Mar 7, 2022

josephsl commented Mar 7, 2022 •

edited

Loading

cary-rowen commented Mar 7, 2022

Neurrone commented Mar 7, 2022

michaelDCurran commented Mar 8, 2022 via email

Qchristensen left a comment

seanbudd left a comment

AppVeyorBot commented Mar 8, 2022

LeonarddeR commented Mar 8, 2022

LeonarddeR commented Mar 8, 2022

michaelDCurran commented Mar 8, 2022

LeonarddeR commented Mar 8, 2022

michaelDCurran commented Mar 8, 2022 via email

CyrilleB79 commented Mar 10, 2022

Qchristensen commented Mar 11, 2022

LittleStar-VIP commented Mar 11, 2022

LittleStar-VIP commented May 18, 2022

codeofdusk commented May 18, 2022 •

edited

Loading

LittleStar-VIP commented May 19, 2022

	profile['UIA']['allowInMSWord'] = 3 # ALWAYS
	profile['UIA']['allowInMSWord'] = AllowUiaInMSWord.ALWAYS

Use UIA in MS Word by default on MS Word version 16.0.15000 or higher #13437

Use UIA in MS Word by default on MS Word version 16.0.15000 or higher #13437

Conversation

michaelDCurran commented Mar 6, 2022 • edited Loading

Link to issue number:

Summary of the issue:

Description of how this pull request fixes the issue:

Testing strategy:

Known issues with pull request:

Change log entries:

Code Review Checklist:

CyrilleB79 commented Mar 7, 2022

seanbudd left a comment

Choose a reason for hiding this comment

seanbudd Mar 7, 2022

Choose a reason for hiding this comment

lukaszgo1 Mar 7, 2022

Choose a reason for hiding this comment

michaelDCurran Mar 8, 2022

Choose a reason for hiding this comment

codeofdusk commented Mar 7, 2022

josephsl commented Mar 7, 2022 • edited Loading

cary-rowen commented Mar 7, 2022

Neurrone commented Mar 7, 2022

michaelDCurran commented Mar 8, 2022 via email

Qchristensen left a comment

Choose a reason for hiding this comment

seanbudd left a comment

Choose a reason for hiding this comment

AppVeyorBot commented Mar 8, 2022

LeonarddeR commented Mar 8, 2022

LeonarddeR commented Mar 8, 2022

michaelDCurran commented Mar 8, 2022

LeonarddeR commented Mar 8, 2022

michaelDCurran commented Mar 8, 2022 via email

CyrilleB79 commented Mar 10, 2022

Qchristensen commented Mar 11, 2022

LittleStar-VIP commented Mar 11, 2022

LittleStar-VIP commented May 18, 2022

codeofdusk commented May 18, 2022 • edited Loading

LittleStar-VIP commented May 19, 2022

michaelDCurran commented Mar 6, 2022 •

edited

Loading

josephsl commented Mar 7, 2022 •

edited

Loading

codeofdusk commented May 18, 2022 •

edited

Loading