Report background and highlighted text in MS Word

[ABuffEr]

Link to issue number:

Fixes #5866, fixes #7396, fixes #12101

Summary of the issue:

In these years, Microsoft Word users working without UIA had no feedback about background and highlighting colors, often used by colleagues, teachers and students.

UIA has provided a partial solution when available, but its support for these features is far from perfect (see later).

Description of user facing changes

Assuming UIA is not available/forcibly enabled:

• when color reporting is enabled in formatting preferences, users will hear background colors too;
• when marked/highlighted text reporting is enabled in formatting preferences, users will hear highlighting colors ("highlighted in {color}" and "no highlighting"), regardless of color reporting.

Description of development approach

It was very difficult to find a way, so I'll add some details.

Initially, I used VBA objects approach, querying Paragraph.Shading.BackgroundPatternColor and Range.HighlightColorIndex over a winwordDocument.Range to calculate each time, so it was very slow.

Then I found this document and, after having understood what I needed and how to find the required wdDISPID_ constants, I was able to move all work on the C++ component of nvdaHelperRemote related to winword, that appears very much faster.

In window/winword.py, to take advantage of winwordColorToNVDAColor work, I added also a dict that collects indexes from wdColorIndex and decimal values from wdColor.

About speech, I had to introduce a new format field, highlight-color.

Now, UIA: sure, it already covers background and highlight colors, but expose them as the same attribute, via UIA_BackgroundColorAttributeId (and no, I asked, even visually they are distinct). There is an AnnotationType_Highlighted identifier, but it seems to not be used in this scenario.

Locally I have a possible workaround, based on the fact that, as you see in Word Home menu, highlighting color is applied to characters, background color to paragraphs; so, if textRange has a background-color, I compare it to that retrieved from paragraph and, if I receive IUnknown/MixedAttributeValue, I assume the initial one is an highlighting color and try to get the real background color from "\r" EOL.

But I'm not totally convinced, so I not included it in this PR for now.

Testing strategy:

See #5866 str and attachment.

Unfortunately, I have access only to a working copy of Word for Microsoft 365 MSO (Version 2302 Build 16.0.16124.20000), 64 bit.

I tried to avoid any dangerous assumption in C++, according to existing code, but a lot of testing is required, maybe.

Known issues with pull request:

Apparently, highlighting is not correctly reported if it's not applied on whole word (investigating).

Highlighting announcements could be too verbose, I considered to use "marked in {color}" and "not marked", but then leaved as is for clarity against Word feature. Suggestions are welcome.

Even the fact highlighted (colored) text is not reported when color reporting is enabled could be confusing for some users, so, again, a discussion is welcome.

Change log entries:

New features
Changes
Bug fixes

For Developers

• new highlight-color format field.

Code Review Checklist:

• Pull Request description:
  • description is up to date
  • change log entries
• Testing:
  • Unit tests
  • System (end to end) tests
  • Manual testing
• API is compatible with existing add-ons.
• Documentation:
  • User Documentation
  • Developer / Technical Documentation
  • Context sensitive help for GUI changes
• UX of all users considered:
  • Speech
  • Braille
  • Low Vision
  • Different web browsers
  • Localization in other languages / culture than English
• Security precautions taken.

[Adriani90]

Wow this is great work, thank you for taking this up I was trying to find a solution for this for a very long time. This will improve the way blind and sighted people work together in a document alot.

Is there also a possibility to implement a key stroke for the browse mode to jump between highlighted text?

[cary-rowen]

Thanks, great work.
Yep, it would be nice to have a key to jump between "highlights".

[ABuffEr]

Hi,
first, thanks for appreciation :)
About quick nav, I think it's possible, but I don't know how much it's fast. I'm studying the relative code (a lot of small classes).
Have you concerns about the aspects I mentioned under "Known issues"?

[CyrilleB79]

Regarding the key to jump between highlight, that's a completely valid request. But it would better to handle this in a subsequent PR. So please, open an issue (if not yet existing) or comment in existing issue.

[LeonarddeR]

Might be related to #9527

[ABuffEr]

Thanks, I'll comment there about quick nav and a new formatting helper (before to elaborate this PR, I worked on a new, related add-on).

[ABuffEr]

Hi,
I initially declared PR as a draft, as suggested in PR template, but the committed parts are ready since I opened it.
If covering UIA is preferred, I can commit also that part, even if it's more a workaround than a solution.

[seanbudd]

Apologies, is this ready for review? This was marked as a draft.

[seanbudd]

This generally looks good to me.

[ABuffEr]

Hi @seanbudd,
with limitations I wrote in PR description (that I suggest to address separately), everything is ok for me.