UIA: add support for layout invalidated event, used to handle auto-suggest accessibility

[josephsl]

Hi,

Different issues talk about this, therefore this issue will serve as a summary issue for a possible pull request:

Overview:

Apps can provide search fields where users can search for terms. The best known example is Start menu/screen where people can search a variety of things, including files, web searches, ans system settings.

Over the years, NVDA added support for search field implementations. These include auto-complete announcements in web browsers, Start menu search results announcement, and even python Console input suggestions.

Is your feature request related to a problem? Please describe.

Back in 2017, NvDA added ability to play a sound when suggestions appear. This is done by listneing to UIA controller for event where the event returns an array of UIA elements that the focused element relies on. For example, in Windows 10/11 Start menu, controller for array for the search box is the results list.

However search suggestions announcement didn't cover a whole class of search fields and their suggestions list friends: those using UIA layout invalidated event. Layout invalidated event (introduced in Windows 8) is used to communicate changes to container elements. The best known application is detecting suggestion count, and one real-life example is Settings app (Windows+I) where a suggestions list uses this event to communicate suggestion count.

In short, the current implementation when encountering this class of search fields/suggestions list combo is:

1. User types search terms.
2. Controller for event is fired.
3. NVDA plays suggestion sound if enabled.
4. Layout invalidated event is fired.
5. User types more terms.
6. Layout invalidated event is fired each time suggestions list changes.
7. NVDA does nothing.

Describe the solution you'd like

Let NVDA announce at least suggestion count, which is applicable to Windows 8 and later provided that NVDA can be told to detect suggestion lists. Specifically:

1. Search fields: announce suggestion count #7330: let NVDA announce suggestion count.
2. Win11: NVDA is not announcing the available suggestions for “Find a setting” search field in settings page  #12758: detect suggestion lists.

Implementation strategy: a real-life implementation is part of Windows App Essentials add-on. Specifically:

1. Add UIA layout invalidated event to a dictionary of UIA events.
2. Define an overlay class for search suggestion lists. So far these have the Automation Id of "SuggestionsList".
3. The overlay class will listen to layout invalidated event, which will simply announce list item count each time it changes.

Describe alternatives you've considered

Leave the code as is. Alternatively, refine controller for event so suggestion count will be announced only when suggestions appear.

Additional context

Implementing auto-suggest: https://docs.microsoft.com/en-us/windows/apps/design/accessibility/accessible-text-requirements#implementing_auto-suggest

Thanks.

[josephsl]

Hi,

One additional task might be to add an option to toggle suggestion count announcement, either a new item or tie into dynamic content changes. I'm fine with either way - the upcoming pull request will leave room for this possibility.

Thanks.