[Excel- Add @mention]: Screen Reader Users are not Notified when Search Suggestions Appear after Typing "@" in the comment

[msftedad]

Steps to reproduce:

Step 1. Turn on NVDA.
Step 2. Open the Excel document.
Step 3. Navigate to 'Comments' by using 'Ctrl+F6', 'Tab' & 'Right/Left' arrow keys.
Step 4: Press Enter key to activate the Comment button.
Step 5: Navigate to any Comment & activate it.
Step 6: Navigate to '@mention or reply,' edit field, start typing "@" and observe the issue.

Actual behavior:

NVDA is not announcing search suggestions are available after typing some characters in '@mention or reply,' edit field and while navigating on search results NVDA just announcing as "N" and it's not announcing the username and user mail ID.

Expected behavior:

For better screen reader user experience NVDA should announce the Serach results information, so that user can select the desired person and send the comment easily.
For example, NVDA should announces "10 Search suggestions available" and while navigating those results NVDA should announce the user information like name of the person and his mail ID as well.

System configuration

NVDA installed/portable/running from source:

Test Environment:
OS Version: Dev (OS Build 22458.1000)
Application Version: 2111 (Build 14606.20000) (Dogfood) (InnerRing)
Application: Excel
Screen Reader: NVDA

NVDA version:

2021.3.5

Windows version:

OS Version: Dev (OS Build 22458.1000)

Name and version of other software in use when reproducing the issue:

Other information about your system:

Other questions

Does the issue still occur after restarting your computer?

Have you tried any other versions of NVDA? If so, please report their behaviors.

If NVDA add-ons are disabled, is your problem still occurring?

Does the issue still occur after you run the COM Registration Fixing Tool in NVDA's tools menu?

[michaelDCurran]

Technical info:
The edit field:

• has a UIA automationId of cardEditor_5_18e12c3c-2f8c-4385-b051-bd809efbd8ca.
• does correctly expose the controllerFor property pointing to the container holding the suggestions.
• correctly fires the controllerFor propertyChange event
• We could map it to the SearchField UIA NVDAObject if automationId startswith cardEditor_
• This would at very least give us the suggestion open and close sounds.

suggestion list item:

• Does correctly fire the elementSelected event
• Does not have a valid selectionContainer, however, its parent is exposed in the edit field's controllerFor list.
• Does not provide any useful UIA className or automationId to easily identify it. These are all just NetUIElements.

I think it may be high time we start to consider a much more generic approach to supporting this suggestion pattern. Not only because in this instance suitable classNames / automationIds are not present, but also this controllerFor containing the parent/selectionContainer of a selected element is becoming very common, including in Chromium's UI and other web ARIA implementations.

I'm imagining we may want to move the functionality of SearchField onto behaviours.EditableText, so that any editableText NVDAObject that starts / stops exposing controllerfor will get the suggestion open/close sounds, and move the functionality of SuggestionListItem on to the base NVDAObject, so that if selected, and its parent/container matches controllerfor of the focus, it will be reported appropriately. Avoiding further need for hardcoding specific classNames etc.
Note that I'm suggesting the base NVDAObject here and not say a generic ListItem NVDAObject, as there are cases, such as in the Chrome UI that select things other than listItems. E.g. the buttons like "Switch to tab".
What do you think about this @josephsl ? As you have maintained much of the suggestions code in the past.
If we think this goes too far for now, we can instead just do this for UIA, and specifically map SearchField to cardEditor_ and add SuggestionListItem functionality to ListItem UIA NVDAObject, but specifically checking that parent and or selectionContainer are in focus.controllerfor. And we can leave SuggestionListItem as is.

[josephsl]

Hi, I think a generic approach is doable and should be tried in 2022.4 and/or 2023.1, as I imagine this would be useful in IA2 as well if Firefox and friends expose similar API. The generic approach would also resolve the UIA Automation Id/class name problem (currently Windows 10/11 People app does expose auto-suggest accessibility but with a different Automation Id), as well as removing the need to define suggestion list/item overlay classes in app modules (File Explorer app module does this, meant to support Windows 8.x Start screen search results announcement, superseded by UIA search field/suggestions list). I’m wondering if the suggestions list in Excel fires UIA layout invalidated event, and if yes, this is something we should keep in mind. CC @derekriemer as I remember him talking about Notepad++ suggestions list, which can benefit greatly from generic implementation albeit with customizations required. Thanks.

[josephsl]

Also worth mentioning: #13772 From: Michael Curran ***@***.***> Sent: Monday, September 19, 2022 1:30 AM To: nvaccess/nvda ***@***.***> Cc: Joseph Lee ***@***.***>; Mention ***@***.***> Subject: Re: [nvaccess/nvda] [Excel- Add @mention]: Screen Reader Users are not Notified when Search Suggestions Appear after Typing "@" in the comment (Issue #13764) Technical info: The edit field: * has a UIA automationId of cardEditor_5_18e12c3c-2f8c-4385-b051-bd809efbd8ca. * does correctly expose the controllerFor property pointing to the container holding the suggestions. * correctly fires the controllerFor propertyChange event * We could map it to the SearchField UIA NVDAObject if automationId startswith cardEditor_ * This would at very least give us the suggestion open and close sounds. suggestion list item: * Does correctly fire the elementSelected event * Does not have a valid selectionContainer, however, its parent is exposed in the edit field's controllerFor list. * Does not provide any useful UIA className or automationId to easily identify it. These are all just NetUIElements. I think it may be high time we start to consider a much more generic approach to supporting this suggestion pattern. Not only because in this instance suitable classNames / automationIds are not present, but also this controllerFor containing the parent/selectionContainer of a selected element is becoming very common, including in Chromium's UI and other web ARIA implementations. I'm imagining we may want to move the functionality of SearchField onto behaviours.EditableText, so that any editableText NVDAObject that starts / stops exposing controllerfor will get the suggestion open/close sounds, and move the functionality of SuggestionListItem on to the base NVDAObject, so that if selected, and its parent/container matches controllerfor of the focus, it will be reported appropriately. Avoiding further need for hardcoding specific classNames etc. Note that I'm suggesting the base NVDAObject here and not say a generic ListItem NVDAObject, as there are cases, such as in the Chrome UI that select things other than listItems. E.g. the buttons like "Switch to tab". What do you think about this @josephsl <https://github.com/josephsl> ? As you have maintained much of the suggestions code in the past. If we think this goes too far for now, we can instead just do this for UIA, and specifically map SearchField to cardEditor_ and add SuggestionListItem functionality to ListItem UIA NVDAObject, but specifically checking that parent and or selectionContainer are in focus.controllerfor. And we can leave SuggestionListItem as is. — Reply to this email directly, view it on GitHub <#13764 (comment)> , or unsubscribe <https://github.com/notifications/unsubscribe-auth/AB4AXEGKTMPAMHXCCPITW6DV7AP7LANCNFSM5XV5ACNA> . You are receiving this because you were mentioned.Message ID: ***@***.***>