Skip to content

Chrome: Omnibox: User is not notified of useful information in the first item of keyword search #13522

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
cdkim1212 opened this issue Mar 21, 2022 · 12 comments · Fixed by #14222
Closed

Chrome: Omnibox: User is not notified of useful information in the first item of keyword search #13522

cdkim1212 opened this issue Mar 21, 2022 · 12 comments · Fixed by #14222
Labels
app/chrome blocked/needs-external-fix close/works-for-me triaged Has been triaged, issue is waiting for implementation.

Comments

@cdkim1212
Copy link

Steps to reproduce:

  1. Turn on NVDA and Chrome Browser
  2. Activate keyword search by typing a letter that corresponds to a saved keyword search in the Omnibox

Actual behavior:

As user types, they don’t hear any useful/friendly information that the sighted user sees in item 0, such as the title of the page they will get, or the fact that there are additional buttons

Expected behavior:

The user should be notified of the title of the page and of the buttons one can tab to and activate as part of suggestions

System configuration

NVDA installed/portable/running from source:

NVDA version: 2021.3.4

Windows version: Windows 10 Version 20H2 (Build 19042.1526)

Name and version of other software in use when reproducing the issue: Chrome Browser, Chrome Version: 100.0.4878.0

Other information about your system:

Other questions

Does the issue still occur after restarting your computer?

Yes

Have you tried any other versions of NVDA? If so, please report their behaviors.

2021.3.3. Same behavior

If NVDA add-ons are disabled, is your problem still occurring?

N/A

Does the issue still occur after you run the COM Registration Fixing Tool in NVDA's tools menu?

N/A
@feerrenrut
Copy link
Contributor

See also: #13145
Related and likely to be addressed by an overhaul of user interactions with the omnibar.

@feerrenrut feerrenrut added app/chrome triaged Has been triaged, issue is waiting for implementation. labels Mar 28, 2022
@michaelDCurran michaelDCurran mentioned this issue Oct 8, 2022
Merged
6 tasks
michaelDCurran added a commit that referenced this issue Oct 17, 2022
@michaelDCurran
Make support for search suggestions more generic, supporting Excel co…
b9a8fbc 
…mment mentions and Google Chrome location bar suggestions (#14222)

Fixes #13772
Fixes #13764
Fixes #13522

Summary of the issue:
• When starting to type an @mention in a Microsoft Excel comment, NVDA does not report available suggestions when arrowing through them. ( [Excel- Add @mention]: Screen Reader Users are not Notified when Search Suggestions Appear after Typing "@" in the comment  #13764)
• When typing in the Google Chrome location bar, NVDA does not report rich information for each suggestion, nor does it report when a suggestions control (switch to tb, remove suggestion) is selected. (Chrome: Omnibox: User is not notified of useful information in the first item of keyword search #13522)
• As a developer, adding support for new suggestion scenarios is tedious as specific mappings must be always made.

Description of user facing changes
• NVDA will now report the selected suggestion when arrowing through suggestions when typing an @mention in a Microsoft Excel comment.
• NVDA will report the selected suggestion or suggestion control (switch to tab, remove suggestion) in the Google Chrome location bar.

Description of development approach
Make support for suggestions on editable text fields more generic, allowing it to be supported in more places without having to specifically choose particular overlay classes.

This adds support for reporting @mention suggestions in Microsoft Excel comments, and reporting of suggestions and related controls in the Google Chrome location bar.

Specific changes:
* IAccessible NVDAObject: implement the controllerFor property making use of IAccessible2 relations
* NVDAObject: add an isDescendantOf method which should return True if the object this method is called on is a descendant of the given obj argument. The base method raises NotImplementedError. Efficient implementations for UIA and Ia2Web have been provided.
* Implement event_selection on the base NVDAObject, which reports the object being selected, if the current focus is controlling an ancestor of the selected object. This replaces SuggestionListItem UIA NVDAObject's event_UIA_elementSelected logic. Now it is more generic, and works purely on tree topology, rather than checking the name and appModule etc.
* Remove event_selection implementation from IAccessible NvDAObject as the base handles this now.
* Add an event_controllerForchange method to the base NVDAObject which does nothing.
* UIA NVDAObject: implement event_UIA_controllerforchange to directly call event_controllerforchange, allowing for more generic code, but still backwards compatibility for subclasses to still override event_UIA_controllerforchange if they need to.
* behaviours:
	* Rename EditableTextWithSuggestions to InputFieldWithSuggestions and inherit directly from NVDAObject making this class more generic.
	* InputFieldWithSuggestions class: implement event_controllerForChange which will fire the suggestionsOpened and suggestionsClosed events based on whether there are objects in the controllerfor list. this code was taken from SearchField UIA NVDAObject's event_UIA_controllerforchange.
	* Rename EditableText to EditabletextBase
	* Add a replacement EditableTextWithSuggestions class that inherits from both InputFieldWithSuggestions and EditableTextBase
	* Add a replacement EditableText class which inherits from EditableTextwithSuggestions and EditableTextBase.
	Essentially all this builds suggestion support into EditableText, but keeps the EditableTextWithSuggestionsClass specifically available for compatibility.
@nvaccessAuto nvaccessAuto added this to the 2023.1 milestone Oct 17, 2022
@amirsol81
Copy link

@feerrenrut @michaelDCurran @seanbudd @cdkim1212 It seems to me that the implementation of this feature is incomplete as of NVDA Alpha 26827. NVDA only reads the very first suggestion as I type the first one or two letters in the Omnibox - ALT+D. So as I keep typing extra letters, even if I wait enough between subsequent letters, NVDA doesn't read the newly highlighted suggestion/website title/etc. So as long as the first suggestion with the first one or two type letters is concerned, this commit works fine, but the rest of the suggestions are simply ignored as we keep typing and new ones are highlighted. Please let me know if I should describe this with more details or examples.

@michaelDCurran
Copy link
Member

michaelDCurran commented Oct 18, 2022 via email

@nidza07
Copy link
Contributor

nidza07 commented Oct 18, 2022

Hello,
I can confirm what @amirsol81 says, here is an example:
Open Chrome and make sure you are in the omnibox.
The next steps will depend on suggestions Chrome is giving to you, but here are mine:
Type G.
NVDA gives the following suggestion:
GitHub github.com location from history, press Tab then Enter to Remove Suggestion. 1 of 8.
Type o.
The next behaviour appears to be inconsistent and random, but one of the following might happen:

  1. NVDA will report just "o unselected", completely useless and meaningless. The selected suggestion is google.com, but NVDA gives no indication about this. If I continue writing the suggestion, i.e. type another O, NVDA again reports "o unselected", and this continues as long as there is a suggestion being offered.
  2. NVDA will report "ogle.com" selected instead, because the selection has indeed changed and the next suggestion being offered is now google.com. Note that this is a little more useful, but it's not always the case, and still the entire suggestion isn't being reported, neither the page title nor where it comes from.

I have also noticed that the suggestion is always spoken correctly if you switch tabs, i.e. type g, suggestion is spoken. Go to a different tab, come back to the previous one, type O, the new suggestion is properly spoken. However, obviously this is inpractical, but it may help when debugging.

Let me know if any logs are needed, this is observed in my case under Chrome 106.0.5249.119.

@amirsol81
Copy link

amirsol81 commented Oct 19, 2022 •
edited
Loading

@michaelDCurran The examples @nidza07 provided are quite helpful, so I won't add extra examples.

@michaelDCurran
Copy link
Member

pr #14222 partially fixes this issue, as NVDA now announces the top suggestion when it first appears, and arrowing up and down the list / tabbing to the inner controls all read.
However, if continuing to type into the field causes the top suggestion to change, NVDA does not announce anything for this.
This is a limitation in the browser, as Chromium fails to fire further EVENT_OBJECT_SELECTION or nameChange events on the top suggestion. Thus NVDA has no good way of telling that the top suggestion has changed. NVDA could keep checking on typed input or on the valuechange event on the edit field, but this would be flaky at best as the list may update slower. It also deceives the whole point of the controllerFor selection event pattern.
this must be reported to Google.

@amirsol81
Copy link

@michaelDCurran Thanks for the explanation. As I don't have any Google contacts, would it be possible for you or NVAccess to report this to Google? If fixed, this would make Omnibox a whole lot more accessible.

@michaelDCurran
Copy link
Member

michaelDCurran commented Oct 20, 2022 via email

@amirsol81
Copy link

@michaelDCurran Interestingly, this is handled much more elegantly with MS Edge 106. That is, NVDA reads the newly highlighted suggestions or page titles as we keep typing letters into the Omnibox although suggestions are read with a short delay of 1 or 2 seconds. Perhaps suggestions pop up with such a delay - I'm not sure. At any event, it seems that this is more of a Google Chrome-specific issue rather than an MS Edge-oriented one.

@michaelDCurran
Copy link
Member

michaelDCurran commented Oct 20, 2022 via email

@seanbudd seanbudd removed this from the 2023.1 milestone Jan 2, 2023
@Adriani90
Copy link
Collaborator

Filled Chromium bug:
https://bugs.chromium.org/p/chromium/issues/detail?id=1433204

@Adriani90 Adriani90 mentioned this issue Apr 14, 2023
Closed
@Adriani90
Copy link
Collaborator

In Chrome 124 with NVDA 2024.1, NVDA reports the title of the page, the URL and the fact that you can press tab to remove the suggestion. This works when the suggestion is already part of your browsing history.As Chrome developers explained, it would be anyway too much verbosity to get all the suggestion related info while typing, you can navigate to the list of suggestions with the tab key and access the information. I am closing this as works for me.
If you are still having any strong arguments why NVDA should still report as much information as possible while typing, please comment and we can rediscuss.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
app/chrome blocked/needs-external-fix close/works-for-me triaged Has been triaged, issue is waiting for implementation.
Projects
None yet
Milestone
No milestone
8 participants
@feerrenrut @seanbudd @michaelDCurran @nvaccessAuto @nidza07 @Adriani90 @amirsol81 @cdkim1212