Proposal: Revert settings search in the Settings editor to order results by keyword matches first

[rzhao271]

This proposal affects Settings editor (Settings UI) users and extension authors.

The search in the Settings editor currently shows results sorted by the table of contents so that commonly used settings can show up at the top, and so search results can be grouped by the entries in the table of contents. The change to sort the settings by table of contents was made as part of #187959 and #190666.

However, the Settings editor performs two searches per query: the first search finds settings based on keyword matching, and the second search attempts to find relevant settings using similarity search algorithms. As a result, the second search is prone to finding less relevant settings, especially when the user had a specific keyword in mind. Because the Settings editor currently orders settings by the table of contents, the less relevant results from the second search sometimes end up showing above the keyword match results because they appear first in the table of contents.

As an example, if I search "title bar style" in the Settings editor, window.titleBarStyle actually shows up fifth due to table of contents sorting:

This issue proposes reverting the settings search so that the keyword matches always appear at the top of the search results again. As part of this proposal, #187959 and #190666 would regress and become out of scope. In other words, ordering the search results so that keyword matches always appear first would break grouping the search results by the table of contents entries, but would make it so that users hopefully don't have to scroll down as much to find what they were looking for. Implementing this proposal would also break sorting extension search results by the settings order numbers given by the extension author in their extension's package.json file.

I noticed that the issue of having to scroll down to find a keyword match is made worse when the Settings editor is "small", such as when I restore/shrink the VS Code window.
Screenreader users might also have to tab down more search results to get to the one they want.

Feel free to upvote the proposal if you'd like to see it implemented, or downvote it if you'd like the settings search results ordering to remain the way it currently is. You're also welcome to comment on this issue with alternative suggestions for how the search results should be shown within the Settings editor.

[rzhao271]

Pinging authors of the issues above along with those who I have seen express interest in reverting the behaviour: @333fred @reddy1508 @TylerLeonhardt @sbatten

[rzhao271]

Another idea I have rather than doing a full revert is to use table of contents order as a last resort sorting key for the keyword matches. Therefore, keyword matches in the title could come before keyword matches in the description, and the settings that all have a description-only keyword match could be sorted amongst themselves by table of contents order. With this concept, some extension settings could preserve their order, but an extension setting found by keyword matching would still always come before an extension setting found by a similarity algorithm.

[indrareddy15]

To revert the search results in the Settings editor to order by keyword matches first in the Visual Studio Code extension, you can follow these steps

Open the Settings editor: You can do this by pressing the Ctrl+, (Windows, Linux) or Cmd+, (macOS) keyboard shortcut, or by clicking on the gear icon in the Activity Bar and selecting "Settings" from the dropdown menu.

Search for the "workbench.settings.settingsSearchTocBehavior" setting: In the search bar at the top of the Settings editor, type "workbench.settings.settingsSearchTocBehavior".

Change the value of the setting to "hide": Click on the dropdown menu next to the setting and select "hide". This will disable the table of contents ordering and prioritize keyword matches.

Restart Visual Studio Code: To apply the changes, you need to restart Visual Studio Code. Once you restart, the search results in the Settings editor will be ordered by keyword matches first.

[rzhao271]

@indrareddy15 that behaviour was implemented as part of #193242, and yes, it is a workaround in a sense.

This proposal is on ordering results by keyword matches first even when the table of contents is shown.

[indrareddy15]

@rzhao271 Try doing this way

In the package.json file, add the following property to the contributes section:
{
"name": "revert-settings-search-to-keyword-matches-first",
"version": "1.0.0",
"description": "Reverts settings search in the Settings editor to order results by keyword matches first.",
"publisher": "your-name",
"categories": ["Settings"],
"activationEvents": ["onStartup"],
"contributes": {
"configuration": {
"properties": {
"search.settings.sortByKeywordMatchesFirst": {
"type": "boolean",
"default": true
}
}
}
},
"main": "./extension.js"
}

2. In the extension's main script file, add the following code:
   const vscode = require('vscode');

vscode.extensions.onDidChangeConfiguration(() => {
const sortByKeywordMatchesFirst = vscode.workspace.getConfiguration('search').get('settings.sortByKeywordMatchesFirst');

// Update the Settings editor's search sorting algorithm here.
});

[McSodbrenner]

Since I am coming from #158720, it would be important to me that the order determined by the extension is retained when only "@ext:*" is called (as you also suggested in #158720). This then serves as a filter for the items in the base order.
If you then place a keyword behind it, you can search "smartly" again.