Marketplace search for file extensions doesn't give good enough results

[ramya-rao-a]

• VSCode Version: 1.19.2
• OS Version: Any

Steps to Reproduce:

1. Create a file test.stylus.
2. The mode is Plain Text. Click on it and then select Select Marketplace Extensions for .stylus...
3. The result is a single extension

4. Now search for "stylus" in the extensions viewlet. The search results has many more extensions with the top one having higher install count and rating than the one in the previous step

[ramya-rao-a]

@sandy081 behind the scenes the the search text being formed is tag:"__ext_stylus". If stylus was one of the supported mimetypes, then the request would have been tag:"__ext_stylus" tag:stylus which yields better results, but the order would not yet desirable

The language-stylus should have been at the top

The difference between the search terms tag:stylus and stylus is the final filter type. The results are the same but in different order

[sandy081]

@ramya-rao-a First of all, While generating the VSIX, we generate internal tags in the format __ext_<ext>. <ext> is read from extensions section under language contributions. So if an extension contribute a language with associated file extensions say .stylus then we generate tag __ext_.stylus.

Note that all extensions contributions under languages section will have . in the beginning.

Looks like we have a bug in our VS Code that we are not adding . while generating the query for extension. For eg., our query is always __ext_stylus and hence it is not finding some extensions with .stylus lang extension contributions. Added a fix for this.

After fixing here are the results when I do Select Marketplace Extensions for .stylus...

stylus extension is at the bottom because, it is not properly tagged. When I looked into its vsixmanifest file, it does not have our internal tags for extensions. Hence it is at the bottom. This can be confirmed by using the query tag:__ext_.stylus. I guess that extension was not packaged properly, hence did not have appropriate tags. May not be maintained too as it was last updated two years back.

I assume, after my fix extensions are returned in proper order. Check it out.

[sandy081]

@ramya-rao-a I think part of my assumption is partially wrong in the above comment. VSCE was generating the tags from file extensions with . character. But there was a fix couple of months ago to remove non-word characters.

So, this resulted in two kinds of tags for file extensions for a language extension, one with . (_ext.stylus) and other without (__ext_stylus).

Update the fix to support both kinds of tags.

[ramya-rao-a]

@sandy081 Though the current fix does fix the issue for extensions with tags starting from ., it doesnt address my initial concern as described in #41781 (comment)

When you have an extension installed to support stylus and then do the search, you get the results similar to #41781 (comment)

But, do the same search after disabling/uninstalling the extension that supports stylus, then you end up with only 2 results

The root cause is that for languages with known mimetypes, tag:languageName gets added here: https://github.com/Microsoft/vscode/blob/de220368be1cf08ece9d5a9bd8bb858f1e921994/src/vs/workbench/parts/extensions/electron-browser/extensionsViews.ts#L231-L237

And for languages with unknown mimetypes, this additional tag is missing in the search.

I suggest to change const languageTag = languageName ? ` tag:"${languageName}"` : '';
to const languageTag = ` tag:"${languageName || ext}" ;

[sandy081]

@ramya-rao-a file extension and language id cannot be same always. So calling that a language tag might not be right. Instead we can append the extension tag to the query.

`tag:"__ext_${ext}" tag:"__ext_.${ext}" tag: ${ext} ${keywords.map(tag => `tag:"${tag}"`).join(' ')}${languageTag}`; 

I think it makes sense to also get extensions tagged by file extension.

[ramya-rao-a]

Works well for stylus now, but there is noise for other file types, mostly due to bad tagging, will dig deeper
Thanks!