encodedSemanticClassifications-full returns empty results for JavaScript file

[mjbvz]

Bug Report

🔎 Search Terms

• encodedSemanticClassifications-full
• semantic highlighting

🕗 Version & Regression Information

4.3.0-dev.20210305

⏯ Playground Link

Playground link with relevant code

💻 Repo

1. In the VS Code codebase
2. Open a ts file from the core project (such as src/vs/base/common/strings.ts)
3. Now open build/gulpfile.editor.js
4. Look for semantic token requests

Bug

encodedSemanticClassifications-full seems to always return empty results for this file:

Info 3889 [12:03:15.050] request:
    {"seq":140,"type":"request","command":"encodedSemanticClassifications-full","arguments":{"file":"/Users/matb/projects/vscode/build/gulpfile.editor.js","start":909,"length":54,"format":"2020"}}
Perf 3890 [12:03:15.050] 140::encodedSemanticClassifications-full: elapsed time (in milliseconds) 0.1368
Info 3891 [12:03:15.050] response:
    {"seq":0,"type":"response","command":"encodedSemanticClassifications-full","request_seq":140,"success":true,"body":{"spans":[],"endOfLineState":0}}

/cc @aeschli

[orta]

Looks like this is intentional, #11404

Given that in the last 5 years, there's been quite a lot of improvements to the JS parsing, I'll remove the TS only checks and see how that turns out.

[orta]

So, https://github.com/orta/TypeScript/pull/new/sem_high_js switches out the hardcoded check for TS/JS with a check for whether the program and sourcefile exists. The tricky thing here is that in the existing JS + semantic tests there isn't a corresponding sourcefile in the program for that file because there isn't the project-wide type checker enabled in JS files by default I believe. Which may mean that it would only do semantic analysis for projects with allowJs turned on.

Gonna pair with someone to get a better sense of the trade-offs for this

[orta]

I asked, people were surprised that I didn't have a reference to the project and we think it might be more of a testing issue rather than an actual blocker. Will take a look tomorrow in production and try get some new tests.