typescript-language-features doesn't seem to work on virtual file system

[aslatter]

• VSCode Version: 1.27.2
• OS Version: Windows 10 Version 1803

Steps to Reproduce:

1. Install the MemFS demo file system provider extension, follow the instructions to set up a MemFS folder
2. Add a new TypeScript file to the MemFS folder, write some example TypeScript code

I expected that auto-complete, signature help, and hover information would be available in the TypeScript file, but it was not. Basic syntax-highliting and some snippets were available, however.

Does this issue occur when all extensions are disabled?: N/A

I thought I had seen a bug for this but I can't find it anymore. Currently extensions/typescript-language-features/src/typescriptServiceClient.ts has explicit checks in the vscode.Uri to block non-file schemes (normalizedPath and getWorkspaceRootForResource). I'm not sure what else needs to be fixed if those checks are removed.

[vscodebot]

(Experimental duplicate detection)
Thanks for submitting this issue. Please also check if it is already covered by an existing one, like:

• vscode.workspace.findFiles doesn't search in user provided file systems (#55982)
• Startup issue, file system provider and new workspace – no provider for memfs:/ (#57989)

[aslatter]

FYI - I'm not interested specifically in MemFS - however we're working on a file system provider we intend to use to offer up a read-only view of TypeScript files for remote viewing and debugging. Having language server support for these would be a big help for us in this.

[DemonOne]

I ran into this issue just now trying to use python language support with the virtual file system example, same thing happens, no completions and populating python.autoComplete.extraPaths doesn't help.

[aslatter]

From digging around in the vscode sources and debugging, the problem with the TypeScript language-service is specific to it - it goes out of its way to prevent itself from working over a virtual file-system.

Any extension which does not support vfs likely needs its own fix, specific to that extension, so if the python support is provided by an extension you may want to file an issue to that extension as well.

[julien-moreau]

Hey @mjbvz
Do you have any news about this issue? Saying "no" is a completely acceptable answer :)
Thanks

[mjbvz]

@julien-moreau This is not scheduled and there's no active eta. There's a risk of breaking existing extensions like live share and after almost half a year this issue has zero upvotes

[julien-moreau]

@mjbvz thanks for your answer :)
Looks ok to me to be a feature request.
Do you have any suggestion or a tip today to activate these languages features? I guess live share has a tip or is it working on the "file" scheme?

[foorschtbar]

@julien-moreau This is not scheduled and there's no active eta. There's a risk of breaking existing extensions like live share and after almost half a year this issue has zero upvotes

I have the same problem...

[terrelln]

Another +1 for this issue. The FileSystemProvder API is a great idea... But it needs first class support if it is going to be more than a toy. Lacking Typescript language support isn't a great sign, because if Microsoft isn't going to make the effort to support FileSystemProviders, who is?

[gjsjohnmurray]

I've got my fingers crossed that the landing of DiskFileSystemProvider in 1.34 may increase motivation to get this built-in extension playing nicely with all FileSystemProviders 🙏

[gjsjohnmurray]

@jrieken is this issue also relevant to your comment on #62290? So even if you enhance breadcrumb to cater for documents coming from a debugger it'll hit the roadblock of the TS/JS DocumentSymbolProvider only handling local files?

[jrieken]

it'll hit the roadblock of the TS/JS DocumentSymbolProvider only handling local files?

Sure, you will get breadcrumbs for the file path but not for symbols

[nbransby]

@julien-moreau This is not scheduled and there's no active eta. There's a risk of breaking existing extensions like live share and after almost half a year this issue has zero upvotes

@mjbvz after 12 month it now has plenty of upvotes, could we get this scheduled now?

[nbransby]

From digging around in the vscode sources and debugging, the problem with the TypeScript language-service is specific to it - it goes out of its way to prevent itself from working over a virtual file-system.

Any extension which does not support vfs likely needs its own fix, specific to that extension, so if the python support is provided by an extension you may want to file an issue to that extension as well.

@aslatter where is the code that prevent itself from working over a virtual file-system? How does liveshare manage it?

[aslatter]

where is the code that prevent itself from working over a virtual file-system?

@nbransby oh wow it's been a while since I dug in to things, so I might be remembering things wrong.

You can start by looking in typescriptServiceClient.ts and looking for references to fileSchemes.file.

I tried ripping out a couple of "if the URI is not a file-path quit" checks, and things like file-outlines started working but I didn't get much further than that.

I don't understand the codebase, but I get the impression that typescript-client is smuggling information in-band in URIs in ways that aren't simple (or were, back when I dug in!), so I think that would need to be untangled (or at minimum understood).

How does liveshare manage it?

Liveshare & vs-code remoting solves a similar problem that virtual file systems were likely intended to solve, but do so in a pretty fundamentally different way, so I don't think anything that makes those work will translate to VFS.

[nbransby]

@aslatter looking at the Liveshare source it does call vscode.workspace.registerFileSystemProvider with the scheme 'vsls' and appears to implement a FileSystemProvider, I thought maybe the vscode source had provisions for that scheme but a quick search doesn't throw anything up: https://github.com/microsoft/vscode/search?q=vsls&unscoped_q=vsls

[lostintangent]

This would be extremely valuable for the GistPad extension, which uses a GitHub Gist-backed file system provider, and enables you to create "web playgrounds" within VS Code. The lack of JS/TS language services keeps the experience from being as awesome as it could be.

[Lzw2016]

Support for smart prompts in virtual file systems, which we also need

[cspotcode]

I need support for returning non-file URIs from the TypeScript language server.

My use case is navigating into dependencies' code when using yarn 2, which keep them in zip files.
It creates a typescript language server that is capable of reading these zip files. Unfortunately, VSCode needs to be able to navigate to them as well, which means installing a zip:// TextDocumentContentProvider or FileSystem provider. I've modified the TypeScript language server to return a zip:// prefix on paths, but VSCode's TS extension is written to assume file: so it's not trying to interpret the scheme, and it's converting everything to file://

[arcanis]

@mjbvz To add to what @cspotcode said, this issue actually got about 127 upvotes in a month. You need to account the people who upvoted the Yarn 2 zip compatibility issue: #75559

The basic fix we found would only require on your side to apply the following change:

Please give it a thought - Yarn is used by many people, including Microsoft teams (including VSCode, in fact!). Helping us would significantly improve our developer workflow 🙏

[mjbvz]

This issue is not related to yarn2; this is is specifically about enabling IntelliSense in files that use custom VS Code file schemes. You should file an issue against TS directly for better yarn2 support

Also, this issue is not as simple as removing a check. Maybe for single file JS files it is but as soon as you have imports and cross file references, things would break. This is because we use a typescript server running as a separate process to power intellisense and this process has no way to read resources that use VS Code file schemes

[cspotcode]

@mjbvz There's some missing context here; I'll try to fill in the blanks as best I can. Roughly, the language service is already able to read these URIs when using yarn 2.

yarn 2 employs a modified TypeScript language service that is able to read the contents of zip files, used via VSCode's "typescript.tsdk" option. Yarn exposes a virtual filesystem to the language service. Internally, when the language service resolves dependencies, it ends up with paths like /foo/bar/something.zip/node_modules/foo/index.d.ts. It can read these paths thanks to the virtual filesystem, so everything works.

When a user hits "go to declaration", the language service gives it a path .../something.zip/foo/index.d.ts. VSCode tries and fails to retrieve this file from disk using the file: scheme.

Since we're already plugging into the language service, we can transform the paths it sends to and receives from VSCode. We can add a zip:// scheme to files that we want VSCode to read using a zip:// filesystem provider. This allows a VSCode extension to implement the zip:// scheme, so when someone navigates to a declaration that lives in a zip file, VSCode is able to open it. And when VSCode asks the language service for semantic diagnostics in that file, we translate the request back to a path that the language service can understand. Everyone is happy in theory, and VSCode does not need to implement #75559.

[mjbvz]

But only when using your special TS version, right? Are you patching TSC directly or using a plugin?

It sounds like you are describing a very specific use case that may be best tracked by a separate issue. The request of this issue is to make it so that normal, out of the box tsserver can understand these special schemes

[arcanis]

@mjbvz Aah, I see what you mean - my apologies, I misunderstood this issue, they are two separate problems indeed. I've opened a new thread at https://github.com/microsoft/vscode/issues/91142 👍

[arcanis]

@mjbvz now that VSCode May supports click to jump, #98830 is fixed. However, an issue remains where VSCode doesn't start the TypeScript server for virtual files, so the click to jump support is limited to a single jump (you can't jump from a zip archive to something else).

I understand it could have adverse effects, but I think our community would be happy to help fix and test this problem while avoiding regressions. Would you have some pointers where we should start? To start, do you have a rough idea where the TS server is spawned that could not take virtual filesystems into account?

[cspotcode]

IIRC, someone said that the problem is VSCode doesn't know if the language server can handle zip paths. If the language server sends a zip path to VSCode, then that's ok, but it can't happen the other way around. VSCode can't launch the language server under the assumption that it'll understand a zip path, because maybe it can't.

…

On Mon, Jun 1, 2020, 10:30 AM Maël Nison ***@***.***> wrote: @mjbvz <https://github.com/mjbvz> now that VSCode May supports click to jump, #98830 <#98830> is fixed. However, an issue remains where VSCode doesn't start the TypeScript server for virtual files, so the click to jump support is limited to a single jump (you can't jump from a zip archive to something else). I understand it could have adverse effects, but I think our community would be happy to help fix and test this problem. Would you have some pointers where we should start? To start, do you have a rough idea where the TS server is spawned that could not take virtual filesystems into account? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#59650 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAC35OAMIZD2BQKP27S4TALRUO3OZANCNFSM4FX6SSXQ> .

[mjbvz]

This was enabled progressively over the last few months to support serverless. Here's basic IntelliSense shown for in a file using the memfs example extension:

Notes:

• Virtual JS/TS files have single file IntelliSense, not project wide IntelliSense. The TypeScript server which provides IntelliSense runs as a separate process, and therefore it cannot read or watch files from a VS Code file system provider. Enabling project wide IntelliSense would require significant engineering work on both the VS Code and TypeScript side. We don't have a good use case to undertake this at this time.

• Some language tooling features are intentionally disabled. Rename for example is disabled because we cannot ensure that we should not have renamed occurrences in other files

• Make sure you are using the latest insiders build for testing. I just pushed two small fixes today (see Initializing JS/TS language features spinning constantly #103740 still happening #108454)

Let me know if you run into any issues with this functionality

[mjbvz]

To verify:

1. Using https://github.com/Microsoft/vscode-extension-samples/tree/master/fsprovider-sample
2. Run MemFS: setup workspace
3. Run MemFS: Create files
4. Try some basic IntelliSense operations in the JS and TS files: hover, suggest, document outline, folding

[merceyz]

Can confirm it's been working with zip files using https://marketplace.visualstudio.com/items?itemName=arcanis.vscode-zipfs for quite a while now, thanks! 🎉