-
Notifications
You must be signed in to change notification settings - Fork 72
Scanning all files in the rootUri makes the server unusable for directories with many files #471
Comments
Exactly.
No, because the file content don't have to come from disk, they may be requested through LSP too, which is always async. |
It sounds like you're saying LSP supports having the server request file contents from the client. However, I don't see that in the protocol. Could you elaborate on what you're referring to? The protocol allows the client to push file content to the server, but obviously after those are stored in memory on the server, they can be served synchronously to Typescript. Looking at how Update: I see now that you've extended LSP with file requests from server to client, such as Have you made attempts to get your LSP extensions merged into LSP? Anyways, I feel that in the scenario that you're not using remote file fetching, this LSP server should stop prefetching and simply use Update: |
@keyboardDrummer seems like you found the answers to most of your questions. To summarize, to support thousands of clients browsing repos at different revisions at the same time, we run language servers in containers, acting as TCP servers (speaking LSP). For performance reasons, the containers don't have access to a fully checked-out workspace on disk, but fetch the files they need on-demand, over the LSP files extension (spec here).
The files are actually not checked out - that would be too expensive for big monorepos, because latest master changes all the time and every connection would require a fresh
We have not for this extension, because every prior attempt to merge things into LSP that didn't solve any problem for VS Code wasn't successful. E.g. see microsoft/language-server-protocol#245 or microsoft/language-server-protocol#182 (comment) (not blaming anyone here, but figuring these things out properly for the official spec is hard, especially when there is little interest from other parties). It may be worth to revisit now, since VS Code is gaining support for file providers to enable e.g. working over FTP. But that wouldn't solve the unvelrying problem in TypeScript: As you mentioned, TypeScript's
I'm suprised that Now of course I would love to get rid of the prefetching. It's a thorn in my eye. To somewhat improve performance, we use heuristics to not fetch all files, but only ones that are TypeScript files belonging to the project and some that "look" like global declaration files (e.g. |
I have a package directory that contains about 200k files, many of which are generated and are not required for providing Typescript language tooling. The
glob
npm package used by this LSP server takes ~45seconds to scan all files in the package directory. During this time the LSP server will not respond to requests, making it unusable.I've compared using the 'glob' package to using the 'find' package. The latter completes in about 1 second, which would be sufficient to make the LSP server usable. However, the
find
package finds slightly different files thanglob
. It seems to include also files in 'hidden' directories such as .git, but then skips files in symlinked directories.Lastly, running
find -type f | wc -l
in my package directory completes instantly.I think the root cause of this issue is that the LSP server aggressively scans all files in the rootUri. This is also a problem when requiring files from outside the rootUri directory. I see that the
LanguageServiceHost
interface that you need to implement for the Typescript compiler contains a synchronousreadFile
method. Is that why you preload all the existing files? Can you not just synchronously read the file at that time?Have you looked at what
tsc
does? It finishes running on this package in a few seconds. Also it allows requiring packages outside of thetsconfig.json
folder.The text was updated successfully, but these errors were encountered: