x/tools/gopls: bug - loading packages startup penalty occurs for each session with remote gopls server.

[mattysweeps]

gopls version

$ gopls version
golang.org/x/tools/gopls v0.7.5
    golang.org/x/tools/gopls@v0.7.5 h1:8Az52YwcFXTWPvrRomns1C0N+zlgTyyPKWvRazO9GG8=

My goal is to reduce the startup penalty when opening my editor (vim) for a large golang project.
An example is kubernetes. It takes 80 seconds for the "Loading packages" stage to finish. I like to open and close my editor as needed. This means each time I open my editor I might have to wait a considerable amount of time before I can explore the code.

I tried to run gopls in the background using gopls -listen 127.0.0.1:8118 and configuring my editor (nvim w/ coc) to execute gopls -remote 127.0.0.1:8118. LSP functionality works, but each time I open the editor I have the same loading penalty. This means there's no benefit to running gopls in the background.

https://pastebin.com/yfucLgc9
My coc-ssettings.json config.

Log message I see:
https://github.com/golang/tools/blob/master/internal/lsp/general.go#L233

It might seem that the gopls -remote command is the problem, but I've done some testing to show that's not the case. I created a small test program which reads/writes back and forth from stdin/stdout to a TCP connection with the gopls -address 127.0.0.1:8118 server.

https://go.dev/play/p/WM2R5Q5borF

I compiled this program and configured my editor to invoke it when loading the LSP for go code. Again, the LSP functionality works, so the connection is good. However, the loading packages penalty persists, meaning this is in fact happening server-side for each session I make.

[findleyr]

@mattysweeps are you closing and then opening-up vim? If so, the type-checked packages will be aggressively dropped from cache, and you'll pay the startup cost anew. This is a behavior we could change, but that's how it works now.

If you're opening multiple vim sessions simultaneously, I think we should get cache hits for a lot of that startup cost, so there might be a bug. We can't cache go list, so you'll always pay that price, but that should only be a fraction (say ~1/4th) of startup.

[mattysweeps]

@mattysweeps are you closing and then opening-up vim? If so, the type-checked packages will be aggressively dropped from cache, and you'll pay the startup cost anew. This is a behavior we could change, but that's how it works now.

If you're opening multiple vim sessions simultaneously, I think we should get cache hits for a lot of that startup cost, so there might be a bug. We can't cache go list, so you'll always pay that price, but that should only be a fraction (say ~1/4th) of startup.

The former: I'm opening and closing vim. One thing to add: it doesn't seem like there's any difference in penalty based on when I open the window. The penalty is the same regardless of the remote gopls was just opened, or opened minutes ago.

I just tested opening two vim sessions in parallel using the same remote gopls. The results are the same: the second window takes an equal amount of time to load packages

[findleyr]

I just tested opening two vim sessions in parallel using the same remote gopls. The results are the same: the second window takes an equal amount of time to load packages

Ack, thanks for testing.

[findleyr]

This is fixed in the gopls@v0.12.0-pre.1 release: gopls persists derived package information to a shared file cache.