Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/gopls: bad performance with multiple workspace folders #40295

Closed
bensussman opened this issue Jul 17, 2020 · 8 comments
Closed

x/tools/gopls: bad performance with multiple workspace folders #40295

bensussman opened this issue Jul 17, 2020 · 8 comments
Labels
Milestone

Comments

@bensussman
Copy link

@bensussman bensussman commented Jul 17, 2020

Describe what you observed.

GPLS uses 200% of my CPU (both cores) indefinitely whenever I rebuild my golang in my terminal. The only way to make my computer responsive again is to force-quit gopls. Importantly, the usability of gopls (go-to-definition in VSCode for example) is broken until I force-quit the process enough times such that the CPU goes down. Only then does it actually work.

Please attach the stack trace from the crash.
A window with the error message should have popped up in the lower half of your screen.
Please copy the stack trace from that window and paste it in this issue.

[Info  - 3:45:30 PM] 2020/07/17 15:45:30 go/packages.Load
	snapshot=0
	directory=/Users/ben/spell/src/spell
	query=[./... builtin]
	packages=191

[Info  - 3:45:36 PM] 2020/07/17 15:45:36 go/packages.Load
	snapshot=6
	directory=/Users/ben/spell
	query=[./]
	packages=1

[Info  - 3:45:37 PM] 2020/07/17 15:45:37 go/packages.Load
	snapshot=6
	directory=/Users/ben/spell
	query=[file=/Users/ben/spell/docker/runs/save/main.go file=/Users/ben/spell/src/spell/constants/paths/paths.go file=/Users/ben/spell/src/spell/db/run.go file=/Users/ben/spell/src/spell/machine/remote_operations.go file=/Users/ben/spell/src/spell/runorchestrator/worker.go file=/Users/ben/spell/src/spell/servers/api/resources/ls.go]
	packages=1

[Info  - 3:45:37 PM] 2020/07/17 15:45:37 go/packages.Load
	snapshot=6
	package_path="command-line-arguments"
	files=[/Users/ben/spell/docker/runs/save/main.go]

[Error - 3:45:37 PM] 2020/07/17 15:45:37 no dep handle: no metadata for gocloud.dev/blob
	package="gocloud.dev/blob"

[Error - 3:45:37 PM] 2020/07/17 15:45:37 no dep handle: no metadata for gocloud.dev/blob/s3blob
	package="gocloud.dev/blob/s3blob"

[Error - 3:45:37 PM] 2020/07/17 15:45:37 no dep handle: no metadata for gocloud.dev/blob/gcsblob
	package="gocloud.dev/blob/gcsblob"

[Error - 3:45:37 PM] 2020/07/17 15:45:37 no dep handle: no metadata for github.com/fluent/fluent-logger-golang/fluent
	package="github.com/fluent/fluent-logger-golang/fluent"

[Error - 3:45:37 PM] 2020/07/17 15:45:37 no dep handle: no metadata for github.com/aws/aws-sdk-go/aws/awserr
	package="github.com/aws/aws-sdk-go/aws/awserr"

[Error - 3:45:40 PM] Connection to server got closed. Server will not be restarted.
@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Jul 17, 2020

It looks like you may opening a directory that gopls doesn't recognize as being inside of a module or your GOPATH. If you're using modules, make sure to open VS Code at the root of your module (instead of a parent directory) - this is a current limitation that we're working on addressing.

More verbose logs would be helpful here - details on how to capture them here.

@bensussman
Copy link
Author

@bensussman bensussman commented Jul 17, 2020

Thanks @stamblerre

feel free to close this, i just clicked the "report" button and let it do its thing.

For the record, i have a large code base that looks kinda like this:

/code
  /python
  /js
  /golang
    /github.com
    /golang.org
    /myapp
       <my golang app code lives in here>

I have a single VSCode project open to the root of /code AND i did "Add Folder To Workspace" of /code/golang/myapp such that in my sidebar i have code and myapp as two "folders in the workspace". This was necessary to get gopls to work (i believe for the exact reason you described, such that the GOPATH is the root of the "workspace folder") but it feels like it barely works: it's constantly crashing, going super slowly, reporting errors. I frequently kill gopls in task manager, and reboot VSCode in order to get performance back (temporarily).

I wonder if there is a superior way to have my multi-language project open in a single VS code window (so i can easily cmd+p find a file or cmd+shift+f search through all my files) while using tools like gopls that need the root of the project to be the base of the golang project (and not the root of all my code).

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Jul 17, 2020

Does the myapp directory contain a go.mod file or is /code/golang/myapp your GOPATH? If it's your GOPATH, you may have better luck adding /code/golang/myapp/src as your workspace folder - we avoid loading all of the packages under a GOPATH to avoid using too much CPU/memory.

If you can share the more details logs (https://github.com/golang/tools/blob/master/gopls/doc/troubleshooting.md#capturing-logs), I'll probably be able to identify the issue a bit quicker.

@bensussman
Copy link
Author

@bensussman bensussman commented Jul 17, 2020

yes the myapp directory contains go.mod and go.sum. I don't think i have a GOPATH set at all (as i was under the impression this was a legacy concept?)

The list i posted above are not the actual names, i renamed them for ease of conversation, but it seems like src is a standard name? Here are the actual names:

~/spell
  /python
  /js
  /src
    /github.com
    /golang.org
    /spell
       go.mod
       go.sum
       ....
       <more golang app code lives in here>

Here are the gopls (server) logs:

[Info  - 6:16:09 PM] 2020/07/17 18:16:09 Build info
----------
golang.org/x/tools/gopls 0.4.3
    golang.org/x/tools/gopls@v0.4.3 h1:irz7Q+XdHNECamFKbNWKvMV2Ak6zBbwdwbZndG4545I=
    github.com/BurntSushi/toml@v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
    github.com/sergi/go-diff@v1.1.0 h1:we8PVUC3FE2uYfodKH/nBHMSetSfHDR6scGdBi+erh0=
    golang.org/x/mod@v0.2.0 h1:KU7oHjnv3XNWfa5COkzUifxZmxp1TyI7ImMXqFxLwvQ=
    golang.org/x/sync@v0.0.0-20190911185100-cd5d95a43a6e h1:vcxGaoTs7kV8m5Np9uUNQin4BrLOthgV7252N8V+FwY=
    golang.org/x/tools@v0.0.0-20200708181441-6004c8539734 h1:Vc0Vx98oU/O3+qPQ36fnTT5UduS55KLh2uSGbL7mqEo=
    golang.org/x/xerrors@v0.0.0-20191204190536-9bdfabe68543 h1:E7g+9GITq07hpfrRu66IVDexMakfv52eLZ2CXBWiKr4=
    honnef.co/go/tools@v0.0.1-2020.1.4 h1:UoveltGrhghAA7ePc+e+QYDHXrBps2PqFZiHkGR/xK8=
    mvdan.cc/xurls/v2@v2.2.0 h1:NSZPykBXJFCetGZykLAxaL6SIpvbVy/UFEniIfHAa8A=

Go info
-------
go version go1.13.6 darwin/amd64



[Info  - 6:16:09 PM] 2020/07/17 18:16:09 go/packages.Load
	snapshot=0
	directory=/Users/ben/spell
	query=[./ builtin]
	packages=2

[Info  - 6:16:09 PM] 2020/07/17 18:16:09 go env for /Users/ben/spell
(valid build configuration = false)
(build flags: [])
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ben/Library/Caches/go-build"
GOENV="/Users/ben/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/ben/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13.6/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13.6/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/68/c2qg0pm111l1m549k_dntx080000gn/T/go-build312012540=/tmp/go-build -gno-record-gcc-switches -fno-common"


[Info  - 6:16:09 PM] 2020/07/17 18:16:09 go/packages.Load
	snapshot=0
	directory=/Users/ben/spell
	query=[./]
	packages=1

[Info  - 6:16:10 PM] 2020/07/17 18:16:10 go env for /Users/ben/spell/src/spell
(valid build configuration = true)
(build flags: [])
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ben/Library/Caches/go-build"
GOENV="/Users/ben/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/ben/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13.6/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13.6/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/ben/spell/src/spell/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/68/c2qg0pm111l1m549k_dntx080000gn/T/go-build272018563=/tmp/go-build -gno-record-gcc-switches -fno-common"


[Info  - 6:16:16 PM] 2020/07/17 18:16:15 go/packages.Load
	snapshot=0
	directory=/Users/ben/spell/src/spell
	query=[./... builtin]
	packages=191

[Info  - 6:16:52 PM] 2020/07/17 18:16:52 background imports cache refresh starting

[Info  - 6:16:59 PM] 2020/07/17 18:16:59 background refresh finished after 6.639360654s

[Error - 6:42:11 PM] Request textDocument/foldingRange failed.
  Message: invalid pos
  Code: 0 
[Error - 6:42:11 PM] Request textDocument/codeAction failed.
  Message: computing fix edits: /Users/ben/spell/src/spell/runorchestrator/worker.go:943:121: missing ',' before newline in argument list (and 1265 more errors)
  Code: 0 
[Error - 6:42:52 PM] Request textDocument/foldingRange failed.
  Message: no parsed file for file:///Users/ben/spell/src/spell/runorchestrator/worker.go
  Code: 0 

EDIT: looks like i do have a GOPATH at ~/go/ TBH i'd like to delete that, as ideally i have all my golang code for this project be colocated in the ~/spell/src directory (thats what i thought was happening with the ~/spell/src/github.com dir for example). Unclear to me if i can just remove it, or if it's helping / hurting.

@bensussman bensussman changed the title gopls: automated issue report (crash) gopls: high cpu and bad performance on large project with non-golang code located in adjacent directory and workspace including all code Jul 17, 2020
@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Jul 19, 2020

EDIT: looks like i do have a GOPATH at ~/go/ TBH i'd like to delete that, as ideally i have all my golang code for this project be colocated in the ~/spell/src directory (thats what i thought was happening with the ~/spell/src/github.com dir for example). Unclear to me if i can just remove it, or if it's helping / hurting.

Don't worry about that - it's the default value, and it's having no effect if you're using modules.

I wonder if the high CPU usage is caused by the fact that you have a multiple workspaces. Does gopls work well if you only open the directory containing your module?

@stamblerre stamblerre transferred this issue from golang/vscode-go Jul 19, 2020
@stamblerre stamblerre changed the title gopls: high cpu and bad performance on large project with non-golang code located in adjacent directory and workspace including all code x/tools/gopls: bad performance with multiple workspace folders Jul 19, 2020
@gopherbot gopherbot added this to the Unreleased milestone Jul 19, 2020
@bensussman
Copy link
Author

@bensussman bensussman commented Jul 20, 2020

Yes @stamblerre it does seem like closing out my multi-workspace-folder VSCode instance, and instead opening just the directory that contains my golang source code (in my case ~/spell/src/spell, where my go.mod lives) has significantly improved performance and CPU utilization.

This is not a great solution for me, because it means that I have to have at least two windows open:

  1. with Golang code dir
  2. with everything else (and this has to be a multi-directory workspace because python and js are siblings).

I think an ideal fix would be to explicitly enable or disable golang/gopls within each directory. That way i could have my one-giant-workspace, but gopls would only be enabled for the one golang directory and not the parent.

@stamblerre stamblerre modified the milestones: Unreleased, gopls/v0.5.0 Jul 20, 2020
@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Jul 21, 2020

Glad to hear it's working better. I agree it's not an ideal solution, and our plan is to address this shortcoming of gopls for the v1.0 release (#32394). Our plan is to make sure that gopls works without needing to add your module as a workspace folder, so I think that will resolve this issue. Unfortunately, you may need to use this workaround for the time being.

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Jul 21, 2020

Closing this, as it will be addressed with the more fundamental shifts in v1.0.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.