Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/gopls: not working on a monorepo #37087

Closed
stamblerre opened this issue Feb 6, 2020 · 14 comments
Closed

x/tools/gopls: not working on a monorepo #37087

stamblerre opened this issue Feb 6, 2020 · 14 comments

Comments

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Feb 6, 2020

Reported on Twitter: https://twitter.com/magiconair/status/1225358273576083456?s=20.

This is probably caused by the fact that the initial workspace load is running on the entire monorepo (if the directory being opened is the monorepo root). The output of gopls -rpc.trace -v check path/to/file.go would be helpful here, as well as the time it takes.

@stamblerre stamblerre added this to the gopls/v0.4.0 milestone Feb 6, 2020
@magiconair
Copy link
Contributor

@magiconair magiconair commented Feb 6, 2020

Thank you @stamblerre, here is the (sanitized) log. Not sure if the numbers here are enough information. I need to see how I can reproduce the errors in VSCode. However, what I've seen is that the CPU Load was constantly 50-60%, fans spinning until I've shut VSCode down. Since I've just re-installed my computer and haven't recreated all settings yet I'm running more or less with default install settings.

❯ time gopls -rpc.trace -v check driver/battery/driver.go
2020/02/06 23:41:30 Info:2020/02/06 23:41:30 Build info
----------
golang.org/x/tools/gopls v0.3.1
    golang.org/x/tools/gopls@v0.3.1 h1:yNTWrf4gc4Or0UecjOas5pzOa3BL0WDDyKDV4Wz5VaM=
    github.com/BurntSushi/toml@v0.3.1 h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
    github.com/sergi/go-diff@v1.0.0 h1:Kpca3qRNrduNnOQeazBd0ysaKrUJiIuISHxogkT9RPQ=
    golang.org/x/mod@v0.1.1-0.20191105210325-c90efee705ee h1:WG0RUwxtNT4qqaXX3DPA8zHFNm/D9xaBpxzHt1WcA/E=
    golang.org/x/sync@v0.0.0-20190911185100-cd5d95a43a6e h1:vcxGaoTs7kV8m5Np9uUNQin4BrLOthgV7252N8V+FwY=
    golang.org/x/tools@v0.0.0-20200204151227-34c67990bfe7 h1:hWZVyLW37WdETuLIGQMvQIhMfXXAOANmAIEAluZMy3c=
    golang.org/x/xerrors@v0.0.0-20191011141410-1b5146add898 h1:/atklqdjdhuosWIl6AIbOeHJjicWYPqR9bpxqxYG2pA=
    honnef.co/go/tools@v0.0.1-2019.2.3 h1:3JgtbtFHMiCmsznwGVTUWbgGov+pVqnlf1dEJTNAXeM=
    mvdan.cc/xurls/v2@v2.1.0 h1:KaMb5GLhlcSX+e+qhbRJODnUUBvlw01jt4yrjFIHAuA=

Go info
-------
go version go1.13.7 darwin/amd64

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/frank/Library/Caches/go-build"
GOENV="/Users/frank/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY="none"
GONOSUMDB="github.com/northvolt"
GOOS="darwin"
GOPATH="/Users/frank/go"
GOPRIVATE="github.com/northvolt"
GOPROXY="https://***@our.proxy.host"
GOROOT="/usr/local/Cellar/go/1.13.7/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13.7/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/frank/src/github.com/northvolt/our-repo/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/v6/zq1l3y4d39d8k7cvmf5wp02r0000gn/T/go-build394917405=/tmp/go-build -gno-record-gcc-switches -fno-common"
2020/02/06 23:41:31 Info:2020/02/06 23:41:31 go/packages.Load
	snapshot = 0
	query = [./... builtin]
	packages = 186
gopls -rpc.trace -v check driver/battery/driver.go  6.94s user 2.40s system 461% cpu 2.027 total
@magiconair
Copy link
Contributor

@magiconair magiconair commented Feb 6, 2020

some more stats in case the repo size is an issue:

github.com/AlDanial/cloc v 1.84  T=7.00 s (355.1 files/s, 119745.1 lines/s)
--------------------------------------------------------------------------------
Language                      files          blank        comment           code
--------------------------------------------------------------------------------
Go                             1799          70956          98003         644352
Markdown                         91           2812              0           8124
Assembly                         39            615           1208           2622
YAML                             85            303             93           1821
Protocol Buffers                 22            394            281           1395
Bourne Shell                     20            161            353            943
GraphQL                          16            580              3            677
JSON                            375              0              0            564
Bourne Again Shell                7             69             53            308
make                             14             88            113            270
TOML                              5            114             82            182
HTML                              2             29              0            120
Python                            2             23             11             86
Dockerfile                        5             15             12             45
C                                 1              8              7             24
diff                              1              4             30             21
INI                               1              2              0             10
--------------------------------------------------------------------------------
SUM:                           2485          76173         100249         661564
--------------------------------------------------------------------------------
@magiconair
Copy link
Contributor

@magiconair magiconair commented Feb 6, 2020

Number of executables built from this repo:

❯ ack -l 'package main' | grep -v vendor | wc -l
      41
@magiconair
Copy link
Contributor

@magiconair magiconair commented Feb 6, 2020

It was reproducible this morning. Now I can't repro it. I'll keep an eye on this to see whether I can find a reliable way of triggering this behavior.

@magiconair
Copy link
Contributor

@magiconair magiconair commented Feb 6, 2020

I think have a way of reproducing this:

  1. open the project in VSCode
  2. recompile the project on the command line with the following flags
# build generators
CGO_ENABLED=0 go build -trimpath -mod=vendor -ldflags '-s -w -X github.com/northvolt/our-repo/lib/mapper.version=v1.0.20200203084248-19-ge9962bf9-master' -o build/ ./gen/...

# delete generated code
find driver -name '*_gen.go' -delete

# generate some code
go generate -mod=vendor ./...
Generated /Users/frank/src/github.com/northvolt/our-repo/driver/battery/driver_init_gen.go
Generated /Users/frank/src/github.com/northvolt/our-repo/driver/cajo_printer/driver_init_gen.go
Generated /Users/frank/src/github.com/northvolt/our-repo/driver/cis_presser/driver_init_gen.go
...
Lots of generated code
...

# run tests
CGO_ENABLED=0 go test -trimpath -mod=vendor -ldflags '-s -w -X github.com/northvolt/our-repo/lib/mapper.version=v1.0.20200203084248-19-ge9962bf9-master' -tags=version ./...

# build for all platforms
CGO_ENABLED=0 go build -trimpath -mod=vendor -ldflags '-s -w -X github.com/northvolt/our-repo/lib/mapper.version=v1.0.20200203084248-19-ge9962bf9-master' -o build/darwin_amd64 ./cmd/...
CGO_ENABLED=0 GOOS=linux GOARCH=amd64 go build -trimpath -mod=vendor -ldflags '-s -w -X github.com/northvolt/our-repo/lib/mapper.version=v1.0.20200203084248-19-ge9962bf9-master' -o build/linux_amd64 ./cmd/...
CGO_ENABLED=0 GOOS=linux GOARCH=arm GOARM=7 go build -trimpath -mod=vendor -ldflags '-s -w -X github.com/northvolt/our-repo/lib/mapper.version=v1.0.20200203084248-19-ge9962bf9-master' -o build/linux_arm ./cmd/...
@magiconair
Copy link
Contributor

@magiconair magiconair commented Feb 6, 2020

After that the VSCode/gopls combo keeps the fan spinning for 3-4 minutes

image

@magiconair
Copy link
Contributor

@magiconair magiconair commented Feb 6, 2020

5 min later it dies down. Note that I didn't do anything during that time. This morning I've tried to work which probably didn't help.

image

@elfgoh
Copy link

@elfgoh elfgoh commented May 12, 2020

@magiconair I also work on a mono repo and have had high cpu utilisation by gopls in VS Code. I have found that turning off deep completion (on by default) seems to help. The following is the snippet, to be added to settings.json. YMMV. Informal use of process explorer in VS Code to see the effects on CPU seems to casually indicate that cpu spikes are way less prolonged

    "go.languageServerExperimentalFeatures": {
        "deepCompletion":false
    },

It also worked for a colleague. Maybe you could give this a try and see if it helps?

@magiconair
Copy link
Contributor

@magiconair magiconair commented May 12, 2020

Will do. It is much better with 0.4.0 but a colleague had high cpu load again lately. We will try it.

@elfgoh
Copy link

@elfgoh elfgoh commented May 12, 2020

I am running gopls 0.4.0. If it does indeed work, then it may even make sense to recommend it as a default setting

@elfgoh
Copy link

@elfgoh elfgoh commented May 16, 2020

@stamblerre I have had a colleague who reported improvement from the high cpu utilisation when working on the same monorepo as I do. Would it make sense to disable deepCompletion by default?

@stamblerre
Copy link
Contributor Author

@stamblerre stamblerre commented May 16, 2020

We would need more evidence before we make a change like that, but we can definitely suggest disabling deep completions as a temporary work-around. I just filed #39113 as a reminder to investigate deep completions further.

@elfgoh
Copy link

@elfgoh elfgoh commented May 17, 2020

but we can definitely suggest disabling deep completions as a temporary work-around

Sure. I think that will be a good course of action

@stamblerre
Copy link
Contributor Author

@stamblerre stamblerre commented Jun 24, 2020

We no longer treat vendored packages as workspace packages, so I hope that will resolve the high CPU utilization issue when running go mod vendor. I'm going to close this issue, please reopen or file a new issue if the problem persists.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.