x/tools/gopls: high memory use and sluggishness

[bhcleek]

What version of Go are you using (go version)?

$ go version
go version go1.16.3 darwin/amd64

Does this issue reproduce with the latest release?

Yes

golang.org/x/tools/gopls v0.6.9
    golang.org/x/tools/gopls@v0.6.9 h1:LBBcE2y3Tb4bp79JVLWCQBbvYCFEs5ADGWsQDoSLk1Q=

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE="on"
GOARCH="amd64"
GOBIN="/Users/bhcleek/go/bin"
GOCACHE="/Users/bhcleek/Library/Caches/go-build"
GOENV="/Users/bhcleek/Library/Application Support/go/env"
GOEXE=""
GOFLAGS="-mod=vendor"
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/bhcleek/go/pkg/mod"
GONOPROXY="$INTERNAL_REPOS"
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/bhcleek/go"
GOPRIVATE="$INTERNAL_REPOS"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.16.3"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/bhcleek/src/$MODULE/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/f9/d6g779t53gq_20w0zm2f_1zm0000gn/T/go-build3331311271=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Start up gopls in our monorepo in module aware mode by opening vscode or Vim w/ vim-go

What did you expect to see?

Similar memory use and performance as when using GOPATH mode.

What did you see instead?

Very high memory use and sluggishness. Using module aware mode, gopls very quickly starts using 10GB of memory or more. Other users internally report even higher memory use.

Performance is generally very sluggish. Completion requests, for example, have a noticable delay before a response is delivered.

In GOPATH mode, memory sits around 300MB and gopls is very responsive.

[heschi]

I'm working with @bhcleek on this privately.