Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/gopls: using a lot of memory #37223

Closed
ajeecai opened this issue Feb 14, 2020 · 11 comments
Closed

x/tools/gopls: using a lot of memory #37223

ajeecai opened this issue Feb 14, 2020 · 11 comments

Comments

@ajeecai
Copy link

@ajeecai ajeecai commented Feb 14, 2020

What version of Go are you using (go version)?

$ go version
go version go1.13.4 linux/amd64

Does this issue reproduce with the latest release?

Yes, I have update gopls with GO111MODULE=on go get golang.org/x/tools/gopls@master golang.org/x/tools@master

$ gopls version
golang.org/x/tools/gopls master
golang.org/x/tools/gopls@v0.1.8-0.20200213144451-3187b3c41574 h1:cDdkE3V1eIXwN6tdzqVXVYz3RkYVbE8y0n+sn6klUn4=

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="auto"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ajee/.cache/go-build"
GOENV="/home/ajee/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/ajee/go"
GOPRIVATE=""
GOPROXY="https://goproxy.io"
GOROOT="/usr/lib/go-1.13"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.13/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build734902289=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Use vs code 1.42.0, download https://github.com/asticode/go-astilectron-demo and browse its code, after a bit change then save to make gopls activated.

What did you expect to see?

From htop, this memory used by gopls should not be excessive.

What did you see instead?

The memory usage is soar, up to 9G, sometimes 12G by gopls, and it never goes down.
image

The similar issue is discussed in #30309, @stamblerre encourages to open a new issue.

The pprofile svg file is attached as required (in zip file as github required). Any information needed, pls tell me.

Thanks

pprof001.zip

@gopherbot gopherbot added the gopls label Feb 14, 2020
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Feb 14, 2020

Thank you for filing a gopls issue! Please take a look at the Troubleshooting guide, and make sure that you have provided all of the relevant information here.

@stamblerre stamblerre changed the title gopls: Use a log of memory x/tools/gopls: using a lot of memory Feb 14, 2020
@gopherbot gopherbot added this to the Unreleased milestone Feb 14, 2020
@gopherbot gopherbot added the Tools label Feb 14, 2020
@stamblerre

This comment has been minimized.

Copy link
Contributor

@stamblerre stamblerre commented Feb 14, 2020

It seems like analysis is causing the high memory usage. Can you try disabling it by adding the following to your VS Code settings?

"gopls": {
	"experimentalDisabledAnalyses": [
		"asmdecl", "assign", "atomic", "atomicalign", 
		"bools", "buildtag", "cgocall", "composite", "copylock",      
		"errorsas", "httpresponse", "loopclosure", "lostcancel",
		"nilfunc", "printf", "shift", "stdmethods", 
		"structtag", "tests", "unmarshal", "unreachable", "unsafeptr",
		"unusedresult", "deepequalerrors", 
		"sortslice", "testinggoroutine"
	],
},

We also just released gopls/v0.3.2 - can you try downloading that to see if affects anything? (GO111MODULE=on go get golang.org/x/tools/gopls@v0.3.2).

@stamblerre stamblerre modified the milestones: Unreleased, gopls/v0.4.0 Feb 14, 2020
@heschik

This comment has been minimized.

Copy link
Contributor

@heschik heschik commented Feb 14, 2020

Can you check your /tmp directory for files named gopls.*? It should have written out some profiles. They're named after the size of the heap; please upload the ones from the biggest heaps.

@ajeecai

This comment has been minimized.

Copy link
Author

@ajeecai ajeecai commented Feb 15, 2020

Hi, I just tried, adding the options in vs code setting doesn't help. And I also update to 0.3.2, looks the same, memory up to 9G. Please see the attached zip file which includes svg for memory profile. Also I didn't find any files name gopls.* under /tmp, instead in my home directory there is a folder named pprof which contains some files with gopls, then I include one is generated at the same timestamp in the zip. Anything else please tell me.

Thanks
pp.zip

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Feb 18, 2020

Change https://golang.org/cl/219897 mentions this issue: internal/lsp/cache: debugging CL for golang/go#37223

@stamblerre

This comment has been minimized.

Copy link
Contributor

@stamblerre stamblerre commented Feb 18, 2020

Thanks for the profiles. Looks like the inspect analysis is still using the majority of the memory. I've disabled in analyses in CL 219897 - can you try rebuilding gopls with that CL patched in and see if that resolves the problem?

This can be done with the following steps:

$ cd $(mktemp -d)
$ git clone https://go.googlesource.com/tools
$ cd tools
$ git fetch "https://go.googlesource.com/tools" refs/changes/97/219897/2 && git cherry-pick FETCH_HEAD
$ cd gopls
$ go install

This supports @heschik's theory that analyses aren't being canceled in time, and that we are running too many analyses concurrently.

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Feb 18, 2020

Change https://golang.org/cl/219958 mentions this issue: internal/lsp: limit diagnostics concurrency

gopherbot pushed a commit to golang/tools that referenced this issue Feb 19, 2020
Diagnostics runs cannot be canceled until they finish a package. If a
user has a very expensive package, we may stack up diagnostics runs to
the point where the machine runs out of memory. We see hints of this in
various issues.

To avoid that, only allow one diagnostics run at a time. We can change
the limit later if we want.

Updates golang/go#37223.

Change-Id: Icd0eec4da936153306cf0a1f7175ae2b4b265272
Reviewed-on: https://go-review.googlesource.com/c/tools/+/219958
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
@ajeecai

This comment has been minimized.

Copy link
Author

@ajeecai ajeecai commented Feb 21, 2020

Hi, good news with CL 219897, now the memory stays low even I remove experimentalDisabledAnalyses from the vs code config.

@heschik

This comment has been minimized.

Copy link
Contributor

@heschik heschik commented Feb 21, 2020

Cool, that confirms that it was a problem with analyses. Can you see if https://golang.org/cl/219958 (which is on master now) also helps?

@ajeecai

This comment has been minimized.

Copy link
Author

@ajeecai ajeecai commented Feb 23, 2020

Hi @heschik , I just git clone https://go.googlesource.com/tools and go install from head, it seems working, no memory soar.

@stamblerre

This comment has been minimized.

Copy link
Contributor

@stamblerre stamblerre commented Feb 24, 2020

Thanks for confirming! I'll mark this as closed, and https://golang.org/cl/219958 will be part of the gopls/v0.4.0 release.

@stamblerre stamblerre closed this Feb 24, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
4 participants
You can’t perform that action at this time.