Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/tools/gopls: using a lot of memory #37223

Closed
ajeecai opened this issue Feb 14, 2020 · 26 comments
Closed

x/tools/gopls: using a lot of memory #37223

ajeecai opened this issue Feb 14, 2020 · 26 comments

Comments

@ajeecai
Copy link

@ajeecai ajeecai commented Feb 14, 2020

What version of Go are you using (go version)?

$ go version
go version go1.13.4 linux/amd64

Does this issue reproduce with the latest release?

Yes, I have update gopls with GO111MODULE=on go get golang.org/x/tools/gopls@master golang.org/x/tools@master

$ gopls version
golang.org/x/tools/gopls master
golang.org/x/tools/gopls@v0.1.8-0.20200213144451-3187b3c41574 h1:cDdkE3V1eIXwN6tdzqVXVYz3RkYVbE8y0n+sn6klUn4=

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="auto"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/ajee/.cache/go-build"
GOENV="/home/ajee/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/ajee/go"
GOPRIVATE=""
GOPROXY="https://goproxy.io"
GOROOT="/usr/lib/go-1.13"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go-1.13/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build734902289=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Use vs code 1.42.0, download https://github.com/asticode/go-astilectron-demo and browse its code, after a bit change then save to make gopls activated.

What did you expect to see?

From htop, this memory used by gopls should not be excessive.

What did you see instead?

The memory usage is soar, up to 9G, sometimes 12G by gopls, and it never goes down.
image

The similar issue is discussed in #30309, @stamblerre encourages to open a new issue.

The pprofile svg file is attached as required (in zip file as github required). Any information needed, pls tell me.

Thanks

pprof001.zip

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 14, 2020

Thank you for filing a gopls issue! Please take a look at the Troubleshooting guide, and make sure that you have provided all of the relevant information here.

@stamblerre stamblerre changed the title gopls: Use a log of memory x/tools/gopls: using a lot of memory Feb 14, 2020
@gopherbot gopherbot added this to the Unreleased milestone Feb 14, 2020
@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Feb 14, 2020

It seems like analysis is causing the high memory usage. Can you try disabling it by adding the following to your VS Code settings?

"gopls": {
	"experimentalDisabledAnalyses": [
		"asmdecl", "assign", "atomic", "atomicalign", 
		"bools", "buildtag", "cgocall", "composite", "copylock",      
		"errorsas", "httpresponse", "loopclosure", "lostcancel",
		"nilfunc", "printf", "shift", "stdmethods", 
		"structtag", "tests", "unmarshal", "unreachable", "unsafeptr",
		"unusedresult", "deepequalerrors", 
		"sortslice", "testinggoroutine"
	],
},

We also just released gopls/v0.3.2 - can you try downloading that to see if affects anything? (GO111MODULE=on go get golang.org/x/tools/gopls@v0.3.2).

@stamblerre stamblerre modified the milestones: Unreleased, gopls/v0.4.0 Feb 14, 2020
@heschi
Copy link
Contributor

@heschi heschi commented Feb 14, 2020

Can you check your /tmp directory for files named gopls.*? It should have written out some profiles. They're named after the size of the heap; please upload the ones from the biggest heaps.

@ajeecai
Copy link
Author

@ajeecai ajeecai commented Feb 15, 2020

Hi, I just tried, adding the options in vs code setting doesn't help. And I also update to 0.3.2, looks the same, memory up to 9G. Please see the attached zip file which includes svg for memory profile. Also I didn't find any files name gopls.* under /tmp, instead in my home directory there is a folder named pprof which contains some files with gopls, then I include one is generated at the same timestamp in the zip. Anything else please tell me.

Thanks
pp.zip

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 18, 2020

Change https://golang.org/cl/219897 mentions this issue: internal/lsp/cache: debugging CL for golang/go#37223

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Feb 18, 2020

Thanks for the profiles. Looks like the inspect analysis is still using the majority of the memory. I've disabled in analyses in CL 219897 - can you try rebuilding gopls with that CL patched in and see if that resolves the problem?

This can be done with the following steps:

$ cd $(mktemp -d)
$ git clone https://go.googlesource.com/tools
$ cd tools
$ git fetch "https://go.googlesource.com/tools" refs/changes/97/219897/2 && git cherry-pick FETCH_HEAD
$ cd gopls
$ go install

This supports @heschik's theory that analyses aren't being canceled in time, and that we are running too many analyses concurrently.

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 18, 2020

Change https://golang.org/cl/219958 mentions this issue: internal/lsp: limit diagnostics concurrency

gopherbot pushed a commit to golang/tools that referenced this issue Feb 19, 2020
Diagnostics runs cannot be canceled until they finish a package. If a
user has a very expensive package, we may stack up diagnostics runs to
the point where the machine runs out of memory. We see hints of this in
various issues.

To avoid that, only allow one diagnostics run at a time. We can change
the limit later if we want.

Updates golang/go#37223.

Change-Id: Icd0eec4da936153306cf0a1f7175ae2b4b265272
Reviewed-on: https://go-review.googlesource.com/c/tools/+/219958
Reviewed-by: Rebecca Stambler <rstambler@golang.org>
@ajeecai
Copy link
Author

@ajeecai ajeecai commented Feb 21, 2020

Hi, good news with CL 219897, now the memory stays low even I remove experimentalDisabledAnalyses from the vs code config.

@heschi
Copy link
Contributor

@heschi heschi commented Feb 21, 2020

Cool, that confirms that it was a problem with analyses. Can you see if https://golang.org/cl/219958 (which is on master now) also helps?

@ajeecai
Copy link
Author

@ajeecai ajeecai commented Feb 23, 2020

Hi @heschik , I just git clone https://go.googlesource.com/tools and go install from head, it seems working, no memory soar.

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Feb 24, 2020

Thanks for confirming! I'll mark this as closed, and https://golang.org/cl/219958 will be part of the gopls/v0.4.0 release.

@nikkolasg
Copy link

@nikkolasg nikkolasg commented Apr 3, 2020

Just want to report my case here since related: Using vim-go, that uses gopls underneath, I see my memory usage just growing, need to restart vim after a couple of hours each time. With latest updated binaries using vim-go :GoUpdateBinaries.

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Apr 3, 2020

@nikkolasg: Just to confirm, the output of gopls version is v0.3.4?

@nikkolasg
Copy link

@nikkolasg nikkolasg commented Apr 3, 2020

golang.org/x/tools/gopls v0.3.4
    golang.org/x/tools/gopls@v0.3.4 h1:4GC7q/pXQ/tsxHBGVdsMdlB4gCxVC06m/7rIXg1Px4E=

Yes! Happy to provide more info if I can help.

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Apr 3, 2020

Thanks! Yes, hopefully we can debug this. There should be some files named gopls.PID- in your temporary directory, named after heap sizes. Please upload the pair for the largest heap -- I would like to see a goroutine dump that matches this heap dump.

@nikkolasg
Copy link

@nikkolasg nikkolasg commented Apr 3, 2020

Any idea where I can find this file ? Only thing relevant I find in /tmp is gopls_stderrsq5q_qoo.log but it's empty.

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Apr 3, 2020

Maybe try looking in whatever the value of TMPDIR is on your machine? (This might be helpful.)

@nikkolasg
Copy link

@nikkolasg nikkolasg commented Apr 3, 2020

my tmpdir is empty , so I'm using the same tmpdir as what go returns I guess ?
find . -name gopls.PID didn't return anything in /tmp ... Maybe vim-go changes that but I don't see it.

@stamblerre
Copy link
Contributor

@stamblerre stamblerre commented Apr 4, 2020

Ah, the PID means it will be a process ID, so you may want to run something like find . -name "gopls.*". If nothing turns up still, then maybe @heschik can advise.

@nikkolasg
Copy link

@nikkolasg nikkolasg commented Apr 4, 2020

nothing turns out there as well :/
/tmp:% find . -name "gopls.*"

@EnriqueL8
Copy link

@EnriqueL8 EnriqueL8 commented Apr 6, 2020

I'm hitting the same problem as well. Same version and nothing found with the find . -name "gopls.*".

@heschi
Copy link
Contributor

@heschi heschi commented Apr 6, 2020

It would be very, very surprising if recent versions of gopls hadn't written any of those files to /tmp while using a problematic amount of memory. Can you triple-check that there isn't an old version anywhere, and that it's actually gopls using the memory?

@nikkolasg
Copy link

@nikkolasg nikkolasg commented Apr 6, 2020

I ran the search on / ;) The only file gopls creates is of the form /tmp/gopls_stderr9hteqnwy.log and it's empty.
And yes, I see a bunch of gopls threads/processes always on top with htop when sorting by memory. And I gain a bunch of free GB by calling pkill gopls.

The only thing I am not sure of is which version vim-go uses, even when doing :GoUpdateBinaries but the one I have when type gopls version is v0.3.4. I never go get it by hand so I think vim-go did it for me.

@heschi
Copy link
Contributor

@heschi heschi commented Apr 6, 2020

Sorry, I don't know how vim-go works at all, but by far the most likely explanation for what you're describing is still that you're using an old version of gopls. If you're on Linux, you can see where the binary comes from using ls -l /proc/$PID/exe.

@nikkolasg
Copy link

@nikkolasg nikkolasg commented Apr 6, 2020

Nice ! In fact it was YouCompleteMe that used an old version, no vim-go, I found it by running your command. Updated it manually and so far it works good !
Thanks a lot !

@ericwq
Copy link

@ericwq ericwq commented Aug 25, 2020

i might find the correct setup method for gopls. see the post

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
7 participants