Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: compiler hangs without optimizations #29144

csnitker opened this issue Dec 7, 2018 · 9 comments

cmd/compile: compiler hangs without optimizations #29144

csnitker opened this issue Dec 7, 2018 · 9 comments


Copy link

@csnitker csnitker commented Dec 7, 2018

What version of Go are you using (go version)?

$ go version
go version go1.11.2 windows/amd64

Does this issue reproduce with the latest release?

Yes, reproduced using go 1.11, 1.11.2 and 1.10

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\csnitker\AppData\Local\go-build
set GOEXE=.exe
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOOS=windows
set GOPATH=C:\Users\csnitker\go
set GOROOT=C:\Go
set GOTOOLDIR=C:\Go\pkg\tool\windows_amd64
set GCCGO=gccgo
set CC=gcc
set CXX=g++
set GOMOD=C:\Users\csnitker\go\src\\RegistrarServices\whois\go.mod
set CGO_CFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\csnitker\AppData\Local\Temp\go-build373743682=/tmp/go-build -gno-record-gcc-switches

What did you do?

Run go build -gcflags=all="-N -l" -x main.go or dlv debug main.go on the following:

package main

import ""

func main() {
    zone := zonedb.PublicZone("")
    if zone != nil {

What did you expect to see?

A successful build.

What did you see instead?

The compiler hangs and consumes large amounts of memory.


The command from the process above shows:

C:\go1.11\pkg\tool\windows_amd64\compile.exe -o C:\Users\csnitker\AppData\Local\Temp\go-build516058638\b002\_pkg_.a -trimpath C:\Users\csnitker\AppData\Local\Temp\go-build516058638\b002 -N -l -p -complete -buildid mw6Bk1kcUZg16C0mq6E3/mw6Bk1kcUZg16C0mq6E3 -goversion go1.11 -D "" -importcfg C:\Users\csnitker\AppData\Local\Temp\go-build516058638\b002\importcfg -pack -c=4 C:\Users\csnitker\go\pkg\mod\\zonedb\zonedb@v0.0.0-20181207082023-cee273e3e759\zone.go C:\Users\csnitker\go\pkg\mod\\zonedb\zonedb@v0.0.0-20181207082023-cee273e3e759\zones.go

It's also interesting that go build -gcflags="-N -l" main.go completes successfully and works using dlv exec main.exe but does not with the commands above. I'm not entirely sure if this is a go problem or an issue with the library (or delve) but figured I would start here since an optimized build completes without issue.

@ianlancetaylor ianlancetaylor changed the title compile hangs without optimizations cmd/compile: compiler hangs without optimizations Dec 7, 2018
Copy link

@ianlancetaylor ianlancetaylor commented Dec 7, 2018

The file has a fairly large composite literal. I suspect that the compiler isn't hanging as such, it's just taking a lot of memory, perhaps causing your system to thrash.

CC @randall77

Copy link

@ianlancetaylor ianlancetaylor commented Dec 7, 2018

Although actually it doesn't take very long to build that package on my laptop.

Copy link

@randall77 randall77 commented Dec 7, 2018

It's the difference between -gcflags=all="-N -l" and -gcflags="-N -l". The former applies the optimizations-off flags to all packages (including the zonedb package) and the latter to only the top-level package and not its imports. The latter will use an optimized version of the zonedb package which compiles somewhat quickly.

It's the -N, removing -l doesn't change anything.

Copy link

@randall77 randall77 commented Dec 7, 2018

Looks like it is spending most of its time (and space) in live variable analysis in the register allocator.
I'm not sure there's a whole lot we can do about that, unfortunately.
Maybe if there was a key phase that, if enabled even with -N, would make the function easier for regalloc to handle.

Copy link

@ianlancetaylor ianlancetaylor commented Dec 7, 2018

That's impressive. With -gcflags=all=-N I let the compiler run for ten minutes before killing it, at which point the virtual memory size was over 15G.

Why do we need live variable analysis with -N? In a big function can we just dump everything on the stack?

Copy link

@randall77 randall77 commented Dec 7, 2018

We could implement a second, simpler register allocator for -N, sure. I was hoping to avoid that.
I'm not sure how much we could share between the two allocators. The live variable analysis is pretty integral to the current design, I don't think we could just say "everything is live" and have it work.

@andybons andybons removed this from the Go1.12 milestone Feb 12, 2019
@andybons andybons added this to the Go1.13 milestone Feb 12, 2019
@andybons andybons removed this from the Go1.13 milestone Jul 8, 2019
@andybons andybons added this to the Go1.14 milestone Jul 8, 2019
@rsc rsc removed this from the Go1.14 milestone Oct 9, 2019
@rsc rsc added this to the Backlog milestone Oct 9, 2019
Copy link

@icamys icamys commented Nov 29, 2019

Hi all! The bug also can be reproduced with go version go1.13 linux/amd64

Copy link

@lzaldivarkt lzaldivarkt commented Dec 28, 2019

Another confirmation. This gets triggered when running go code in intellij on debug mode, if the zonedb package is added to the code, as Intellij adds those gcflags by default.

Version: go version go1.13.3 linux/amd64

Copy link

@josharian josharian commented Dec 29, 2019

If someone wants to dig into this, it might be useful to know whether there is one key optimization omitted by -N that makes the difference. If so, perhaps (say) there might be a way to do a subset of that optimization even with -N to keep sizes reasonable.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
8 participants