New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: internal compiler error: bvbulkalloc too big #26560

Closed
meilihao opened this Issue Jul 24, 2018 · 5 comments

Comments

Projects
None yet
4 participants
@meilihao

meilihao commented Jul 24, 2018

What version of Go are you using (go version)?

go version go1.11beta2 linux/amd64

Does this issue reproduce with the latest release?

using go version go1.10.3 linux/amd6 gets the same error.

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN=""
GOCACHE="/home/chen/.cache/go-build"
GOEXE=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/chen/git/go"
GOPROXY=""
GORACE=""
GOROOT="/opt/go"
GOTMPDIR=""
GOTOOLDIR="/opt/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build035443180=/tmp/go-build -gno-record-gcc-switches"

What did you do?

go run github.com/meilihao/pinyin/blob/master/example/case.go

What did you expect to see?

go run or go build can work.

What did you see instead?

# github.com/meilihao/pinyin
<autogenerated>:1: internal compiler error: bvbulkalloc too big: nbit=20900 count=877877 nword=654 size=574131558

Please file a bug report including a short program that triggers the error.
https://golang.org/issue/new

@mvdan mvdan added the NeedsFix label Jul 24, 2018

@randall77

This comment has been minimized.

Contributor

randall77 commented Jul 24, 2018

This is also a problem at tip.

It's the really big pinyin_dict.go file. Compiling the initializer for PinyinDict fails. It has a 20,000 line map initializer, and that function has too many stack slots (~20,000) times blocks (125,000). The resulting live bitmap is too big.

There are things we can do that might fix this, like #26552. Unfortunately, they will be 1.12 at best.

But if you want to make progress now, you might try reorganizing your code. Use a slice as the underlying data structure and copy it into a map in an init function. Presumably that code is autogenerated so it might require only a small change to the generator.

@meilihao

This comment has been minimized.

meilihao commented Jul 24, 2018

@randall77 Thanks for your answer, i put it into func init(), and it works for now.

@agnivade

This comment has been minimized.

Member

agnivade commented Jul 24, 2018

@randall77 - Since OP has resolved the issue, do we want to track this separately or mark as dup of #26552 ?

@randall77

This comment has been minimized.

Contributor

randall77 commented Jul 24, 2018

Let's track this separately. #26552 might fix it but I'm not sure.

@randall77 randall77 added this to the Go1.12 milestone Jul 24, 2018

@ALTree ALTree referenced this issue Jul 31, 2018

Closed

bvbulkalloc #26705

@ianlancetaylor ianlancetaylor changed the title from internal compiler error: bvbulkalloc too big to cmd/compile: internal compiler error: bvbulkalloc too big Nov 28, 2018

@randall77

This comment has been minimized.

Contributor

randall77 commented Dec 10, 2018

This seems fixed at tip. CL 140301 helped, but even without that CL tip can compile the OP's code (which is now the commented-out initializer in that github repository, right?). So something else has helped also, not sure what. It does still take ~4GB or so to compile, but the live variable map is not >2^31 any more.

I'm going to close this now. #26552 might help some more, but we're ok for now.

@randall77 randall77 closed this Dec 10, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment