Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: large variable not initialized to 0 #42054

Closed
zephyrtronium opened this issue Oct 18, 2020 · 8 comments
Closed

cmd/compile: large variable not initialized to 0 #42054

zephyrtronium opened this issue Oct 18, 2020 · 8 comments

Comments

@zephyrtronium
Copy link
Contributor

@zephyrtronium zephyrtronium commented Oct 18, 2020

What version of Go are you using (go version)?

Initially spotted on playground (1.14.9), but also reproduces with 1.15.2 and:

$ go version
go version devel +3036b76df0 Sat Oct 10 16:06:07 2020 +0000 windows/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
PS E:\go> go env
set GO111MODULE=on
set GOARCH=amd64
set GOBIN=
set GOCACHE=C:\Users\zephyr\AppData\Local\go-build
set GOENV=C:\Users\zephyr\AppData\Roaming\go\env
set GOEXE=.exe
set GOFLAGS=
set GOHOSTARCH=amd64
set GOHOSTOS=windows
set GOINSECURE=
set GOMODCACHE=E:\go\pkg\mod
set GONOPROXY=
set GONOSUMDB=
set GOOS=windows
set GOPATH=E:\go
set GOPRIVATE=
set GOPROXY=https://proxy.golang.org,direct
set GOROOT=E:\gotip
set GOSUMDB=sum.golang.org
set GOTMPDIR=
set GOTOOLDIR=E:\gotip\pkg\tool\windows_amd64
set GCCGO=gccgo
set AR=ar
set CC=gcc
set CXX=g++
set CGO_ENABLED=1
set GOMOD=NUL
set CGO_CFLAGS=-g -O2
set CGO_CPPFLAGS=
set CGO_CXXFLAGS=-g -O2
set CGO_FFLAGS=-g -O2
set CGO_LDFLAGS=-g -O2
set PKG_CONFIG=pkg-config
set GOGCCFLAGS=-m64 -mthreads -fmessage-length=0 -fdebug-prefix-map=C:\Users\zephyr\AppData\Local\Temp\go-build322924098=/tmp/go-build -gno-record-gcc-switches

What did you do?

Create a large array and print the first few elements of it:

https://play.golang.org/p/66eTJdT8Ctu

What did you expect to see?

Either a slice of 0s, or a compilation error. (I was originally trying to demonstrate an error that can be reported by dowidth. An error does occur when the size of the array is greater than 2e9 but less than 1<<32, but it originates elsewhere.)

What did you see instead?

[1 1 0 0 0 0 0 0 0 0 0 1 0 0 0 1 1 1 0 1], with other apparently random combinations of 1s and 0s for other sizes. Testing locally, I also see different combinations on the same size between my (slightly dated) copy of tip and 1.15.2.

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Oct 19, 2020

Loading

@mdempsky
Copy link
Member

@mdempsky mdempsky commented Oct 19, 2020

I think this is actually a linker issue. Looking at go tool compile -S output, it looks like we're correctly emitting code to generate a 4GiB variable. It seems like the linker is truncating the size to 32-bits somewhere. I thought it was here:

o.SetSiz(uint32(s.Size))

but that code was newly merged from the dev.link branch, and the issue reproduces identically under Go 1.15 (including the compiler's -S output looking right).

/cc @cherrymui @thanm

Loading

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Oct 19, 2020

but that code was newly merged from the dev.link branch, and the issue reproduces identically under Go 1.15 (including the compiler's -S output looking right).

Go 1.15 has the same code as well (in a different place).

Go 1.14 rejects it.

main.x: symbol too large (4294967296 bytes)
too much data in section SNOPTRBSS (over 2e+09 bytes)

We never supported symbol larger than 2GB (issue #9862), so the object file uses 32-bit for symbol size. I'll add a check before truncation.

Loading

@ALTree ALTree changed the title cmd/compile: Large variable not initialized to 0 cmd/compile: large variable not initialized to 0 Oct 19, 2020
@go101
Copy link

@go101 go101 commented Oct 19, 2020

It looks that gc 1.15.3 rejects var x [1<<31]byte but accepts var x [1<<32]byte.

[edit]: it looks 31 is the only rejected number. And it is ok when x is declared locally.

Loading

@gopherbot
Copy link

@gopherbot gopherbot commented Oct 19, 2020

Change https://golang.org/cl/263641 mentions this issue: cmd/internal/obj: reject too large symbols

Loading

@go101
Copy link

@go101 go101 commented Oct 19, 2020

What is the difficulty to declare very large sized arrays?
If var x = make([]byte, 1<<32) can be allocated without problem, then why to limit the length of package-level arrays?

Loading

@gopherbot gopherbot closed this in c9c6488 Oct 20, 2020
@go101
Copy link

@go101 go101 commented Oct 20, 2020

It looks the fix makes the old code "var x [1<<31]byte" fail to compile now. An incompatibility?

Loading

@go101
Copy link

@go101 go101 commented Oct 20, 2020

Sorry, I forgot "var x [1<<31]byte" doesn't compile before. ;)

Loading

@golang golang locked and limited conversation to collaborators Oct 20, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants