Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: compiler crashes on programs with 1<<30 array key #23780

Closed
dotaheor opened this issue Feb 11, 2018 · 13 comments

Comments

Projects
None yet
9 participants
@dotaheor
Copy link

commented Feb 11, 2018

What version of Go are you using (go version)?

go version go1.9.3 linux/amd64

Does this issue reproduce with the latest release?

yes

What did you do?

a.go:

package main

import (
	"fmt"
)

func main() {
	fmt.Println("Hello, playground")
	const k uint64 = 1 << 30
	_ = []int{k:1}
}}

go build a.go

What did you expect to see?

produce a binary in one second.

What did you see instead?

computer becomes lagging, almost hanging.
I must press CTRL-C for several times to stop the building.

@cznic

This comment has been minimized.

Copy link
Contributor

commented Feb 11, 2018

I can reproduce the issue with go version go1.9.3 linux/amd64 when replacing the final }} with } to make the code syntactically valid.

@dotaheor

This comment has been minimized.

Copy link
Author

commented Feb 11, 2018

The hang may be caused by my computer (8G memory) has not enough memory.
But I think this should only happen at run time, not compile time.

If I change the slice element type to byte, it fails to compile with the following error:

./a.go:10:15: prepwrite: bad off=1073741824 siz=1 s="".statictmp_1
@agnivade

This comment has been minimized.

Copy link
Member

commented Feb 11, 2018

You are trying to create a slice of 1073741824 integers. Which would mean you are trying to get 1073741824 * 8 bytes = 8GB of memory approximately.

@dotaheor

This comment has been minimized.

Copy link
Author

commented Feb 11, 2018

Yes. But I think the hang should occur at run time, instead of compile time.

@ALTree

This comment has been minimized.

Copy link
Member

commented Feb 11, 2018

The second issue you reported (the fact that with byte the compiler just prints

./a.go:10:15: prepwrite: bad off=1073741824 siz=1 s="".statictmp_1

) looks more serious.

I suspect it would do the same thing with int64 on a machine with enough memory.

@ALTree ALTree changed the title cmd/compile: "go build" makes computer lag, almost hang. cmd/compile: compiler crashes on programs with 1<<30 array key Feb 11, 2018

@ALTree

This comment has been minimized.

Copy link
Member

commented Feb 11, 2018

I changed the issue's title to highlight the linker(?) crash you reported in your second post, since I believe it is more interesting that your machine getting slow on the program having the int64 array.

@agnivade

This comment has been minimized.

Copy link
Member

commented Feb 11, 2018

I think it is to do with the way the slice is being initialized. []int{k:1} creates a slice of 0,0,...1. But if you change the code to make([]int, k) everything works fine. Frankly, I have never seen syntax like this - []int{k:1}.

@bradfitz bradfitz added this to the Go1.11 milestone Feb 11, 2018

@cznic

This comment has been minimized.

Copy link
Contributor

commented Feb 12, 2018

FTR: foo := []T{n:1} makes foo with n+1 elements in contrast to foo := make([]t, n).

Note to the compiler guys: I guess the compiler tries to be nice wrt to initialization time and puts the images for composite literals in the text and/or data segments - triggering this issue. If that's the case I think the solution would be to detect when the composite literal is very sparse/mostly zero-values and perform instead a run time all-zero value creation and initialization of the non zero-valued items only all in code.

@odeke-em

This comment has been minimized.

Copy link
Member

commented May 3, 2018

@griesemer

This comment has been minimized.

Copy link
Contributor

commented May 3, 2018

@cznic Yes, the compiler could probably be a bit smarter here - on the other hand, perhaps one shouldn't write code like this. But I agree that the compiler shouldn't just appear to be hanging.

To summarize:

  1. $ go tool compile x.go
    x.go:4:22: prepwrite: bad off=1073741824 siz=1 s="".statictmp_0
    for x.go:
package p

func f() {
	_ = []byte{1 << 30: 0}
}
  1. $ go tool compile x.go
    x.go:4:21: prepwrite: bad off=8589934592 siz=8 s="".statictmp_0
    for x.go (same as above, but int instead of byte):
package p

func f() {
	_ = []int{1 << 30: 0}
}
  1. There's no issue with:
package p

var X = []int{1 << 30: 0}
@griesemer

This comment has been minimized.

Copy link
Contributor

commented Jun 13, 2018

Not urgent. Moving to 1.12.

@griesemer griesemer modified the milestones: Go1.11, Go1.12 Jun 13, 2018

@randall77

This comment has been minimized.

Copy link
Contributor

commented Nov 26, 2018

Related to #27447

@gopherbot

This comment has been minimized.

Copy link

commented Nov 26, 2018

Change https://golang.org/cl/151319 mentions this issue: cmd/compile: initialize sparse slice literals dynamically

@gopherbot gopherbot closed this in 6fff980 Nov 26, 2018

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.