Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: ICE: panic during late deadcode while compiling (*ServeMux).shouldRedirectRLocked #41770

Open
zhsj opened this issue Oct 3, 2020 · 8 comments

Comments

@zhsj
Copy link

@zhsj zhsj commented Oct 3, 2020

What version of Go are you using (go version)?

$ go version
1.15.2

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

go env Output
$ go env

What did you do?

When building packer on Debian armhf machines, we face an ICE.
https://buildd.debian.org/status/fetch.php?pkg=packer&arch=armhf&ver=1.6.4%2Bdfsg-2&stamp=1601746051&raw=0

# net/http
/usr/lib/go-1.15/src/net/http/server.go:2318:14: internal compiler error: '(*ServeMux).shouldRedirectRLocked': panic during late deadcode while compiling (*ServeMux).shouldRedirectRLocked:

runtime error: makeslice: cap out of range

goroutine 39 [running]:
cmd/compile/internal/ssa.Compile.func1(0x342d71c, 0x558e300)
	/usr/lib/go-1.15/src/cmd/compile/internal/ssa/compile.go:48 +0x7c
panic(0x760ce8, 0x89eae0)
	/usr/lib/go-1.15/src/runtime/panic.go:969 +0x118
cmd/compile/internal/ssa.ReachableBlocks(0x558e300, 0xa6b000b8, 0x1, 0x3)
	/usr/lib/go-1.15/src/cmd/compile/internal/ssa/deadcode.go:23 +0x38
cmd/compile/internal/ssa.deadcode(0x558e300)
	/usr/lib/go-1.15/src/cmd/compile/internal/ssa/deadcode.go:181 +0x40
cmd/compile/internal/ssa.Compile(0x558e300)
	/usr/lib/go-1.15/src/cmd/compile/internal/ssa/compile.go:93 +0x98c
cmd/compile/internal/gc.buildssa(0x1f395f0, 0x3, 0x0)
	/usr/lib/go-1.15/src/cmd/compile/internal/gc/ssa.go:460 +0xbf4
cmd/compile/internal/gc.compileSSA(0x1f395f0, 0x3)
	/usr/lib/go-1.15/src/cmd/compile/internal/gc/pgen.go:317 +0x3c
cmd/compile/internal/gc.compileFunctions.func2(0x30171c0, 0x1ae2e50, 0x3)
	/usr/lib/go-1.15/src/cmd/compile/internal/gc/pgen.go:382 +0x38
created by cmd/compile/internal/gc.compileFunctions
	/usr/lib/go-1.15/src/cmd/compile/internal/gc/pgen.go:380 +0x100

It uses external link, other settings are standard.

GOFLAGS := -ldflags=-linkmode=external

We choose to use external link, because previously(when we still use go1.14) we meet link error, but it's another issue.
https://buildd.debian.org/status/fetch.php?pkg=packer&arch=armhf&ver=1.6.0%2Bdfsg-2&stamp=1594662885&raw=0

# armel/armhf fail to link:
# os/user(.text): direct call too far: .plt 87db4e

It's hard for me to reproduce locally and to write down a detailed issue, since it happens on an armhf machine, . But since it's an ICE, so I report here anyway, maybe you will know something from the trace.

What did you expect to see?

What did you see instead?

@randall77
Copy link
Contributor

@randall77 randall77 commented Oct 3, 2020

cap out of range is a very strange error to get at that point.

	reachable := make([]bool, f.NumBlocks())

Sounds like some sort of corruption happened. I don't see any way f.NumBlocks() could return strange (negative, or very large) values.

Even stranger, we got an error on cap instead of len. makeslice is explicitly designed to report len in preference to cap. For the 2-arg make, we have len==cap, and in that scenario there's no path through makeslice that can generate a cap error.

How repeatable is this error? Can you get it to happen on two different machines? It would be nice to eliminate the machine itself.

@odeke-em odeke-em changed the title ICE: panic during late deadcode while compiling (*ServeMux).shouldRedirectRLocked runtime: ICE: panic during late deadcode while compiling (*ServeMux).shouldRedirectRLocked Oct 4, 2020
@zhsj
Copy link
Author

@zhsj zhsj commented Oct 4, 2020

How repeatable is this error? Can you get it to happen on two different machines? It would be nice to eliminate the machine itself.

I don't have access to these builders. When I trigger a rebuild, it goes to another machine, which is an arm64 hardware.

The log is https://buildd.debian.org/status/logs.php?pkg=packer&ver=1.6.4%2Bdfsg-2&arch=armhf
The failed machine is https://db.debian.org/machines.cgi?host=henze

I will try to see if I can get access to similar hardware.

@cherrymui cherrymui changed the title runtime: ICE: panic during late deadcode while compiling (*ServeMux).shouldRedirectRLocked cmd/compile: ICE: panic during late deadcode while compiling (*ServeMux).shouldRedirectRLocked Oct 4, 2020
@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Oct 4, 2020

@dr2chase is it related to #41780 ? This is also on ARM. Maybe the compiler itself is miscompiled?

@zhsj
Copy link
Author

@zhsj zhsj commented Oct 4, 2020

I find a machine which I can access, which has same CPU(Marvell Armada 370/XP CPU @ 1.6GHz on a Marvell MV78460 SoC Development Board (ARM v7)) https://db.debian.org/machines.cgi?host=abel
But I can't reproduce the ICE.

@dr2chase
Copy link
Contributor

@dr2chase dr2chase commented Oct 5, 2020

Can't reproduce, make fails with:

common/bootcommand/config.go:1: running "struct-markdown": exec: "struct-markdown": executable file not found in $PATH

Where does struct-markdown come from? I'm trying to see if the fix for #41780 ( https://go-review.googlesource.com/c/go/+/259450/ ) affects this. I'm on an Odroid (arm64) with GOARCH=arm in my environment and running an arm-compiled compiler.

Answer: be sure $GOPATH/bin is on your $PATH.

@dr2chase
Copy link
Contributor

@dr2chase dr2chase commented Oct 5, 2020

FYI, my current repro attempt for compilation on arm64 ODroid Linux:

git diff Makefile
diff --git a/Makefile b/Makefile
index 41bfd38c9..3f358d6cb 100644
--- a/Makefile
+++ b/Makefile
@@ -1,3 +1,5 @@
+GOFLAGS := -ldflags=-linkmode=external
+
 TEST?=$(shell go list ./...)
 VET?=$(shell go list ./...)
 ACC_TEST_BUILDERS?=all

Cross compilation for cgo:

sudo apt install gcc-arm-linux-gnueabihf libc6-armhf-armel-cross 

Environment variables for the make command:

PATH=$HOME/work/go-linux-arm-bootstrap/bin:$GOPATH/bin:$PATH \
GOARCH=arm \
LD_LIBRARY_PATH=/usr/arm-linux-gnueabihf/lib \
CC=arm-linux-gnueabihf-gcc \
make

Unfortunately it still fails with the linking error mentioned above, despite making the recommended change to Makefile:

...
go run ./cmd/generate-fixer-deprecations
# github.com/hashicorp/packer
os/user(.text): direct call too far: .plt a6fb8c
os/user(.text): direct call too far: .plt a6fb84
os/user(.text): direct call too far: .plt a6fb83
...

Perhaps someone more familiar with the Makefile can help me along to the next roadblock.

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Oct 5, 2020

@dr2chase for direct call too far, try -ldflags=-linkmode=external.

@dr2chase
Copy link
Contributor

@dr2chase dr2chase commented Oct 6, 2020

Successful build (i.e., sadly I failed to reproduce the failure) with the following diff to the Makefile, which may not all be necessary:

git diff Makefile
diff --git a/Makefile b/Makefile
index 41bfd38c9..e94bcc84b 100644
--- a/Makefile
+++ b/Makefile
@@ -1,3 +1,5 @@
+GOFLAGS := -ldflags=-linkmode=external
+
 TEST?=$(shell go list ./...)
 VET?=$(shell go list ./...)
 ACC_TEST_BUILDERS?=all
@@ -17,7 +19,7 @@ GIT_DIRTY=$(shell test -n "`git status --porcelain`" && echo "+CHANGES" || true)
 GIT_COMMIT=$(shell git rev-parse --short HEAD)
 GIT_IMPORT=github.com/hashicorp/packer/version
 UNAME_S := $(shell uname -s)
-LDFLAGS=-s -w
+LDFLAGS=-linkmode=external
 GOLDFLAGS=-X $(GIT_IMPORT).GitCommit=$(GIT_COMMIT)$(GIT_DIRTY) $(LDFLAGS)
 
 export GOLDFLAGS

I notice that the failure actually occurs compiling part of net/http in go-1.15, so this ought to be happening rather more often.
Working from the assumption that this might be a compiler-compiled-on-Arm problem (because the bootstrap was compiled on another platform), I tried to provoke that:

LD_LIBRARY_PATH=/usr/arm-linux-gnueabihf/lib CC=arm-linux-gnueabihf-gcc  GOSSAHASH=n go install -a cmd/compile

(setting GOSSAHASH forces the compiler to run again) and then

LD_LIBRARY_PATH=/usr/arm-linux-gnueabihf/lib CC=arm-linux-gnueabihf-gcc go test net/http

but no failure. I retried the packer bug with the self-compiled compiler, no luck (i.e., it built).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.