Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile: internal compiler error: devirtualization failed (x/text, x/build, protobuf) #42279

Closed
zikaeroh opened this issue Oct 29, 2020 · 9 comments

Comments

@zikaeroh
Copy link
Contributor

@zikaeroh zikaeroh commented Oct 29, 2020

What version of Go are you using (go version)?

$ go version
go version devel +5cc43c51c9 Thu Oct 29 19:06:32 2020 +0000 linux/amd64

Does this issue reproduce with the latest release?

On tip, yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jake/.cache/go-build"
GOENV="/home/jake/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/jake/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/jake/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/jake/sdk/gotip"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/jake/sdk/gotip/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/jake/zikaeroh/hortbot/hortbot2/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build236306354=/tmp/go-build -gno-record-gcc-switches"
GOROOT/bin/go version: go version devel +5cc43c51c9 Thu Oct 29 19:06:32 2020 +0000 linux/amd64
GOROOT/bin/go tool compile -V: compile version devel +5cc43c51c9 Thu Oct 29 19:06:32 2020 +0000
uname -sr: Linux 5.9.1-zen2-1-zen
/usr/lib/libc.so.6: GNU C Library (GNU libc) release release version 2.32.
gdb --version: GNU gdb (GDB) 9.2

What did you do?

Ran gotip test in my project to check for performance changes from the new inlining/devirtualization changes.

I don't think I need to provide the exact code; the error message shows which files in my module cache are failing to be built.

I'm guessing a build dashboard will reveal this, I'm just probably too quick to test changes I'm excited to see...

What did you expect to see?

A build success.

What did you see instead?

An ICE:

# golang.org/x/text/encoding
../../../go/pkg/mod/golang.org/x/text@v0.3.3/encoding/encoding.go:293:5: internal compiler error: devirtualization failed: transform.t.(*Decoder).Transformer.Reset

goroutine 1 [running]:
runtime/debug.Stack(0xdaf700, 0xc00012a008, 0x0)
	/home/jake/sdk/gotip/src/runtime/debug/stack.go:24 +0x9f
cmd/compile/internal/gc.Fatalf(0xca427c, 0x1b, 0xc0004c95c0, 0x1, 0x1)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/subr.go:199 +0x1b0
cmd/compile/internal/gc.devirtualizeCall(0xc0004b8080)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/inl.go:1454 +0x2ab
cmd/compile/internal/gc.devirtualize.func1(0xc0004b8080, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/inl.go:1428 +0x45
cmd/compile/internal/gc.inspect(0xc0004b8080, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1072 +0x43
cmd/compile/internal/gc.inspectList(0xc0004b2480, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.inspect(0xc0004b8880, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1075 +0x66
cmd/compile/internal/gc.inspect(0xc00048d680, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1076 +0x85
cmd/compile/internal/gc.inspectList(0xc00011d8c0, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.inspect(0xc00046fb00, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1078 +0xc5
cmd/compile/internal/gc.inspectList(0xc00011d8d8, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.devirtualize(0xc000192580)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/inl.go:1426 +0x4b
cmd/compile/internal/gc.Main(0xcbbce0)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/main.go:706 +0x324b
main.main()
	/home/jake/sdk/gotip/src/cmd/compile/main.go:52 +0xb1

# github.com/golang/protobuf/proto
../../../go/pkg/mod/github.com/golang/protobuf@v1.4.3/proto/deprecated.go:18:2: internal compiler error: devirtualization failed: xtd.(*impl.extensionTypeDescriptor).ExtensionDescriptor.HasDefault

goroutine 1 [running]:
runtime/debug.Stack(0xdaf700, 0xc00000e018, 0x0)
	/home/jake/sdk/gotip/src/runtime/debug/stack.go:24 +0x9f
cmd/compile/internal/gc.Fatalf(0xca427c, 0x1b, 0xc0007235e0, 0x1, 0x1)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/subr.go:199 +0x1b0
cmd/compile/internal/gc.devirtualizeCall(0xc000429e80)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/inl.go:1454 +0x2ab
cmd/compile/internal/gc.devirtualize.func1(0xc000429e80, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/inl.go:1428 +0x45
cmd/compile/internal/gc.inspect(0xc000429e80, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1072 +0x43
cmd/compile/internal/gc.inspectList(0xc00000d8f0, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.inspect(0xc000429d80, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1078 +0xc5
cmd/compile/internal/gc.inspectList(0xc00000d950, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.inspect(0xc000429a00, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1078 +0xc5
cmd/compile/internal/gc.inspectList(0xc00000da58, 0xcbbe50)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.devirtualize(0xc0001542c0)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/inl.go:1426 +0x4b
cmd/compile/internal/gc.Main(0xcbbce0)
	/home/jake/sdk/gotip/src/cmd/compile/internal/gc/main.go:706 +0x324b
main.main()
	/home/jake/sdk/gotip/src/cmd/compile/main.go:52 +0xb1

FAIL	github.com/hortbot/hortbot/internal/bot [build failed]
FAIL

cc @mdempsky

@zikaeroh zikaeroh changed the title cmd/compile: internal compiler error: devirtualization failed cmd/compile: internal compiler error: devirtualization failed (x/text, protobuf) Oct 29, 2020
@mdempsky mdempsky self-assigned this Oct 29, 2020
@mdempsky mdempsky added this to the Go1.16 milestone Oct 29, 2020
@mdempsky
Copy link
Member

@mdempsky mdempsky commented Oct 29, 2020

Thanks, on it.

Loading

@gopherbot
Copy link

@gopherbot gopherbot commented Oct 29, 2020

Change https://golang.org/cl/266359 mentions this issue: cmd/compile: gracefully fail when devirtualization fails

Loading

@zikaeroh
Copy link
Contributor Author

@zikaeroh zikaeroh commented Oct 29, 2020

Confusingly, golang.org/x/text@v0.3.3/encoding/encoding.go at line 293 isn't even what the error reports, its an errors.New call with a string.

var ErrInvalidUTF8 = errors.New("encoding: invalid UTF-8")

Loading

@kevinburke1
Copy link

@kevinburke1 kevinburke1 commented Oct 29, 2020

In case more test cases helps

$ go build ./...
# github.com/meterup/auth/vendor/github.com/golang/protobuf/proto
vendor/github.com/golang/protobuf/proto/deprecated.go:18:2: internal compiler error: devirtualization failed: xtd.(*impl.extensionTypeDescriptor).ExtensionDescriptor.HasDefault

goroutine 1 [running]:
runtime/debug.Stack(0x19adbe0, 0xc0000b8008, 0x0)
	/Users/kevin/go/src/runtime/debug/stack.go:24 +0x9f
cmd/compile/internal/gc.Fatalf(0x18a1cc4, 0x1b, 0xc0008df5e0, 0x1, 0x1)
	/Users/kevin/go/src/cmd/compile/internal/gc/subr.go:199 +0x1b0
cmd/compile/internal/gc.devirtualizeCall(0xc000142200)
	/Users/kevin/go/src/cmd/compile/internal/gc/inl.go:1454 +0x2ab
cmd/compile/internal/gc.devirtualize.func1(0xc000142200, 0x18b9748)
	/Users/kevin/go/src/cmd/compile/internal/gc/inl.go:1428 +0x45
cmd/compile/internal/gc.inspect(0xc000142200, 0x18b9748)
	/Users/kevin/go/src/cmd/compile/internal/gc/syntax.go:1072 +0x43
cmd/compile/internal/gc.inspectList(0xc00041a108, 0x18b9748)
	/Users/kevin/go/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.inspect(0xc000142100, 0x18b9748)
	/Users/kevin/go/src/cmd/compile/internal/gc/syntax.go:1078 +0xc5
cmd/compile/internal/gc.inspectList(0xc00041a168, 0x18b9748)
	/Users/kevin/go/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.inspect(0xc00013fd80, 0x18b9748)
	/Users/kevin/go/src/cmd/compile/internal/gc/syntax.go:1078 +0xc5
cmd/compile/internal/gc.inspectList(0xc00041a270, 0x18b9748)
	/Users/kevin/go/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.devirtualize(0xc000419340)
	/Users/kevin/go/src/cmd/compile/internal/gc/inl.go:1426 +0x4b
cmd/compile/internal/gc.Main(0x18b95d8)
	/Users/kevin/go/src/cmd/compile/internal/gc/main.go:706 +0x324b
main.main()
	/Users/kevin/go/src/cmd/compile/main.go:52 +0xb1

that's github.com/golang/protobuf v1.4.2

Loading

@mdempsky
Copy link
Member

@mdempsky mdempsky commented Oct 29, 2020

Confusingly, golang.org/x/text@v0.3.3/encoding/encoding.go at line 293 isn't even what the error reports, its an errors.New call with a string.

Yeah, I noticed that too. I think that's an artifact of Fatalf using lineno, but devirtualizeCall being new and written in a style that avoids setting lineno. So Fatalf is misreporting against the last time we set lineno (which was probably the last function to be analyzed for inlining).

Loading

@mdempsky
Copy link
Member

@mdempsky mdempsky commented Oct 29, 2020

Looks like both of the reported failure cases are because of a struct type having a promoted method from an embedded interface-typed field. So we can't fully devirtualize those calls, but we can still improve them somewhat.

Loading

gopherbot pushed a commit that referenced this issue Oct 29, 2020
We should still be able to devirtualize here, but I need to understand
the AST better. While I'm doing that, at least switch to a graceful
failure case (i.e., skip the optimization and print a warning message)
to fix the x/text builders.

Updates #42279.

Change-Id: Ie2b0b701fccf590d0cabfead703fc2fa999072cf
Reviewed-on: https://go-review.googlesource.com/c/go/+/266359
Trust: Matthew Dempsky <mdempsky@google.com>
Run-TryBot: Matthew Dempsky <mdempsky@google.com>
Reviewed-by: David Chase <drchase@google.com>
TryBot-Result: Go Bot <gobot@golang.org>
@gopherbot
Copy link

@gopherbot gopherbot commented Oct 29, 2020

Change https://golang.org/cl/266300 mentions this issue: cmd/compile: fix devirtualization of promoted interface methods

Loading

@dmitshur dmitshur changed the title cmd/compile: internal compiler error: devirtualization failed (x/text, protobuf) cmd/compile: internal compiler error: devirtualization failed (x/text, x/build, protobuf) Oct 30, 2020
@dmitshur
Copy link
Contributor

@dmitshur dmitshur commented Oct 30, 2020

Two more data points from x/build:


# golang.org/x/build/cmd/coordinator [golang.org/x/build/cmd/coordinator.test]
cmd/coordinator/builders.go:43:5: internal compiler error: devirtualization failed: tlsLn.(*tls.listener).Listener.Addr

goroutine 1 [running]:
runtime/debug.Stack(0x19adbc0, 0xc0000ba008, 0x0)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/runtime/debug/stack.go:24 +0x9f
cmd/compile/internal/gc.Fatalf(0x18a1cc4, 0x1b, 0xc0008f3618, 0x1, 0x1)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/subr.go:199 +0x1b0
cmd/compile/internal/gc.devirtualizeCall(0xc000077d80)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/inl.go:1454 +0x2ab
cmd/compile/internal/gc.devirtualize.func1(0xc000077d80, 0x0)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/inl.go:1428 +0x45
cmd/compile/internal/gc.inspect(0xc000077d80, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1072 +0x43
cmd/compile/internal/gc.inspect(0xc0014b9800, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1076 +0x85
cmd/compile/internal/gc.inspectList(0xc0000afda0, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.inspect(0xc000077c00, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1078 +0xc5
cmd/compile/internal/gc.inspectList(0xc0000afe30, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.devirtualize(0xc0001242c0)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/inl.go:1426 +0x4b
cmd/compile/internal/gc.Main(0x18b95d0)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/main.go:706 +0x324b
main.main()
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/main.go:52 +0xb1
# golang.org/x/build/devapp [golang.org/x/build/devapp.test]
devapp/data.go:12:2: internal compiler error: devirtualization failed: tlsLn.(*tls.listener).Listener.Addr

goroutine 1 [running]:
runtime/debug.Stack(0x19adbc0, 0xc0000c0008, 0x0)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/runtime/debug/stack.go:24 +0x9f
cmd/compile/internal/gc.Fatalf(0x18a1cc4, 0x1b, 0xc000a29618, 0x1, 0x1)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/subr.go:199 +0x1b0
cmd/compile/internal/gc.devirtualizeCall(0xc000399180)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/inl.go:1454 +0x2ab
cmd/compile/internal/gc.devirtualize.func1(0xc000399180, 0x0)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/inl.go:1428 +0x45
cmd/compile/internal/gc.inspect(0xc000399180, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1072 +0x43
cmd/compile/internal/gc.inspect(0xc000efcf00, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1076 +0x85
cmd/compile/internal/gc.inspectList(0xc00000ced0, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.inspect(0xc000399000, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1078 +0xc5
cmd/compile/internal/gc.inspectList(0xc00000cf18, 0x18b9740)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/syntax.go:1085 +0x58
cmd/compile/internal/gc.devirtualize(0xc00042d4a0)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/inl.go:1426 +0x4b
cmd/compile/internal/gc.Main(0x18b95d0)
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/internal/gc/main.go:706 +0x324b
main.main()
	/private/var/folders/kh/5zzynz152r94t18yzstnrwx80000gn/T/workdir-host-darwin-10_15/go/src/cmd/compile/main.go:52 +0xb1

From https://build.golang.org/?repo=golang.org%2fx%2fbuild for Go commit 5cc43c5. (Other Go commits look okay.)

Loading

@gopherbot gopherbot closed this in e62adb1 Oct 30, 2020
@mdempsky
Copy link
Member

@mdempsky mdempsky commented Oct 30, 2020

@dmitshur Thanks. Those are both instances of promoted interface methods too, and should be handled correctly by e62adb1.

Loading

@golang golang locked and limited conversation to collaborators Oct 30, 2021
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants