Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/compile, x/sync/errgroup: TestWithContext failing after devirtualization CL #42284

Closed
zikaeroh opened this issue Oct 29, 2020 · 15 comments
Closed

Comments

@zikaeroh
Copy link
Contributor

@zikaeroh zikaeroh commented Oct 29, 2020

What version of Go are you using (go version)?

$ go version
go version devel +fe70a3a0fd Thu Oct 29 21:46:54 2020 +0000 linux/amd64

Does this issue reproduce with the latest release?

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/jake/.cache/go-build"
GOENV="/home/jake/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/jake/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/jake/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/jake/sdk/gotip"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/jake/sdk/gotip/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/jake/zikaeroh/sync/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build875916662=/tmp/go-build -gno-record-gcc-switches"
GOROOT/bin/go version: go version devel +fe70a3a0fd Thu Oct 29 21:46:54 2020 +0000 linux/amd64
GOROOT/bin/go tool compile -V: compile version devel +fe70a3a0fd Thu Oct 29 21:46:54 2020 +0000
uname -sr: Linux 5.9.1-zen2-1-zen
/usr/lib/libc.so.6: GNU C Library (GNU libc) release release version 2.32.
gdb --version: GNU gdb (GDB) 9.2

What did you do?

In the golang.org/x/sync/errgroup package, run go test.

What did you expect to see?

Passing tests.

What did you see instead?

--- FAIL: TestWithContext (0.00s)
panic: interface conversion: context.Context is *context.cancelCtx, not *context.emptyCtx [recovered]
	panic: interface conversion: context.Context is *context.cancelCtx, not *context.emptyCtx

goroutine 29 [running]:
testing.tRunner.func1.2(0x5b0e80, 0xc0000a4e70)
	/home/jake/sdk/gotip/src/testing/testing.go:1123 +0x332
testing.tRunner.func1(0xc000083080)
	/home/jake/sdk/gotip/src/testing/testing.go:1126 +0x4b6
panic(0x5b0e80, 0xc0000a4e70)
	/home/jake/sdk/gotip/src/runtime/panic.go:965 +0x1b9
golang.org/x/sync/errgroup_test.TestWithContext(0xc000083080)
	/home/jake/zikaeroh/sync/errgroup/errgroup_test.go:166 +0x652
testing.tRunner(0xc000083080, 0x5f4c88)
	/home/jake/sdk/gotip/src/testing/testing.go:1173 +0xef
created by testing.(*T).Run
	/home/jake/sdk/gotip/src/testing/testing.go:1218 +0x2b3
exit status 2
FAIL	golang.org/x/sync/errgroup	0.005s

https://build.golang.org/log/8759ffb3dd087325958f6d2d67773f3dd2f70cf2
https://build.golang.org/log/32b25547fddc3b46c217c9efffb9aa124df3766e

The dashboard shows this test consistently failing after the devirtualization CL, but this differs from #42279 as it affects runtime behavior.

cc @mdempsky

@dmitshur dmitshur added this to the Backlog milestone Oct 30, 2020
@dmitshur
Copy link
Contributor

@dmitshur dmitshur commented Oct 30, 2020

Now that #42279 is resolved, is this still an issue? I'm not sure I understand how this issue is different.

Loading

@zikaeroh
Copy link
Contributor Author

@zikaeroh zikaeroh commented Oct 30, 2020

See https://build.golang.org/?repo=golang.org%2fx%2fsync; the page is all red on tip.

Loading

@dmitshur
Copy link
Contributor

@dmitshur dmitshur commented Oct 30, 2020

Thanks. I can reproduce locally:


$ gotip version
go version devel +f7e26467b Fri Oct 30 01:31:10 2020 +0000 darwin/amd64
$ gotip test golang.org/x/sync/errgroup
--- FAIL: TestWithContext (0.00s)
panic: interface conversion: context.Context is *context.cancelCtx, not *context.emptyCtx [recovered]
	panic: interface conversion: context.Context is *context.cancelCtx, not *context.emptyCtx

goroutine 14 [running]:
testing.tRunner.func1.2(0x11eab00, 0xc00007ae70)
	/Users/dmitshur/sdk/gotip/src/testing/testing.go:1123 +0x332
testing.tRunner.func1(0xc000154000)
	/Users/dmitshur/sdk/gotip/src/testing/testing.go:1126 +0x4b6
panic(0x11eab00, 0xc00007ae70)
	/Users/dmitshur/sdk/gotip/src/runtime/panic.go:965 +0x1b9
golang.org/x/sync/errgroup_test.TestWithContext(0xc000154000)
	/Users/dmitshur/go/src/golang.org/x/sync/errgroup/errgroup_test.go:166 +0x652
testing.tRunner(0xc000154000, 0x1233b40)
	/Users/dmitshur/sdk/gotip/src/testing/testing.go:1173 +0xef
created by testing.(*T).Run
	/Users/dmitshur/sdk/gotip/src/testing/testing.go:1218 +0x2b3
FAIL	golang.org/x/sync/errgroup	0.323s
FAIL

Loading

@dmitshur dmitshur removed this from the Backlog milestone Oct 30, 2020
@dmitshur dmitshur added this to the Go1.16 milestone Oct 30, 2020
@gopherbot
Copy link

@gopherbot gopherbot commented Oct 30, 2020

Change https://golang.org/cl/266518 mentions this issue: cmd/compile: mark inlined functions's results as reassigned

Loading

@mdempsky mdempsky self-assigned this Oct 30, 2020
@mdempsky
Copy link
Member

@mdempsky mdempsky commented Oct 30, 2020

Sorry, missed this yesterday. Investigating.

Loading

@mdempsky
Copy link
Member

@mdempsky mdempsky commented Oct 30, 2020

A workaround for the issue would be to to modify errgroup.WithContext like this:

 func WithContext(ctx context.Context) (*Group, context.Context) {
-       ctx, cancel := context.WithCancel(ctx)
-       return &Group{cancel: cancel}, ctx
+       ctx1, cancel := context.WithCancel(ctx)
+       return &Group{cancel: cancel}, ctx1
 }

It's a compiler bug that the current code (i.e., reassigning ctx) is miscompiled, but this change would be desirable anyway because it would play better with the current devirtualization code.

Loading

@zikaeroh
Copy link
Contributor Author

@zikaeroh zikaeroh commented Oct 30, 2020

It's a compiler bug that the current code (i.e., reassigning ctx) is miscompiled, but this change would be desirable anyway because it would play better with the current devirtualization code.

Reassigning over ctx is probably the most common thing to do with contexts; it'd be a shame if things didn't work properly without making variable name changes. But, maybe in fixing this bug it'll fix that too.

Loading

@gopherbot
Copy link

@gopherbot gopherbot commented Oct 30, 2020

Change https://golang.org/cl/266618 mentions this issue: cmd/compile: fix reassignVisitor

Loading

@mdempsky
Copy link
Member

@mdempsky mdempsky commented Oct 30, 2020

Reassigning over ctx is probably the most common thing to do with contexts; it'd be a shame if things didn't work properly without making variable name changes.

Once we move escape analysis to SSA, this should be straightforward to handle. But right now, the Node AST doesn't make it easy to do. We'd have to effectively reimplement SSA renaming in the frontend.

Loading

@zikaeroh
Copy link
Contributor Author

@zikaeroh zikaeroh commented Oct 30, 2020

Ah, I had no idea this was at the AST level where you couldn't distinguish it. Clearly I'm not paying enough attention during your streams. 🙂

Loading

@gopherbot gopherbot closed this in 7191f11 Oct 30, 2020
@cespare
Copy link
Contributor

@cespare cespare commented Oct 30, 2020

@mdempsky I'm really excited that we're getting these devirtualization optimizations and there are several places in my own code where I anticipate them making a big difference. That said, it's a little concerning when an optimization applies only when code is written in a non-standard (and arguably worse) way. Depending on how many release cycles the devirtualization handles the ctx1 case but not the ctx case, it's the kind of thing that can work its way into the performance lore. I really hope in a year or two I don't start seeing high-performance code has been manually SSA-ified to satisfy the devirtualizer :)

Loading

@mdempsky
Copy link
Member

@mdempsky mdempsky commented Oct 30, 2020

@cespare That's a fair point. There's certainly a trade off though: the more effort put into improving the early devirtualization code is effort taken away from moving escape analysis to SSA. The current early devirtualization pass is basically just a minor extension of the inlining / escape analysis improvements implemented for #41474

If you do see that practice developing, let us know and we can revisit whether there are more cases that we can handle without too much trouble.

Loading

@kivikakk
Copy link

@kivikakk kivikakk commented Dec 20, 2020

I might be off base here, disregard if so, but this ticket is the one search result for

panic: interface conversion: context.Context is *context.cancelCtx, not *context.emptyCtx

and I'm hitting it, so thought it may be related. It's unfortunately deeply nested in use (websocket library used by a Discord client library), I'm not quite able to extract a simple repro yet, and I'm on ARM64. Not a support request, just dropping this in case it's useful.

Here's the line that it hits on:

		case <-readCtx.Done():

I've tried to extract this into a self-contained reproduction, but it doesn't fail :( Here's what it looks like running:

$ ~/Code/gopath/bin/gotip version
go version devel +55b5801 Sat Dec 19 00:20:38 2020 +0000 darwin/arm64
$ ~/Code/gopath/bin/gotip run main.go 
panic: interface conversion: context.Context is *context.cancelCtx, not *context.emptyCtx

goroutine 66 [running]:
nhooyr.io/websocket.(*Conn).timeoutLoop(0x140000141a0)
        /Users/kameliya/Code/gopath/pkg/mod/nhooyr.io/websocket@v1.8.6/conn_notjs.go:161 +0x3d0
created by nhooyr.io/websocket.newConn
        /Users/kameliya/Code/gopath/pkg/mod/nhooyr.io/websocket@v1.8.6/conn_notjs.go:114 +0x37c
exit status 2

Loading

@mdempsky
Copy link
Member

@mdempsky mdempsky commented Dec 20, 2020

@kivikakk Thanks for the report! I was able to create a reproducer.

Loading

@kivikakk
Copy link

@kivikakk kivikakk commented Dec 20, 2020

Nice, thank you! I'll watch with interest.

Loading

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
6 participants