Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

net: apparent deadlock in TestCloseWrite on darwin-arm64-corellium #34837

Open
bcmills opened this issue Oct 11, 2019 · 5 comments

Comments

@bcmills
Copy link
Member

@bcmills bcmills commented Oct 11, 2019

From the darwin-arm64-corellium builder (https://build.golang.org/log/0f26cd7aadb20043bcb06081b5b9c0a633bcb9fe):

panic: test timed out after 3m0s

goroutine 601 [running]:
testing.(*M).startAlarm.func1()
	/tmp/workdir-host-darwin-arm64-corellium-ios/go/src/testing/testing.go:1377 +0xc0
created by time.goFunc
	/tmp/workdir-host-darwin-arm64-corellium-ios/go/src/time/sleep.go:168 +0x38

[…]

goroutine 599 [IO wait, 2 minutes]:
internal/poll.runtime_pollWait(0x10578ce98, 0x72, 0x102f20380)
	/tmp/workdir-host-darwin-arm64-corellium-ios/go/src/runtime/netpoll.go:184 +0x3c
internal/poll.(*pollDesc).wait(0x130356718, 0x72, 0x0, 0x1, 0xffffffffffffffff)
	/tmp/workdir-host-darwin-arm64-corellium-ios/go/src/internal/poll/fd_poll_runtime.go:87 +0x30
internal/poll.(*pollDesc).waitRead(...)
	/tmp/workdir-host-darwin-arm64-corellium-ios/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).Read(0x130356700, 0x13011000f, 0x1, 0x1, 0x0, 0x0, 0x0)
	/tmp/workdir-host-darwin-arm64-corellium-ios/go/src/internal/poll/fd_unix.go:169 +0x1b8
net.(*netFD).Read(0x130356700, 0x13011000f, 0x1, 0x1, 0x130356700, 0x102e44fd4, 0x1)
	/tmp/workdir-host-darwin-arm64-corellium-ios/go/src/net/fd_unix.go:202 +0x3c
net.(*conn).Read(0x130018010, 0x13011000f, 0x1, 0x1, 0x0, 0x0, 0x0)
	/tmp/workdir-host-darwin-arm64-corellium-ios/go/src/net/net.go:184 +0x68
net.TestCloseWrite(0x13017c600)
	/tmp/workdir-host-darwin-arm64-corellium-ios/go/src/net/net_test.go:151 +0x3cc
testing.tRunner(0x13017c600, 0x102f1ee80)
	/tmp/workdir-host-darwin-arm64-corellium-ios/go/src/testing/testing.go:909 +0xb0
created by testing.(*T).Run
	/tmp/workdir-host-darwin-arm64-corellium-ios/go/src/testing/testing.go:960 +0x29c

[…]

FAIL	net	180.181s

CC @mikioh @bradfitz @ianlancetaylor

@bcmills bcmills added this to the Go1.14 milestone Oct 11, 2019
@bcmills

This comment has been minimized.

Copy link
Member Author

@bcmills bcmills commented Oct 11, 2019

Another one on darwin-arm64-corellium. Is it possible that this is a recent regression?

https://build.golang.org/log/175d78eebd4aa686657b7faf57755ff9ee52d02e

@odeke-em

This comment was marked as off-topic.

Copy link
Member

@odeke-em odeke-em commented Oct 11, 2019

I was plagued by this failure today when preparing a CL https://go-review.googlesource.com/c/go/+/200437/3#message-6a3671e972d7a38bd009fee322ab2b627c5d43ff that happened on 9 different builders and it wasn't just for TestCloseWrite

panic: test timed out after 3m0s

goroutine 1438 [running]:
testing.(*M).startAlarm.func1()
	/tmp/workdir/go/src/testing/testing.go:1377 +0xdf
created by time.goFunc
	/tmp/workdir/go/src/time/sleep.go:168 +0x44

goroutine 1 [chan receive, 2 minutes]:
testing.(*T).Run(0xc0004dff00, 0x9697fe, 0x14, 0x9944a0, 0x4b9801)
	/tmp/workdir/go/src/testing/testing.go:961 +0x378
testing.runTests.func1(0xc000103600)
	/tmp/workdir/go/src/testing/testing.go:1202 +0x78
testing.tRunner(0xc000103600, 0xc00004cd98)
	/tmp/workdir/go/src/testing/testing.go:909 +0xc9
testing.runTests(0xc00009c420, 0xd6e8c0, 0x1f9, 0x1f9, 0x0)
	/tmp/workdir/go/src/testing/testing.go:1200 +0x2a7
testing.(*M).Run(0xc000100680, 0x0)
	/tmp/workdir/go/src/testing/testing.go:1117 +0x176
net/http_test.TestMain(0xc000100680)
	/tmp/workdir/go/src/net/http/main_test.go:23 +0x2b
main.main()
	_testmain.go:1118 +0x135

goroutine 39 [chan receive, 3 minutes]:
testing.(*T).Parallel(0xc000124d00)
	/tmp/workdir/go/src/testing/testing.go:814 +0x1d5
net/http.TestCmdGoNoHTTPServer(0xc000124d00)
	/tmp/workdir/go/src/net/http/http_test.go:86 +0x43
testing.tRunner(0xc000124d00, 0x992b48)
	/tmp/workdir/go/src/testing/testing.go:909 +0xc9
created by testing.(*T).Run
	/tmp/workdir/go/src/testing/testing.go:960 +0x351

9 of 19 TryBots failed:
Failed on freebsd-amd64-12_0: https://storage.googleapis.com/go-build-log/7f1ab5d0/freebsd-amd64-12_0_c04127fb.log
Failed on linux-amd64: https://storage.googleapis.com/go-build-log/7f1ab5d0/linux-amd64_8f96dcf8.log
Failed on linux-386: https://storage.googleapis.com/go-build-log/7f1ab5d0/linux-386_e5da7e31.log
Failed on openbsd-amd64-64: https://storage.googleapis.com/go-build-log/7f1ab5d0/openbsd-amd64-64_05cd85d9.log
Failed on linux-amd64-race: https://storage.googleapis.com/go-build-log/7f1ab5d0/linux-amd64-race_6b7636a1.log
Failed on android-amd64-emu: https://storage.googleapis.com/go-build-log/7f1ab5d0/android-amd64-emu_93d00e21.log
Failed on windows-386-2008: https://storage.googleapis.com/go-build-log/7f1ab5d0/windows-386-2008_b9f149d8.log
Failed on js-wasm: https://storage.googleapis.com/go-build-log/7f1ab5d0/js-wasm_365c9070.log
Failed on windows-amd64-2016: https://storage.googleapis.com/go-build-log/7f1ab5d0/windows-amd64-2016_92bf52aa.log

not just darwin-arm64-corellium and it wasn't for TestCloseWrite only, seems like a testing/runtime issue.

@bcmills

This comment was marked as off-topic.

Copy link
Member Author

@bcmills bcmills commented Oct 11, 2019

@odeke-em, that looks like a very different failure mode. You want to look for the goroutines that are not blocked on testing.(*T).Parallel, which for the links you provided seem to be exclusively net/http.(*connReader).abortPendingRead in TestWriteAfterHijack.

The defining symptom for this issue, in contrast, seems to be a hang in net.(*netFD).Read.

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Oct 11, 2019

I don't really see how but it's conceivable that this is a recent regression due to https://golang.org/cl/197938.

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Oct 11, 2019

The test is both fairly straightforward and not all that important. If someone wants to debug it, great, but I would be inclined to just skip it on darwim/arm64.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.