Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: TestArenaCollision failing on linux/amd64 #24670

Closed
quasilyte opened this issue Apr 3, 2018 · 4 comments

Comments

Projects
None yet
5 participants
@quasilyte
Copy link
Contributor

commented Apr 3, 2018

linux-amd64-race build failed during CL104555 trybots run.

linux-amd64-race_cc3944b7.log:

--- FAIL: TestArenaCollision (0.24s)
	malloc_test.go:174: === RUN   TestArenaCollision
		==2548==ERROR: ThreadSanitizer failed to allocate 0x20000000 (536870912) bytes at address 21d06a0000000 (errno: 12)
		==2548==ERROR: ThreadSanitizer failed to allocate 0x10000000 (268435456) bytes at address 21d0690000000 (errno: 12)
		==2548==ERROR: ThreadSanitizer failed to allocate 0x10000000 (268435456) bytes at address 21d0670000000 (errno: 12)
		--- PASS: TestArenaCollision (0.19s)
		unexpected fault address 0x0
		fatal error: fault
		unexpected fault address 0x8013c9
		fatal error: fault
		[signal SIGSEGV: segmentation violation code=0x2 addr=0x8013c9 pc=0x8013c9]
		
		goroutine 1 [running, locked to thread]:
		runtime: unexpected return pc for runtime.throw called from 0x3
		stack: frame={sp:0x7ffe1ed44540, fp:0x7ffe1ed44570} stack=[0xc0000a0000,0xc0000a2000)
		
... lots of output

		runtime.gcBgMarkWorker(0xc000068500)
			/tmp/workdir/go/src/runtime/mgc.go:1775 +0x115 fp=0xc00007dfd8 sp=0xc00007df50 pc=0x441d95
		runtime.goexit()
			/tmp/workdir/go/src/runtime/asm_amd64.s:1385 +0x1 fp=0xc00007dfe0 sp=0xc00007dfd8 pc=0x486401
		created by runtime.gcBgMarkStartWorkers
			/tmp/workdir/go/src/runtime/mgc.go:1723 +0x77
		
		(exit status exit status 2)
FAIL
FAIL	runtime	34.593s
2018/04/03 19:44:22 Failed: exit status 1


Error: tests failed: dist test failed: go_test:runtime: exit status 1

I can reproduce this failure locally with race.bash.

Trying to find a commit that caused this.
(Update: it looks like a hit from a distance, investigation is needed to find real cause.)

@rasky

This comment has been minimized.

Copy link
Member

commented Apr 3, 2018

@bcmills bcmills added this to the Go1.11 milestone Apr 3, 2018

@aclements

This comment has been minimized.

Copy link
Member

commented Apr 4, 2018

I'm not sure why this just started, but it's almost certainly a consequence of #24133. I'm working on confirming that now. Most likely this test should just be disabled under the race detector until we resolve #24133.

@aclements

This comment has been minimized.

Copy link
Member

commented Apr 4, 2018

Yep:

$ cd src/runtime
$ go test -c -race
$ TEST_ARENA_COLLISION=1 ./runtime.test -test.short -test.run TestArenaCollision
map 0xc000000000
map 0x1c000000000
map 0x2c000000000
map 0x7ff36c000000
==112144==ERROR: ThreadSanitizer failed to allocate 0x10000000 (268435456) bytes at address 21fcdb0000000 (errno: 12)
map 0x7ff368000000
==112144==ERROR: ThreadSanitizer failed to allocate 0x10000000 (268435456) bytes at address 21fcda0000000 (errno: 12)
map 0x7ff360000000
==112144==ERROR: ThreadSanitizer failed to allocate 0x10000000 (268435456) bytes at address 21fcd80000000 (errno: 12)

When TestArenaCollision causes the runtime to run out of hint addresses, the heap not only skips forward ~128GB (which would mess up the race detector's assumption that the heap is contiguous), but moves well outside the race detector's supported range for the heap mapping. Given the latter, even if we were to fix #24133, I don't think this test would work under the race detector.

@gopherbot

This comment has been minimized.

Copy link

commented Apr 4, 2018

Change https://golang.org/cl/104717 mentions this issue: runtime: stop when we run out of hints in race mode

@gopherbot gopherbot closed this in 4946d9e Apr 4, 2018

@golang golang locked and limited conversation to collaborators Apr 4, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.