Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: "attempt to execute system stack code on user stack" during heap scavenging #34773

Closed
bcmills opened this issue Oct 8, 2019 · 4 comments

Comments

@bcmills
Copy link
Member

commented Oct 8, 2019

From the linux-amd64-nocgo builder (https://build.golang.org/log/abbb70348ec45659c8032defde341c61e90ec995):

fatal error: attempt to execute system stack code on user stack

goroutine 8 [running]:
runtime.throw(0x65ad83, 0x32)
	/workdir/go/src/runtime/panic.go:774 +0x72 fp=0xc000030610 sp=0xc0000305e0 pc=0x430562
runtime.morestackc()
	/workdir/go/src/runtime/stack.go:1315 +0x27 fp=0xc000030630 sp=0xc000030610 pc=0x44b117
runtime.stackfree(0xc000492000, 0xc000494000)
	/workdir/go/src/runtime/stack.go:422 +0x35c fp=0xc000030638 sp=0xc000030630 pc=0x448b2c
runtime.allocm(0xc00001e500, 0x65e5f8, 0x1)
	/workdir/go/src/runtime/proc.go:1517 +0xdb fp=0xc000030678 sp=0xc000030638 pc=0x43556b
runtime.newm(0x65e5f8, 0xc00001e500)
	/workdir/go/src/runtime/proc.go:1841 +0x39 fp=0xc0000306a8 sp=0xc000030678 pc=0x435d39
runtime.startm(0x0, 0xc000000c01)
	/workdir/go/src/runtime/proc.go:2006 +0x12a fp=0xc0000306d8 sp=0xc0000306a8 pc=0x43630a
runtime.wakep(...)
	/workdir/go/src/runtime/proc.go:2087
runtime.ready(0xc000000c00, 0x0, 0xd5731d01)
	/workdir/go/src/runtime/proc.go:666 +0x12f fp=0xc000030728 sp=0xc0000306d8 pc=0x43359f
runtime.wakeScavenger()
	/workdir/go/src/runtime/mgcscavenge.go:236 +0x85 fp=0xc000030750 sp=0xc000030728 pc=0x422d55
runtime.bgscavenge.func1(0x0, 0x0, 0x0)
	/workdir/go/src/runtime/mgcscavenge.go:295 +0x20 fp=0xc000030760 sp=0xc000030750 pc=0x45e6b0
runtime.timerproc(0x806140)
	/workdir/go/src/runtime/time.go:294 +0x6e fp=0xc0000307d8 sp=0xc000030760 pc=0x44fabe
runtime.goexit()
	/workdir/go/src/runtime/asm_amd64.s:1375 +0x1 fp=0xc0000307e0 sp=0xc0000307d8 pc=0x463cd1
created by runtime.(*timersBucket).addtimerLocked
	/workdir/go/src/runtime/time.go:166 +0x10e

The failure message is the same as in #30591, but the entry here seems to be through runtime.wakeScavenger rather than runtime.LockOSThread so the root cause may differ.

CC @mknyszek @aclements

@bcmills bcmills added this to the Go1.14 milestone Oct 8, 2019
@mknyszek

This comment has been minimized.

Copy link
Contributor

commented Oct 8, 2019

Looking into it.

@mknyszek mknyszek self-assigned this Oct 8, 2019
@mknyszek

This comment has been minimized.

Copy link
Contributor

commented Oct 8, 2019

OK so ready() needs to be called on the system stack.

Either wakeScavenger should be go:systemstack or ready should be goready. I'm not sure whether it actually matters which one we do. No other part of wakeScavenger needs the system stack and I have a slight preference for allowing it to be run mostly anywhere, though in some cases (like SetGCPercent) this means we'll go onto the system stack twice. It doesn't happen in any place that's particularly performance sensitive.

@aclements, @randall77, WDYT?

@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@randall77

This comment has been minimized.

Copy link
Contributor

commented Oct 14, 2019

I'd rather just put things on the system stack that need it. It serves as useful documentation, if nothing else. If that means we have to switch stacks twice for non-performance-sensitive code, so be it.

@randall77 randall77 modified the milestones: Backlog, Go1.14 Oct 14, 2019
@gopherbot

This comment has been minimized.

Copy link

commented Oct 14, 2019

Change https://golang.org/cl/200999 mentions this issue: runtime: call goready in wakeScavenger instead of ready

@gopherbot gopherbot closed this in 2c87be4 Oct 15, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.