Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: unexpected signal during async preemption on windows/amd64 #44679

Closed
zx2c4 opened this issue Feb 28, 2021 · 10 comments
Closed

runtime: unexpected signal during async preemption on windows/amd64 #44679

zx2c4 opened this issue Feb 28, 2021 · 10 comments

Comments

@zx2c4
Copy link
Contributor

@zx2c4 zx2c4 commented Feb 28, 2021

With @rsc's arm64 changes, I'm getting reports of crashes on amd64 involving signals during async preemption:

2021-02-28 10:31:31.978278: [TUN] fatal error: unexpected signal during runtime execution
2021-02-28 10:31:31.981817: [TUN] [signal 0xc0000005 code=0x0 addr=0xb1 pc=0xaf626a]
2021-02-28 10:31:31.981817: [TUN] runtime stack:
2021-02-28 10:31:31.981817: [TUN] runtime.throw(0xeffc67, 0x2a)
2021-02-28 10:31:31.981817: [TUN] runtime/panic.go:1117 +0x79
2021-02-28 10:31:31.981817: [TUN] runtime.sigpanic()
2021-02-28 10:31:31.981817: [TUN] runtime/signal_windows.go:237 +0x317
2021-02-28 10:31:31.981817: [TUN] runtime.wantAsyncPreempt(...)
2021-02-28 10:31:31.981817: [TUN] runtime/preempt.go:344
2021-02-28 10:31:31.981817: [TUN] runtime.preemptM(0xc000035400)
2021-02-28 10:31:31.981817: [TUN] runtime/os_windows.go:1306 +0x24a
2021-02-28 10:31:31.981817: [TUN] runtime.preemptone(0xc000027800, 0xf9061ffdc8)
2021-02-28 10:31:31.981817: [TUN] runtime/proc.go:5307 +0xb2
2021-02-28 10:31:31.981817: [TUN] runtime.retake(0x2de1c0936154, 0x200000000)
2021-02-28 10:31:31.981817: [TUN] runtime/proc.go:5214 +0x245
2021-02-28 10:31:31.981817: [TUN] runtime.sysmon()
2021-02-28 10:31:31.981817: [TUN] runtime/proc.go:5156 +0x196
2021-02-28 10:31:31.981817: [TUN] runtime.mstart1()
2021-02-28 10:31:31.981817: [TUN] runtime/proc.go:1317 +0xda
2021-02-28 10:31:31.981817: [TUN] runtime.mstart0()
2021-02-28 10:31:31.981817: [TUN] runtime/proc.go:1275 +0x6d
2021-02-28 10:31:31.981817: [TUN] runtime.mstart(0x7ffe62037034, 0x0, 0x0, 0x0, 0x0, 0x0, 0x7ffe634fd241, 0x0, 0x0, 0x0, ...)
2021-02-28 10:31:31.981817: [TUN] runtime/asm_amd64.s:256 +0x5
2021-02-28 10:31:31.981817: [TUN] goroutine 1 [syscall, locked to thread]:
2021-02-28 10:31:31.981817: [TUN] syscall.Syscall(0x7ffe615c7c00, 0x1, 0xc00010fc58, 0x0, 0x0, 0x0, 0x0, 0x0)
2021-02-28 10:31:31.981817: [TUN] runtime/syscall_windows.go:332 +0xe9
2021-02-28 10:31:31.981817: [TUN] golang.org/x/sys/windows.StartServiceCtrlDispatcher(0xc00010fc58, 0xf1e5a8, 0xc000043540)
2021-02-28 10:31:31.981817: [TUN] golang.org/x/sys@v0.0.0-20210225134936-a50acf3fe073/windows/zsyscall_windows.go:1154 +0x8a
2021-02-28 10:31:31.981817: [TUN] golang.org/x/sys/windows/svc.Run(0xc000012258, 0x18, 0xf7cf00, 0xc000037cf0, 0x8, 0xc000012258)
2021-02-28 10:31:31.981817: [TUN] golang.org/x/sys@v0.0.0-20210225134936-a50acf3fe073/windows/svc/service.go:366 +0x1c8

Cc @aclements who wrote preemption support.

@zx2c4
Copy link
Contributor Author

@zx2c4 zx2c4 commented Feb 28, 2021

Looking more carefully,

func wantAsyncPreempt(gp *g) bool {
	// Check both the G and the P.
	return (gp.preempt || gp.m.p != 0 && gp.m.p.ptr().preempt) && readgstatus(gp)&^_Gscan == _Grunning
}

Is this panicking because gp.m.p.ptr() points somewhere bogus?

@zx2c4
Copy link
Contributor Author

@zx2c4 zx2c4 commented Feb 28, 2021

Ahhh more likely is that gp is bogus. That comes from:

	// Does it want a preemption and is it safe to preempt?
	gp := gFromSP(mp, c.sp())
	if wantAsyncPreempt(gp) {
@zx2c4
Copy link
Contributor Author

@zx2c4 zx2c4 commented Feb 28, 2021

@rsc might be an issue with ece954d#diff-fc175f04ebb256c1d34c14d27b8915f38928b71df55a35bfbd86fcb4618ff5a9

Maybe it's simply enough to check for nil... I'll send a CL.

@gopherbot
Copy link

@gopherbot gopherbot commented Feb 28, 2021

Change https://golang.org/cl/297390 mentions this issue: runtime: see whether gp==nil before checking preemption state

@zx2c4
Copy link
Contributor Author

@zx2c4 zx2c4 commented Feb 28, 2021

I've deployed https://golang.org/cl/297390 and it evidently fixes the crash users were seeing.

@networkimprov
Copy link

@networkimprov networkimprov commented Mar 1, 2021

@gopherbot add OS-Windows

@gopherbot gopherbot added the OS-Windows label Mar 1, 2021
@dmitshur dmitshur added this to the Backlog milestone Mar 2, 2021
@dmitshur dmitshur changed the title windows/amd64: unexpected signal during async preemption runtime: unexpected signal during async preemption on windows/amd64 Mar 2, 2021
@bcmills
Copy link
Member

@bcmills bcmills commented Mar 17, 2021

Is https://build.golang.org/log/995be8dc15fe8289ca78257701e522e5b5d51336 (windows-386) likely to be due to the same root cause?

@bcmills
Copy link
Member

@bcmills bcmills commented Apr 6, 2021

Another windows-386 TryBot failure with unexpected signal during runtime execution in runtime.wantAsyncPreempt:
https://storage.googleapis.com/go-build-log/764770a7/windows-386-2008_4820c228.log

@zx2c4
Copy link
Contributor Author

@zx2c4 zx2c4 commented Apr 6, 2021

Looks familiar. https://golang.org/cl/297390 fixes the issue, but I'm not able to reproduce it myself to get more info supporting that CL.

@zx2c4
Copy link
Contributor Author

@zx2c4 zx2c4 commented Apr 6, 2021

@gopherbot gopherbot closed this in 98dd205 Apr 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants