Join GitHub today
GitHub is home to over 40 million developers working together to host and review code, manage projects, and build software together.Sign up
runtime: sigprof handler crashes backtracing runtime.nanotime #24925
If a sigprof is received while runtime.nanotime is executing, the runtime may crash with a stack like this:
To add the running stack trace to the profile, the handler needs to walk the stack frame-by-frame, and to do that it needs to know how big each stack frame is. The compiler records that information in the pc-to-sp table, but runtime.nanotime does manual stack alignment:
which isn't (can't be) tracked in the table. When the signal handler walks the stack, it computes the wrong size for that frame, and then reads the wrong place looking for the return address, finding stack garbage instead and crashing.
The problem was introduced in a158382, which shipped in 1.10, so this should be considered for a backport for 1.10.2.
Hey, was this ever cherrypicked into an official release?
I unfortunately had a service running with 1.10.1 that crashed a few times daily if I enabled pprof with the same stack trace above. I took the cherrypick from https://golang.org/cl/107778 and it fixed it for good.
I didn't see this CL referenced in the Go 1.10.2 or 1.10.3 milestones. Looking over the source (below), I don't see it there either (or was this abandoned for another way to fix it?)
From runtime/proc.go in 1.10.3 line 3689 - 3691: