runtime: "runtime: unknown pc" from C abort()

[prattmic]

What version of Go are you using (go version)?

$ go version
go version go1.16.6 linux/amd64

Does this issue reproduce with the latest release?

Yes, and tip, 1.15, 1.13 (didn't test further back)

What operating system and processor architecture are you using (go env)?

go env Output

$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/usr/local/google/home/mpratt/.cache/go-build"
GOENV="/usr/local/google/home/mpratt/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/usr/local/google/home/mpratt/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/usr/local/google/home/mpratt/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/google/home/mpratt/src/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/google/home/mpratt/src/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.6"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/usr/local/google/home/mpratt/Downloads/crash/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2193872406=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Run a program that makes a cgo call which ultimately calls abort().

https://play.golang.org/p/eu2cpyzjPyy

(Note that cgosymbolizer is not required here, it simply demonstrates that the number still has full information about the crash).

What did you expect to see?

Something kind of like this:

SIGABRT: abort
PC=0x7f00694ebce1 m=0 sigcode=18446744073709551610

goroutine 1 [syscall]:
__GI_raise
        ../sysdeps/unix/sysv/linux/raise.c:51 pc=0x7f00694ebce1
__GI_abort
        ./stdlib/abort.c:79 pc=0x7f00694d5536
non-Go function
        pc=0x4033e5
non-Go function
        pc=0x46124f
runtime.cgocall(0x46ac40, 0xc00005ef78, 0x0)
        /usr/local/google/home/mpratt/src/go/src/runtime/cgocall.go:154 +0x5b fp=0xc00005ef48 sp=0xc00005ef10 pc=0x40553b
main._Cfunc_Abort()
        _cgo_gotypes.go:38 +0x3c fp=0xc00005ef78 sp=0xc00005ef48 pc=0x46abdc
main.main()
        /usr/local/google/home/mpratt/Downloads/crash/crash.go:15 +0x25 fp=0xc00005ef88 sp=0xc00005ef78 pc=0x46ac25
runtime.main()
        /usr/local/google/home/mpratt/src/go/src/runtime/proc.go:225 +0x256 fp=0xc00005efe0 sp=0xc00005ef88 pc=0x434f16
runtime.goexit()
        /usr/local/google/home/mpratt/src/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc00005efe8 sp=0xc00005efe0 pc=0x4615c1

What did you see instead?

SIGABRT: abort
PC=0x7f00694ebce1 m=0 sigcode=18446744073709551610

goroutine 0 [idle]:
runtime: unknown pc 0x7f00694ebce1
stack: frame={sp:0x7ffe5a1c9120, fp:0x0} stack=[0x7ffe599ca3b8,0x7ffe5a1c93f0)
00007ffe5a1c9020:  000000c00005efe0  000000c00005efe8 
00007ffe5a1c9030:  000000c00005efe0  000000c00005efe8 
00007ffe5a1c9040:  0000000000000000  0000000000000000 
00007ffe5a1c9050:  00007ffe5a1c9208  000000000044a28c <runtime.copystack+780> 
00007ffe5a1c9060:  000000c000050000  000000c000050800 
00007ffe5a1c9070:  0000000000000000  000000c000000180 
00007ffe5a1c9080:  0000000000000000  0000000000000000 
00007ffe5a1c9090:  000000007fffffff  0000000000496f00 
00007ffe5a1c90a0:  00007ffe5a1c90e8  0000000000000000 
00007ffe5a1c90b0:  0000000000000006  0000000000000470 
00007ffe5a1c90c0:  000000c00005f000  000000c00005e000 
00007ffe5a1c90d0:  000000c000050390  000000c000050000 
00007ffe5a1c90e0:  000000c000050800  000000c000050000 
00007ffe5a1c90f0:  000000c000050800  000000000000e800 
00007ffe5a1c9100:  0000000000434ec4 <runtime.main+516>  000000020000f73c 
00007ffe5a1c9110:  0000000000000000  0000000000000000 
00007ffe5a1c9120: <0000000000000000  0000000000000000 
00007ffe5a1c9130:  0000000000000000  0000000000000000 
00007ffe5a1c9140:  00000000004425b1 <runtime.doInit+113>  0000000000014142 
00007ffe5a1c9150:  0000000000000000  0000000000000000 
00007ffe5a1c9160:  0000000000442b96 <runtime.doInit+1622>  ffffffff00014142 
00007ffe5a1c9170:  0000000000000000  0000000000000000 
00007ffe5a1c9180:  0000000000000000  0000000000000000 
00007ffe5a1c9190:  0000000000000000  0000000000000000 
00007ffe5a1c91a0:  fffffffe7fffffff  ffffffffffffffff 
00007ffe5a1c91b0:  ffffffffffffffff  ffffffffffffffff 
00007ffe5a1c91c0:  ffffffffffffffff  ffffffffffffffff 
00007ffe5a1c91d0:  ffffffffffffffff  ffffffffffffffff 
00007ffe5a1c91e0:  ffffffffffffffff  ffffffffffffffff 
00007ffe5a1c91f0:  ffffffffffffffff  ffffffffffffffff 
00007ffe5a1c9200:  ffffffffffffffff  ffffffffffffffff 
00007ffe5a1c9210:  ffffffffffffffff  ffffffffffffffff 
runtime: unknown pc 0x7f00694ebce1
stack: frame={sp:0x7ffe5a1c9120, fp:0x0} stack=[0x7ffe599ca3b8,0x7ffe5a1c93f0)
00007ffe5a1c9020:  000000c00005efe0  000000c00005efe8 
00007ffe5a1c9030:  000000c00005efe0  000000c00005efe8 
00007ffe5a1c9040:  0000000000000000  0000000000000000 
00007ffe5a1c9050:  00007ffe5a1c9208  000000000044a28c <runtime.copystack+780> 
00007ffe5a1c9060:  000000c000050000  000000c000050800 
00007ffe5a1c9070:  0000000000000000  000000c000000180 
00007ffe5a1c9080:  0000000000000000  0000000000000000 
00007ffe5a1c9090:  000000007fffffff  0000000000496f00 
00007ffe5a1c90a0:  00007ffe5a1c90e8  0000000000000000 
00007ffe5a1c90b0:  0000000000000006  0000000000000470 
00007ffe5a1c90c0:  000000c00005f000  000000c00005e000 
00007ffe5a1c90d0:  000000c000050390  000000c000050000 
00007ffe5a1c90e0:  000000c000050800  000000c000050000 
00007ffe5a1c90f0:  000000c000050800  000000000000e800 
00007ffe5a1c9100:  0000000000434ec4 <runtime.main+516>  000000020000f73c 
00007ffe5a1c9110:  0000000000000000  0000000000000000 
00007ffe5a1c9120: <0000000000000000  0000000000000000 
00007ffe5a1c9130:  0000000000000000  0000000000000000 
00007ffe5a1c9140:  00000000004425b1 <runtime.doInit+113>  0000000000014142 
00007ffe5a1c9150:  0000000000000000  0000000000000000 
00007ffe5a1c9160:  0000000000442b96 <runtime.doInit+1622>  ffffffff00014142 
00007ffe5a1c9170:  0000000000000000  0000000000000000 
00007ffe5a1c9180:  0000000000000000  0000000000000000 
00007ffe5a1c9190:  0000000000000000  0000000000000000 
00007ffe5a1c91a0:  fffffffe7fffffff  ffffffffffffffff 
00007ffe5a1c91b0:  ffffffffffffffff  ffffffffffffffff 
00007ffe5a1c91c0:  ffffffffffffffff  ffffffffffffffff 
00007ffe5a1c91d0:  ffffffffffffffff  ffffffffffffffff 
00007ffe5a1c91e0:  ffffffffffffffff  ffffffffffffffff 
00007ffe5a1c91f0:  ffffffffffffffff  ffffffffffffffff 
00007ffe5a1c9200:  ffffffffffffffff  ffffffffffffffff 
00007ffe5a1c9210:  ffffffffffffffff  ffffffffffffffff 

goroutine 1 [syscall]:
__GI_raise
        ../sysdeps/unix/sysv/linux/raise.c:51 pc=0x7f00694ebce1
__GI_abort
        ./stdlib/abort.c:79 pc=0x7f00694d5536
non-Go function
        pc=0x4033e5
non-Go function
        pc=0x46124f
runtime.cgocall(0x46ac40, 0xc00005ef78, 0x0)
        /usr/local/google/home/mpratt/src/go/src/runtime/cgocall.go:154 +0x5b fp=0xc00005ef48 sp=0xc00005ef10 pc=0x40553b
main._Cfunc_Abort()
        _cgo_gotypes.go:38 +0x3c fp=0xc00005ef78 sp=0xc00005ef48 pc=0x46abdc
main.main()
        /usr/local/google/home/mpratt/Downloads/crash/crash.go:15 +0x25 fp=0xc00005ef88 sp=0xc00005ef78 pc=0x46ac25
runtime.main()
        /usr/local/google/home/mpratt/src/go/src/runtime/proc.go:225 +0x256 fp=0xc00005efe0 sp=0xc00005ef88 pc=0x434f16
runtime.goexit()
        /usr/local/google/home/mpratt/src/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc00005efe8 sp=0xc00005efe0 pc=0x4615c1

rax    0x0
rbx    0x7f00694ad740
rcx    0x7f00694ebce1
rdx    0x0
rdi    0x2
rsi    0x7ffe5a1c9120
rbp    0xc00005ef38
rsp    0x7ffe5a1c9120
r8     0x0
r9     0x7ffe5a1c9120
r10    0x8
r11    0x246
r12    0x1
r13    0x1
r14    0x1e
r15    0xffffffffffffffff
rip    0x7f00694ebce1
rflags 0x246
cs     0x33
fs     0x0
gs     0x0
exit status 2

[prattmic]

The immediate issue is:

• cgocall sets g.syscallsp;
• We then calls asmcgocall, which switches to g0 before calling the C function
• When the signal arrives, sigtramp's getg() gets g0.
• In traceback1, even though we have cgoCallers, this check fails because g0.syscallsp is not set (g0.m.curg.syscallsp is set instead), so we don't print the cgo traceback and try to print this as a Go traceback.

[prattmic]

https://cs.opensource.google/go/go/+/master:src/runtime/signal_unix.go;l=630-633?ss=go is supposed to handle this case, however http://golang.org/cl/64070 removed lockOSThread from cgocall.

Sure enough, I get the expected output on Go 1.9 and the current output on Go 1.10 (after CL 64070).

[gopherbot]

Change https://golang.org/cl/339989 mentions this issue: runtime: fix cgo signals detection