runtime: crash when C library resets sigaltstack/sigaction settings

[niemeyer]

Someone reported the following crash while using the qml package:

  fatal error: runtime: stack split during syscall

  runtime stack:
  runtime.throw(0x4299eac)
      /Users/quarnster/code/3rdparty/go/src/pkg/runtime/panic.c:462 +0x69
  runtime.newstack()
      /Users/quarnster/code/3rdparty/go/src/pkg/runtime/stack.c:261 +0x6cb
  runtime.morestack()
      /Users/quarnster/code/3rdparty/go/src/pkg/runtime/asm_amd64.s:225 +0x61

  goroutine 4 [stack split]:
  syscall.Syscall6(0x7, 0xf68, 0xc2100001b0, 0x0, 0xc21005f360, ...)
      /.../syscall/asm_darwin_amd64.s:41 +0x5 fp=0x5ffc078
  syscall.wait4(0xf68, 0xc2100001b0, 0x0, 0xc21005f360, 0x40450a2, ...)
      /.../syscall/zsyscall_darwin_amd64.go:32 +0x7d fp=0x5ffc0d8
  syscall.Wait4(0xf68, 0x5ffc15c, 0x0, 0xc21005f360, 0xc21000a260, ...)
      /.../syscall/syscall_bsd.go:126 +0x6e fp=0x5ffc120
  os.(*Process).wait(0xc21000a520, 0x0, 0x0, 0x0)

  (...)

The full stack trace is available in the issue at:

  go-qml/qml#26

[dvyukov]

Comment 1:

Is it reproducible? If yes, please run it with GOTRACEBACK=2

[niemeyer]

Comment 2:

It seems to be reproducible, but not from here. I'll ask the original reporter about it.

[niemeyer]

Comment 4:

As a hint, this is a case where Go calls C calls Go calls C.

[quarnster]

Comment 5:

13:15 /tmp $ GOTRACEBACK=2 go run main.go
2014/01/31 13:16:12 here
fatal error: runtime: stack split during syscall
runtime stack:
runtime.throw(0x42740cc)
    /Users/quarnster/code/3rdparty/go/src/pkg/runtime/panic.c:464 +0x69
runtime.newstack()
    /Users/quarnster/code/3rdparty/go/src/pkg/runtime/stack.c:261 +0x6c3
runtime.morestack()
    /Users/quarnster/code/3rdparty/go/src/pkg/runtime/asm_amd64.s:225 +0x61
goroutine 4 [stack split]:
runtime: unexpected return pc for runtime.sighandler called from 0x7fff8aee85aa
runtime.sighandler(0x6, 0x0, 0x0, 0x0)
    /Users/quarnster/code/3rdparty/go/src/pkg/runtime/signal_amd64.c:43 fp=0x83bcea0
created by github.com/niemeyer/qml.Init
    /Users/quarnster/code/go/src/github.com/niemeyer/qml/qml.go:58 +0xa0
goroutine 1 [chan receive]:
runtime.park(0x404db20, 0xc210038110, 0x427322d)
    /Users/quarnster/code/3rdparty/go/src/pkg/runtime/proc.c:1342 +0x66
runtime.chanrecv(0x40f6960, 0xc2100380c0, 0x83a1e08, 0x0, 0x0)
    /Users/quarnster/code/3rdparty/go/src/pkg/runtime/chan.c:354 +0x50b
runtime.chanrecv1(0x40f6960, 0xc2100380c0)
    /Users/quarnster/code/3rdparty/go/src/pkg/runtime/chan.c:446 +0x38
github.com/niemeyer/qml.gui(0xc210048140)
    /Users/quarnster/code/go/src/github.com/niemeyer/qml/bridge.go:69 +0xbd
github.com/niemeyer/qml.(*Common).Create(0xc21000a310, 0x0, 0x1a, 0x8211190)
    /Users/quarnster/code/go/src/github.com/niemeyer/qml/qml.go:636 +0x117
main.main()
    /private/tmp/main.go:33 +0x28a
runtime.main()
    /Users/quarnster/code/3rdparty/go/src/pkg/runtime/proc.c:220 +0x11f
runtime.goexit()
    /Users/quarnster/code/3rdparty/go/src/pkg/runtime/proc.c:1394
goroutine 2 [syscall]:
runtime.notetsleepg(0x83bff60, 0xdf8475800)
    /Users/quarnster/code/3rdparty/go/src/pkg/runtime/lock_sema.c:254 +0x71
runtime.MHeap_Scavenger()
    /Users/quarnster/code/3rdparty/go/src/pkg/runtime/mheap.c:463 +0xa3
runtime.goexit()
    /Users/quarnster/code/3rdparty/go/src/pkg/runtime/proc.c:1394
created by runtime.main
    /Users/quarnster/code/3rdparty/go/src/pkg/runtime/proc.c:179
goroutine 3 [syscall]:
runtime.goexit()
    /Users/quarnster/code/3rdparty/go/src/pkg/runtime/proc.c:1394
exit status 2
13:15 /tmp $ go version
go version go1.2 darwin/amd64

[dvyukov]

Comment 6:

I suspect that qml calls sigaltstack. This is not supported. And we need better
diagnostics for this.

[niemeyer]

Comment 7:

How are you checking this?  An uninformed check doesn't bring up anything..
[~/src/qt5]% grep sigalstack * -r
[~/src/qt5]%
 
[~/src/qt5]% nm -D /usr/lib/x86_64-linux-gnu/libQt5*.so | grep sigalstack
[~/src/qt5]%

[dvyukov]

Comment 8:

I think so because sighandler triggers stack split.
Try to run under gdb and set breakpoint to sigaltstack/signal/sigaction

[niemeyer]

Comment 9:

Understood. gdb doesn't find it either, though.

[dvyukov]

Comment 10:

The signal is SIGABRT. Why does it call abort()?

[niemeyer]

Comment 11:

Hard to tell without access to an environment where the problem happens.
It's curious that the traceback above is very different from the one in the original
report.

[dvyukov]

Comment 12:

Yes, but I suspect that a signal is still involved in the original crash stack.
This may also explain why it fails only on some machines -- on the bad machines
environment is not setup properly and qml call abort().

[ianlancetaylor]

Comment 13:

I agree with Dmitriy: I can only explain this if something is calling sigaltstack.  It
sounds like you have a test case that can recreate this; run it under strace -f and see
if sigaltstack is called anywhere.

[quarnster]

Comment 14:

No strace on OSX, but attached output of "sudo dtruss -f -a -s ./main". 
sigaltstack does indeed appear to be called. Unfortunately, if I read the output
correctly, it's called by Apple's system library CoreText.

Attachments:

1. dtruss.txt (2454978 bytes)

[quarnster]

Comment 15:

And if it's of any use, I'm on 10.9.1 and:
17:14 ~/code/go/src/github.com/niemeyer/qml $ git log -1
commit 9c937b147a7ce9fc9560593fa7d6e2ae49d8203b
Author: Gustavo Niemeyer <gustavo@niemeyer.net>
Date:   Wed Jan 29 16:17:41 2014 -0200
    Use pre-resolved propIndex when setting property.
17:14 ~/code/go/src/github.com/niemeyer/qml $ brew info qt5 gcc48 | grep /usr
/usr/local/Cellar/qt5/5.0.2 (2899 files, 119M)
/usr/local/Cellar/qt5/5.2.0 (5482 files, 172M) *
.app bundles were installed to /usr/local/Cellar/qt5/5.2.0 (or libexec).
/usr/local/Cellar/gcc48/4.8.1 (965 files, 90M) *
17:14 ~/code/go/src/github.com/niemeyer/qml $ go version
go version go1.2 darwin/amd64
17:14 ~/code/go/src/github.com/niemeyer/qml $

[quarnster]

Comment 16:

So if sigaltstack is to blame, I guess that makes this issue a dup of 4216 (and 5287),
which has been closed with a "unfortunate" label. Is there really nothing that can be
done? Would using kqueue aid in any way? From what I understand kqueue can be used to
"catch" signals without "owning" the signal handler:
http://doc.geoffgarside.co.uk/kqueue/signal.html.