Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: systemstack call caused SIGSEGV under race mode #45768

Closed
PureWhiteWu opened this issue Apr 26, 2021 · 6 comments
Closed

runtime: systemstack call caused SIGSEGV under race mode #45768

PureWhiteWu opened this issue Apr 26, 2021 · 6 comments

Comments

@PureWhiteWu
Copy link
Contributor

@PureWhiteWu PureWhiteWu commented Apr 26, 2021

Hi all,

I'm trying to get time using CLOCK_REALTIME_COARSE and CLOCK_MONOTONIC_COARSE for performance reasons, and need to use vdso call by hand-written assembly code. That is, I want to reimplement time.Now using CLOCK_REALTIME_COARSE and CLOCK_MONOTONIC_COARSE.

I referenced the code in runtime and found that there's an issue #20427 indicates that I need to switch to g0 for vdso calls.

I tried to link the runtime.systemstack and use it to do vdso calls:

//go:linkname systemstack runtime.systemstack
//go:noescape
func systemstack(fn func())

My code is something like this:

// now calls vdso and is implemented in asm
func now() (sec int64, nsec int32, mono int64)

func Now() {
  var sec, mono int64
  var nsec int32
  systemstack(func() {
    sec, nsec, mono = now()
  })
  ... // logic copied from time.Now()
}

The code runs well without -race, but I encountered fatal error during go test -race:

fatal error: unexpected signal during runtime execution
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x418975]

runtime stack:
runtime.throw(0x6de30d, 0x2a)
	/usr/local/go/src/runtime/panic.go:1117 +0x72
runtime.sigpanic()
	/usr/local/go/src/runtime/signal_unix.go:718 +0x2e5

goroutine 20 [running]:
runtime.systemstack_switch()
	/usr/local/go/src/runtime/asm_amd64.s:339 fp=0xc000040540 sp=0xc000040538 pc=0x4a3920
xxx/xxx.Now(0xc019b3c31d55505d, 0xca7404, 0x85d4a0)
	/xxx/time_linux_amd64.go:27 +0xb0 fp=0xc0000405e8 sp=0xc000040540 pc=0x6745b0
xxx/xxx.TestNowStd(0xc000082f00)
	/xxx/time_test.go:27 +0x85 fp=0xc0000406d0 sp=0xc0000405e8 pc=0x674dc5
testing.tRunner(0xc000082f00, 0x6e6040)
	/usr/local/go/src/testing/testing.go:1193 +0x203 fp=0xc0000407d0 sp=0xc0000406d0 pc=0x5518e3
runtime.goexit()
	/usr/local/go/src/runtime/asm_amd64.s:1371 +0x1 fp=0xc0000407d8 sp=0xc0000407d0 pc=0x4a5761
created by testing.(*T).Run
	/usr/local/go/src/testing/testing.go:1238 +0x5d8

I debuged and found that this error exists after the systemstack call and before calling my closure(which means this sigsegv is not caused by my code), so I wonder if this is a bug of the race detector or this is my misuse of systemstack?

gdb result Output
(gdb) b xxx/xxx.Now
Breakpoint 1 at 0x674500: file xxx/time_linux_amd64.go, line 24.
(gdb) r
Starting program: xxx.test 
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffd09ee700 (LWP 18203)]
[New Thread 0x7fffcbfff700 (LWP 18204)]
[New Thread 0x7fffcb7fe700 (LWP 18205)]
[New Thread 0x7fffcaffd700 (LWP 18206)]
[New Thread 0x7fffca7fc700 (LWP 18207)]

Thread 1 "xxx.test" hit Breakpoint 1,xxx.Now (~r0=...)
at xxx/time_linux_amd64.go:24
24 func Now() time.Time {
(gdb) s
runtime.racefuncenter () at /usr/local/go/src/runtime/race_amd64.s:179
179 MOVQ callpc+0(FP), R11
(gdb) s
180 JMP racefuncenter<>(SB)
(gdb) s
racefuncenter () at /usr/local/go/src/runtime/race_amd64.s:185
185 MOVQ DX, R15 // save function entry context (for closures)
(gdb) s
187 MOVQ g(R12), R14
(gdb) s
188 MOVQ g_racectx(R14), RARG0 // goroutine context
(gdb) s
189 MOVQ R11, RARG1
(gdb) s
191 MOVQ $__tsan_func_enter(SB), AX
(gdb) p __tsan_func_enter
$3 = {<text variable, no debug info>} 0x418960 <__tsan_func_enter>
(gdb) s
193 CALL racecall<>(SB)
(gdb) p $rax
$4 = 4295008
(gdb) s
racecall () at /usr/local/go/src/runtime/race_amd64.s:403
403 MOVQ g(R12), R14
(gdb) s
404 MOVQ g_m(R14), R13
(gdb) s
406 MOVQ SP, R12 // callee-saved, preserved across the CALL
(gdb) s
407 MOVQ m_g0(R13), R10
(gdb) s
408 CMPQ R10, R14
(gdb) s
409 JE call // already on g0
(gdb) s
410 MOVQ (g_sched+gobuf_sp)(R10), SP
(gdb) s
racecall () at /usr/local/go/src/runtime/race_amd64.s:412
412 ANDQ $~15, SP // alignment for gcc ABI
(gdb) s
racecall () at /usr/local/go/src/runtime/race_amd64.s:413
413 CALL AX
(gdb) s
414 MOVQ R12, SP
(gdb) s
racecall () at /usr/local/go/src/runtime/race_amd64.s:415
415 RET
(gdb) s
racefuncenter () at /usr/local/go/src/runtime/race_amd64.s:194
194 MOVQ R15, DX // restore function entry context
(gdb) s
racefuncenter () at /usr/local/go/src/runtime/race_amd64.s:195
195 RET
(gdb) s
xxx.Now (~r0=...) at xxx/time_linux_amd64.go:25
25 var sec, mono int64
(gdb) s
26 var nsec int32
(gdb) s
27 systemstack(func() {
(gdb) s
runtime.systemstack () at /usr/local/go/src/runtime/asm_amd64.s:343
343 MOVQ fn+0(FP), DI // DI = fn
(gdb) s
345 MOVQ g(CX), AX // AX = g
(gdb) s
346 MOVQ g_m(AX), BX // BX = m
(gdb) s
348 CMPQ AX, m_gsignal(BX)
(gdb) s
349 JEQ noswitch
(gdb) s
351 MOVQ m_g0(BX), DX // DX = g0
(gdb) s
352 CMPQ AX, DX
(gdb) s
353 JEQ noswitch
(gdb) s
355 CMPQ AX, m_curg(BX)
(gdb) s
356 JNE bad
(gdb) s
361 MOVQ $runtime·systemstack_switch(SB), SI
(gdb) s
362 MOVQ SI, (g_sched+gobuf_pc)(AX)
(gdb) s
363 MOVQ SP, (g_sched+gobuf_sp)(AX)
(gdb) s
364 MOVQ AX, (g_sched+gobuf_g)(AX)
(gdb) s
365 MOVQ BP, (g_sched+gobuf_bp)(AX)
(gdb) s
368 MOVQ DX, g(CX)
(gdb) s
369 MOVQ (g_sched+gobuf_sp)(DX), BX
(gdb) s
371 SUBQ $8, BX
(gdb) s
372 MOVQ $runtime·mstart(SB), DX
(gdb) s
373 MOVQ DX, 0(BX)
(gdb) s
374 MOVQ BX, SP
(gdb) s
runtime.systemstack () at /usr/local/go/src/runtime/asm_amd64.s:377
377 MOVQ DI, DX
(gdb) s
378 MOVQ 0(DI), DI
(gdb) s
379 CALL DI
(gdb) s
xxx.Now.func1 () at xxx/time_linux_amd64.go:27
27 systemstack(func() {
(gdb) s
runtime.racefuncenter () at /usr/local/go/src/runtime/race_amd64.s:179
179 MOVQ callpc+0(FP), R11
(gdb) s
180 JMP racefuncenter<>(SB)
(gdb) s
racefuncenter () at /usr/local/go/src/runtime/race_amd64.s:185
185 MOVQ DX, R15 // save function entry context (for closures)
(gdb) s
187 MOVQ g(R12), R14
(gdb) s
188 MOVQ g_racectx(R14), RARG0 // goroutine context
(gdb) s
189 MOVQ R11, RARG1
(gdb) s
191 MOVQ $__tsan_func_enter(SB), AX
(gdb) p __tsan_func_enter
$7 = {<text variable, no debug info>} 0x418960 <__tsan_func_enter>
(gdb) s
193 CALL racecall<>(SB)
(gdb) p $rax
$8 = 4295008
(gdb) s
racecall () at /usr/local/go/src/runtime/race_amd64.s:403
403 MOVQ g(R12), R14
(gdb) s
404 MOVQ g_m(R14), R13
(gdb) s
406 MOVQ SP, R12 // callee-saved, preserved across the CALL
(gdb) s
407 MOVQ m_g0(R13), R10
(gdb) s
408 CMPQ R10, R14
(gdb) s
409 JE call // already on g0
(gdb) s
412 ANDQ $~15, SP // alignment for gcc ABI
(gdb) s
413 CALL AX
(gdb) p __tsan_func_enter
$9 = {<text variable, no debug info>} 0x418960 <__tsan_func_enter>
(gdb) p $rax
$10 = 4295008
(gdb) s

Thread 1 "timex.test" received signal SIGSEGV, Segmentation fault.
0x0000000000418975 in __tsan_func_enter ()
(gdb) bt
#0 0x0000000000418975 in __tsan_func_enter ()
#1 0x000000c000082a80 in ?? ()
#2 0x000000000085d760 in runtime.sched ()
#3 0x00007fffffffde00 in ?? ()
#4 0x00007fffffffddf0 in ?? ()
#5 0x00007fffffffdd90 in ?? ()
#6 0x00000000004a6f03 in racecall () at /usr/local/go/src/runtime/race_amd64.s:413
#7 0x00000000004a6e22 in racefuncenter () at /usr/local/go/src/runtime/race_amd64.s:193
#8 0x0000000000675554 in xxx/xxx.Now.func1 () at xxx/time_linux_amd64.go:27
#9 0x00000000004a39a6 in runtime.systemstack () at /usr/local/go/src/runtime/asm_amd64.s:379
#10 0x00000000004707c0 in ?? () at /usr/local/go/src/runtime/signal_unix.go:1035
#11 0x00000000004a37f4 in runtime.rt0_go () at /usr/local/go/src/runtime/asm_amd64.s:226
#12 0x0000000000000000 in ?? ()

What version of Go are you using (go version)?

$ go version
go version go1.16.3 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/purewhite/.cache/go-build"
GOENV="/home/purewhite/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/purewhite/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/home/purewhite/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="go1.16.3"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/xxx/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build1995173489=/tmp/go-build -gno-record-gcc-switches"
@seankhliao
Copy link
Contributor

@seankhliao seankhliao commented Apr 26, 2021

Unlike many projects, the Go project does not use GitHub Issues for general discussion or asking questions. GitHub Issues are used for tracking bugs and proposals only.

For questions please refer to https://github.com/golang/go/wiki/Questions

@seankhliao seankhliao closed this Apr 26, 2021
@PureWhiteWu PureWhiteWu changed the title question: How to do vdso calls in my own code? runtime: systemstack call caused SIGSEGV under race mode Apr 26, 2021
@PureWhiteWu
Copy link
Contributor Author

@PureWhiteWu PureWhiteWu commented Apr 26, 2021

@seankhliao hello, I've changed to title and I think this is a (maybe) bug report now. Could you please reopen the issue?

@davecheney davecheney reopened this Apr 26, 2021
@PureWhiteWu
Copy link
Contributor Author

@PureWhiteWu PureWhiteWu commented Apr 26, 2021

@davecheney Thanks very much!

@cherrymui
Copy link
Contributor

@cherrymui cherrymui commented Apr 26, 2021

I tried to link the runtime.systemstack

runtime.systemstack is a runtime internal function. It is not supposed to be called outside of the runtime. Your use case is not supported. Thanks.

@cherrymui cherrymui closed this Apr 26, 2021
@PureWhiteWu
Copy link
Contributor Author

@PureWhiteWu PureWhiteWu commented Apr 27, 2021

@cherrymui
Hello, thanks for your reply!
I know this is not supposed to be called outside of the runtime, but I just want to know if this is a bug of the race detector or a by-design behaviour.
If this is a by-design behaviour, than I will go and find another way to get over this.
Thanks again!

@ianlancetaylor
Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Apr 27, 2021

Calling systemstack outside of the runtime package is not supported. I don't know the exact reason why it fails, and finding out that exact reason is of purely academic interest. If that reason is fixed today, some new reason will appear tomorrow. We are never going to support calling systemstack outside of the runtime package, and that is by design.

Sorry this is not helpful, but I believe that it is accurate.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants