Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: panic while TestASAN with "stack growth after fork" #50391

Closed
mengzhuo opened this issue Dec 30, 2021 · 3 comments
Closed

runtime: panic while TestASAN with "stack growth after fork" #50391

mengzhuo opened this issue Dec 30, 2021 · 3 comments
Labels
NeedsFix release-blocker
Milestone

Comments

@mengzhuo
Copy link
Contributor

@mengzhuo mengzhuo commented Dec 30, 2021

What version of Go are you using (go version)?

$ go version
go version devel go1.18-91e782106e Wed Dec 29 04:10:07 2021 +0000 linux/amd64

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/root/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/root/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/root/godev"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/root/godev/pkg/tool/linux_amd64"
GOVCS=""
GOVERSION="devel go1.18-91e782106e Wed Dec 29 04:10:07 2021 +0000"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build2639948742=/tmp/go-build -gno-record-gcc-switches"
GOROOT/bin/go version: go version devel go1.18-91e782106e Wed Dec 29 04:10:07 2021 +0000 linux/amd64
GOROOT/bin/go tool compile -V: compile version devel go1.18-91e782106e Wed Dec 29 04:10:07 2021 +0000
uname -sr: Linux 5.10.0-9-amd64
Distributor ID:	Debian
Description:	Debian GNU/Linux 11 (bullseye)
Release:	11
Codename:	bullseye
/lib/x86_64-linux-gnu/libc.so.6: GNU C Library (Debian GLIBC 2.31-13+deb11u2) stable release version 2.31.
gdb --version: GNU gdb (Debian 10.1-1.7) 10.1.90.20210103-git

What did you do?

cd misc/cgo/testsanitizers/ && go test -asan -v

What did you expect to see?

PASSED

What did you see instead?

fatal error: stack growth after fork

runtime stack:
runtime.throw({0x5de782?, 0x200000008?})
/root/godev/src/runtime/panic.go:992 +0x71
runtime.newstack()
/root/godev/src/runtime/stack.go:964 +0xc07
runtime.morestack()
/root/godev/src/runtime/asm_amd64.s:547 +0x8b

goroutine 6 [running]:
runtime.asanwrite(0x10c0001091c0?, 0x20?)
/root/godev/src/runtime/asan.go:35 +0x5c fp=0x10c000109140 sp=0x10c000109138 pc=0x405a1c
runtime.sigaction(0x1, 0x10c0001091c0, 0x0)
/root/godev/src/runtime/cgo_sigaction.go:31 +0x32 fp=0x10c0001091a8 sp=0x10c000109140 pc=0x405db2
runtime.setsig(0x6d95c8?, 0x0?)
/root/godev/src/runtime/os_linux.go:468 +0x76 fp=0x10c0001091f0 sp=0x10c0001091a8 pc=0x4342f6
runtime.clearSignalHandlers()
/root/godev/src/runtime/signal_unix.go:261 +0x35 fp=0x10c000109218 sp=0x10c0001091f0 pc=0x44bd95
syscall.runtime_AfterForkInChild()
/root/godev/src/runtime/proc.go:4186 +0x1a fp=0x10c000109250 sp=0x10c000109218 pc=0x4641ba
syscall.forkAndExecInChild1(0x10c000018210, {0x10c00000e2a0, 0x4, 0x80000?}, {0x10c00012c3c0, 0x16, 0x8?}, 0x0, 0x0, 0x10c0001097a0, ...)
/root/godev/src/syscall/exec_linux.go:296 +0x76a fp=0x10c000109488 sp=0x10c000109250 pc=0x4748ea
syscall.forkAndExecInChild(0x10c0000535a8?, {0x10c00000e2a0?, 0x7f3ca2c35ed8?, 0x340?}, {0x10c00012c3c0?, 0x440ec5?, 0x10c00010b0e0?}, 0x300000002?, 0x10c00010b0e0?, 0x10c0001097a0, ...)
/root/godev/src/syscall/exec_linux.go:84 +0x65 fp=0x10c000109548 sp=0x10c000109488 pc=0x473fc5
syscall.forkExec({0x10c000018180, 0x14}, {0x10c00007e540, 0x3, 0x3}, 0x10c000012680?)
/root/godev/src/syscall/exec_unix.go:209 +0x51c fp=0x10c0001096c8 sp=0x10c000109548 pc=0x476abc
syscall.StartProcess(...)
/root/godev/src/syscall/exec_unix.go:255
os.startProcess({0x10c000018180, 0x14}, {0x10c00007e540, 0x3, 0x3}, 0x10c000109a90)
/root/godev/src/os/exec_posix.go:54 +0x5cd fp=0x10c0001097f8 sp=0x10c0001096c8 pc=0x49c7cd
os.StartProcess({0x10c000018180, 0x14}, {0x10c00007e540, 0x3, 0x3}, 0x10c0000181b0?)
/root/godev/src/os/exec.go:109 +0x5a fp=0x10c000109840 sp=0x10c0001097f8 pc=0x49c11a
os/exec.(*Cmd).Start(0x10c000012340)
/root/godev/src/os/exec/exec.go:422 +0xa1d fp=0x10c000109ae8 sp=0x10c000109840 pc=0x51a0fd
os/exec.(*Cmd).Run(0x518265?)
/root/godev/src/os/exec/exec.go:338 +0x1e fp=0x10c000109b08 sp=0x10c000109ae8 pc=0x51969e
os/exec.(*Cmd).Output(0x10c000012340)
/root/godev/src/os/exec/exec.go:546 +0x245 fp=0x10c000109b98 sp=0x10c000109b08 pc=0x51b405
misc/cgo/testsanitizers_test.goEnv.func1()
/root/godev/misc/cgo/testsanitizers/cc_test.go:63 +0x6f fp=0x10c000109c20 sp=0x10c000109b98 pc=0x594fcf
sync.(*Once).doSlow(0x6ef640, 0x5e6230)
/root/godev/src/sync/once.go:68 +0xcf fp=0x10c000109c80 sp=0x10c000109c20 pc=0x47118f
sync.(*Once).Do(...)
/root/godev/src/sync/once.go:59
misc/cgo/testsanitizers_test.goEnv({0x5d9aeb, 0x4})
/root/godev/misc/cgo/testsanitizers/cc_test.go:61 +0x3f fp=0x10c000109cd8 sp=0x10c000109c80 pc=0x59517f
misc/cgo/testsanitizers_test.TestASAN(0x10c00010af00)
/root/godev/misc/cgo/testsanitizers/asan_test.go:13 +0x45 fp=0x10c000109f38 sp=0x10c000109cd8 pc=0x593da5
testing.tRunner(0x10c00010af00, 0x5e6208)
/root/godev/src/testing/testing.go:1440 +0x19e fp=0x10c000109fc0 sp=0x10c000109f38 pc=0x507abe
testing.(*T).Run.func1()
/root/godev/src/testing/testing.go:1487 +0x2a fp=0x10c000109fe0 sp=0x10c000109fc0 pc=0x50938a
runtime.goexit()
/root/godev/src/runtime/asm_amd64.s:1571 +0x1 fp=0x10c000109fe8 sp=0x10c000109fe0 pc=0x468881
created by testing.(*T).Run
/root/godev/src/testing/testing.go:1487 +0x777

goroutine 1 [chan receive]:
testing.(*T).Run(0x10c00010ad20, {0x5da571, 0x8}, 0x5e6208)
/root/godev/src/testing/testing.go:1488 +0x7a7
testing.runTests.func1(0x7f3ca2c35f60?)
/root/godev/src/testing/testing.go:1840 +0x95
testing.tRunner(0x10c00010ad20, 0x10c000107c18)
/root/godev/src/testing/testing.go:1440 +0x19e
testing.runTests(0x10c0001182a0?, {0x6d5400, 0x4, 0x4}, {0x10c000107d10?, 0x4ed6c8?, 0x0?})
/root/godev/src/testing/testing.go:1838 +0x83f
testing.(*M).Run(0x10c0001182a0)
/root/godev/src/testing/testing.go:1720 +0xb37
main.main()
_testmain.go:53 +0x2fd
fatal: morestack on g0
panic during panic
SIGTRAP: trace trap
PC=0x4686e2 m=0 sigcode=128

goroutine 0 [idle]:
runtime.abort()
/root/godev/src/runtime/asm_amd64.s:1036 +0x2
runtime.morestack()
/root/godev/src/runtime/asm_amd64.s:515 +0x25

goroutine 6 [running]:
runtime.asanwrite(0x10c0001091c0?, 0x20?)
/root/godev/src/runtime/asan.go:35 +0x5c fp=0x10c000109140 sp=0x10c000109138 pc=0x405a1c
runtime.sigaction(0x1, 0x10c0001091c0, 0x0)
/root/godev/src/runtime/cgo_sigaction.go:31 +0x32 fp=0x10c0001091a8 sp=0x10c000109140 pc=0x405db2
runtime.setsig(0x6d95c8?, 0x0?)
/root/godev/src/runtime/os_linux.go:468 +0x76 fp=0x10c0001091f0 sp=0x10c0001091a8 pc=0x4342f6
runtime.clearSignalHandlers()
/root/godev/src/runtime/signal_unix.go:261 +0x35 fp=0x10c000109218 sp=0x10c0001091f0 pc=0x44bd95
syscall.runtime_AfterForkInChild()
/root/godev/src/runtime/proc.go:4186 +0x1a fp=0x10c000109250 sp=0x10c000109218 pc=0x4641ba
syscall.forkAndExecInChild1(0x10c000018210, {0x10c00000e2a0, 0x4, 0x80000?}, {0x10c00012c3c0, 0x45817b, 0x2?}, 0x5de48b, 0x17, 0x10c0001097a0, ...)
/root/godev/src/syscall/exec_linux.go:217 +0x3ed fp=0x10c000109488 sp=0x10c000109250 pc=0x47456d
syscall.forkAndExecInChild(0x10c0000535a8?, {0x10c00000e2a0?, 0x7f3ca2c35ed8?, 0x340?}, {0x10c00012c3c0?, 0x440ec5?, 0x10c00010b0e0?}, 0x300000002?, 0x10c00010b0e0?, 0x71ead0, ...)
/root/godev/src/syscall/exec_linux.go:86 +0x98 fp=0x10c000109548 sp=0x10c000109488 pc=0x473ff8
syscall.forkExec({0x10c000018180, 0x14}, {0x10c00007e540, 0x3, 0x3}, 0x10c000012680?)
/root/godev/src/syscall/exec_unix.go:216 +0x53d fp=0x10c0001096c8 sp=0x10c000109548 pc=0x476add
syscall.StartProcess(...)
/root/godev/src/syscall/exec_unix.go:255
os.startProcess({0x10c000018180, 0x14}, {0x10c00007e540, 0x3, 0x3}, 0x10c000109a90)
/root/godev/src/os/exec_posix.go:54 +0x5cd fp=0x10c0001097f8 sp=0x10c0001096c8 pc=0x49c7cd
os.StartProcess({0x10c000018180, 0x14}, {0x10c00007e540, 0x3, 0x3}, 0x10c0000181b0?)
/root/godev/src/os/exec.go:109 +0x5a fp=0x10c000109840 sp=0x10c0001097f8 pc=0x49c11a
os/exec.(*Cmd).Start(0x10c000012340)
/root/godev/src/os/exec/exec.go:422 +0xa1d fp=0x10c000109ae8 sp=0x10c000109840 pc=0x51a0fd
os/exec.(*Cmd).Run(0x518265?)
/root/godev/src/os/exec/exec.go:338 +0x1e fp=0x10c000109b08 sp=0x10c000109ae8 pc=0x51969e
os/exec.(*Cmd).Output(0x10c000012340)
/root/godev/src/os/exec/exec.go:546 +0x245 fp=0x10c000109b98 sp=0x10c000109b08 pc=0x51b405
misc/cgo/testsanitizers_test.goEnv.func1()
/root/godev/misc/cgo/testsanitizers/cc_test.go:63 +0x6f fp=0x10c000109c20 sp=0x10c000109b98 pc=0x594fcf
sync.(*Once).doSlow(0x6ef640, 0x5e6230)
/root/godev/src/sync/once.go:68 +0xcf fp=0x10c000109c80 sp=0x10c000109c20 pc=0x47118f
sync.(*Once).Do(...)
/root/godev/src/sync/once.go:59
misc/cgo/testsanitizers_test.goEnv({0x5d9aeb, 0x4})
/root/godev/misc/cgo/testsanitizers/cc_test.go:61 +0x3f fp=0x10c000109cd8 sp=0x10c000109c80 pc=0x59517f
misc/cgo/testsanitizers_test.TestASAN(0x10c00010af00)
/root/godev/misc/cgo/testsanitizers/asan_test.go:13 +0x45 fp=0x10c000109f38 sp=0x10c000109cd8 pc=0x593da5
testing.tRunner(0x10c00010af00, 0x5e6208)
/root/godev/src/testing/testing.go:1440 +0x19e fp=0x10c000109fc0 sp=0x10c000109f38 pc=0x507abe
testing.(*T).Run.func1()
/root/godev/src/testing/testing.go:1487 +0x2a fp=0x10c000109fe0 sp=0x10c000109fc0 pc=0x50938a
runtime.goexit()
/root/godev/src/runtime/asm_amd64.s:1571 +0x1 fp=0x10c000109fe8 sp=0x10c000109fe0 pc=0x468881
created by testing.(*T).Run
/root/godev/src/testing/testing.go:1487 +0x777

goroutine 1 [chan receive]:
testing.(*T).Run(0x10c00010ad20, {0x5da571, 0x8}, 0x5e6208)
/root/godev/src/testing/testing.go:1488 +0x7a7
testing.runTests.func1(0x7f3ca2c35f60?)
/root/godev/src/testing/testing.go:1840 +0x95
testing.tRunner(0x10c00010ad20, 0x10c000107c18)
/root/godev/src/testing/testing.go:1440 +0x19e
testing.runTests(0x10c0001182a0?, {0x6d5400, 0x4, 0x4}, {0x10c000107d10?, 0x4ed6c8?, 0x0?})
/root/godev/src/testing/testing.go:1838 +0x83f
testing.(*M).Run(0x10c0001182a0)
/root/godev/src/testing/testing.go:1720 +0xb37
main.main()
_testmain.go:53 +0x2fd

rax 0x17
rbx 0x5de48b
rcx 0x46a095
rdx 0x17
rdi 0x2
rsi 0x5de48b
rbp 0x10c0001096b8
rsp 0x10c000109530
r8 0x0
r9 0x0
r10 0x8
r11 0x246
r12 0x47456d
r13 0x6f00c0
r14 0x6efb20
r15 0xffffffffffffffff
rip 0x4686e2
rflags 0x206
cs 0x33
fs 0x0
gs 0x0

kindly cc @zhangfannie @cherrymui

@zhangfannie
Copy link
Contributor

@zhangfannie zhangfannie commented Dec 30, 2021

@mengzhuo Thank you for reporting this issue. The reason is that asanwrite/asanread function should be NOSPLIT. The fixed patch will be submitted soon. Thank you.

@gopherbot
Copy link

@gopherbot gopherbot commented Dec 30, 2021

Change https://golang.org/cl/374398 mentions this issue: src/runtime: mark asanread and asanwrite functions as NOSPLIT

@ALTree
Copy link
Member

@ALTree ALTree commented Dec 30, 2021

Putting this in the 1.18 milestone because it looks like -asan will be shipped in 1.18, feel free to re-assess if this is not right.

@ALTree ALTree added this to the Go1.18 milestone Dec 30, 2021
@ALTree ALTree added release-blocker NeedsFix labels Dec 30, 2021
jproberts pushed a commit to jproberts/go that referenced this issue Jun 21, 2022
The asan runtime functions may run on stacks that cannot grow, and
they do not have large local variables, so it is safe to mark them
as NOSPLIT.

Add test case.

Fixes golang#50391

Change-Id: Iadcbf1ae0c837d9b64da5be208c7f424e6ba11de
Reviewed-on: https://go-review.googlesource.com/c/go/+/374398
Trust: Emmanuel Odeke <emmanuel@orijtech.com>
Trust: Fannie Zhang <Fannie.Zhang@arm.com>
Reviewed-by: Cherry Mui <cherryyz@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
NeedsFix release-blocker
Projects
None yet
Development

No branches or pull requests

4 participants