Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: Panic if newstack at runtime.acquireLockRank #40843

Closed
chainhelen opened this issue Aug 17, 2020 · 6 comments
Closed

runtime: Panic if newstack at runtime.acquireLockRank #40843

chainhelen opened this issue Aug 17, 2020 · 6 comments
Labels
Milestone

Comments

@chainhelen
Copy link
Contributor

@chainhelen chainhelen commented Aug 17, 2020

What version of Go are you using (go version)?

$ go version
go version go1.15 linux/386

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env

GO111MODULE=""
GOARCH="386"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOENV="/root/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="386"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/root/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/root/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/root/go.master_v2"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/root/go.master_v2/pkg/tool/linux_386"
GCCGO="gccgo"
GO386="sse2"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/root/go.master_v2/src/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m32 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build983115557=/tmp/go-build -gno-record-gcc-switches"

What did you do?

// cat main.go
package main

import "fmt"

func main() {
        fmt.Println("hello world!")
}
// Please notice it can't be reproducted if without -gcflags="all=-N -l".

go build -gcflags="all=-N -l" -o main main.go
./main

What did you expect to see?

No painc.

What did you see instead?

runtime: newstack at runtime.acquireLockRank+0x18 sp=0x888adac stack=[0x888a000, 0x888b000]
	morebuf={pc:0x807b8d8 sp:0x888adb0 lr:0x0}
	sched={pc:0x8052258 sp:0x888adac lr:0x0 ctxt:0x0}
runtime.casgstatus(0x88000e0, 0x2, 0x3)
	/root/go.master/src/runtime/proc.go:801 +0x58 fp=0x888ade0 sp=0x888adb0 pc=0x807b8d8
runtime.reentersyscall(0x80d68e5, 0x888ae0c)
	/root/go.master/src/runtime/proc.go:3027 +0x66 fp=0x888ae00 sp=0x888ade0 pc=0x80805a6
runtime.entersyscall()
	/root/go.master/src/runtime/proc.go:3076 +0x17 fp=0x888ae0c sp=0x888ae00 pc=0x80a1217
syscall.Syscall6(0x131, 0xffffff9c, 0x88a6000, 0x88a4000, 0x80, 0x0, 0x0, 0xa758010c, 0x0, 0x8069e02)
	/root/go.master/src/syscall/asm_linux_386.s:44 +0x5 fp=0x888ae10 sp=0x888ae0c pc=0x80d68e5
syscall.readlinkat(0xffffff9c, 0x80ff7ee, 0xe, 0x88a4000, 0x80, 0x80, 0x0, 0x0, 0x0)
	/root/go.master/src/syscall/zsyscall_linux_386.go:90 +0x123 fp=0x888ae6c sp=0x888ae10 pc=0x80d63b3
syscall.Readlink(0x80ff7ee, 0xe, 0x88a4000, 0x80, 0x80, 0x0, 0x0, 0x0)
	/root/go.master/src/syscall/syscall_linux.go:150 +0x75 fp=0x888aeac sp=0x888ae6c pc=0x80d5e35
os.Readlink(0x80ff7ee, 0xe, 0x0, 0x0, 0x0, 0x0)
	/root/go.master/src/os/file_unix.go:370 +0xba fp=0x888af18 sp=0x888aeac pc=0x80d990a
os.glob..func1(0x0, 0x0, 0x0, 0x0)
	/root/go.master/src/os/executable_procfs.go:29 +0x93 fp=0x888af5c sp=0x888af18 pc=0x80d9bf3
os.init()
	/root/go.master/src/os/executable_procfs.go:30 +0x198 fp=0x888af80 sp=0x888af5c pc=0x80d9e08
runtime.doInit(0x8160ae0)
	/root/go.master/src/runtime/proc.go:5559 +0x91 fp=0x888af98 sp=0x888af80 pc=0x8085e81
runtime.doInit(0x81607a0)
	/root/go.master/src/runtime/proc.go:5554 +0x5b fp=0x888afb0 sp=0x888af98 pc=0x8085e4b
runtime.doInit(0x815f530)
	/root/go.master/src/runtime/proc.go:5554 +0x5b fp=0x888afc8 sp=0x888afb0 pc=0x8085e4b
runtime.main()
	/root/go.master/src/runtime/proc.go:191 +0x185 fp=0x888aff0 sp=0x888afc8 pc=0x8079cc5
runtime.goexit()
	/root/go.master/src/runtime/asm_386.s:1333 +0x1 fp=0x888aff4 sp=0x888aff0 pc=0x80a3da1
fatal error: runtime: stack split at bad time

runtime stack:
runtime.throw(0x8103221, 0x20)
	/root/go.master/src/runtime/panic.go:1116 +0x6a fp=0xbff0c668 sp=0xbff0c654 pc=0x80779ba
runtime.newstack()
	/root/go.master/src/runtime/stack.go:965 +0xa90 fp=0xbff0c738 sp=0xbff0c668 pc=0x808dfc0
runtime.morestack()
	/root/go.master/src/runtime/asm_386.s:471 +0x7f fp=0xbff0c73c sp=0xbff0c738 pc=0x80a297f

goroutine 1 [running, locked to thread]:
runtime.casgstatus(0x88000e0, 0x2, 0x3)
	/root/go.master/src/runtime/proc.go:801 +0x58 fp=0x888ade0 sp=0x888adb0 pc=0x807b8d8
runtime.reentersyscall(0x80d68e5, 0x888ae0c)
	/root/go.master/src/runtime/proc.go:3027 +0x66 fp=0x888ae00 sp=0x888ade0 pc=0x80805a6
runtime.entersyscall()
	/root/go.master/src/runtime/proc.go:3076 +0x17 fp=0x888ae0c sp=0x888ae00 pc=0x80a1217
syscall.Syscall6(0x131, 0xffffff9c, 0x88a6000, 0x88a4000, 0x80, 0x0, 0x0, 0xa758010c, 0x0, 0x8069e02)
	/root/go.master/src/syscall/asm_linux_386.s:44 +0x5 fp=0x888ae10 sp=0x888ae0c pc=0x80d68e5
syscall.readlinkat(0xffffff9c, 0x80ff7ee, 0xe, 0x88a4000, 0x80, 0x80, 0x0, 0x0, 0x0)
	/root/go.master/src/syscall/zsyscall_linux_386.go:90 +0x123 fp=0x888ae6c sp=0x888ae10 pc=0x80d63b3
syscall.Readlink(0x80ff7ee, 0xe, 0x88a4000, 0x80, 0x80, 0x0, 0x0, 0x0)
	/root/go.master/src/syscall/syscall_linux.go:150 +0x75 fp=0x888aeac sp=0x888ae6c pc=0x80d5e35
os.Readlink(0x80ff7ee, 0xe, 0x0, 0x0, 0x0, 0x0)
	/root/go.master/src/os/file_unix.go:370 +0xba fp=0x888af18 sp=0x888aeac pc=0x80d990a
os.glob..func1(0x0, 0x0, 0x0, 0x0)
	/root/go.master/src/os/executable_procfs.go:29 +0x93 fp=0x888af5c sp=0x888af18 pc=0x80d9bf3
os.init()
	/root/go.master/src/os/executable_procfs.go:30 +0x198 fp=0x888af80 sp=0x888af5c pc=0x80d9e08
runtime.doInit(0x8160ae0)
	/root/go.master/src/runtime/proc.go:5559 +0x91 fp=0x888af98 sp=0x888af80 pc=0x8085e81
runtime.doInit(0x81607a0)
	/root/go.master/src/runtime/proc.go:5554 +0x5b fp=0x888afb0 sp=0x888af98 pc=0x8085e4b
runtime.doInit(0x815f530)
	/root/go.master/src/runtime/proc.go:5554 +0x5b fp=0x888afc8 sp=0x888afb0 pc=0x8085e4b
runtime.main()
	/root/go.master/src/runtime/proc.go:191 +0x185 fp=0x888aff0 sp=0x888afc8 pc=0x8079cc5
runtime.goexit()
	/root/go.master/src/runtime/asm_386.s:1333 +0x1 fp=0x888aff4 sp=0x888aff0 pc=0x80a3da1

goroutine 2 [force gc (idle)]:
runtime.gopark(0x81060c4, 0x81709f0, 0x8171411, 0x1)
	/root/go.master/src/runtime/proc.go:306 +0xaf fp=0x8828fc8 sp=0x8828fb4 pc=0x807a09f
runtime.goparkunlock(0x81709f0, 0x1411, 0x1)
	/root/go.master/src/runtime/proc.go:312 +0x45 fp=0x8828fdc sp=0x8828fc8 pc=0x807a145
runtime.forcegchelper()
	/root/go.master/src/runtime/proc.go:255 +0xc5 fp=0x8828ff0 sp=0x8828fdc pc=0x8079f65
runtime.goexit()
	/root/go.master/src/runtime/asm_386.s:1333 +0x1 fp=0x8828ff4 sp=0x8828ff0 pc=0x80a3da1
created by runtime.init.5
	/root/go.master/src/runtime/proc.go:243 +0x2b

goroutine 3 [GC sweep wait]:
runtime.gopark(0x81060c4, 0x8170b00, 0x806140c, 0x1)
	/root/go.master/src/runtime/proc.go:306 +0xaf fp=0x88297c4 sp=0x88297b0 pc=0x807a09f
runtime.goparkunlock(0x8170b00, 0x810140c, 0x1)
	/root/go.master/src/runtime/proc.go:312 +0x45 fp=0x88297d8 sp=0x88297c4 pc=0x807a145
runtime.bgsweep(0x881c040)
	/root/go.master/src/runtime/mgcsweep.go:163 +0x98 fp=0x88297e8 sp=0x88297d8 pc=0x8067ae8
runtime.goexit()
	/root/go.master/src/runtime/asm_386.s:1333 +0x1 fp=0x88297ec sp=0x88297e8 pc=0x80a3da1
created by runtime.gcenable
	/root/go.master/src/runtime/mgc.go:217 +0x4d

goroutine 4 [GC scavenge wait]:
runtime.gopark(0x81060c4, 0x8170aa0, 0x806140d, 0x1)
	/root/go.master/src/runtime/proc.go:306 +0xaf fp=0x8829f98 sp=0x8829f84 pc=0x807a09f
runtime.goparkunlock(0x8170aa0, 0x810140d, 0x1)
	/root/go.master/src/runtime/proc.go:312 +0x45 fp=0x8829fac sp=0x8829f98 pc=0x807a145
runtime.bgscavenge(0x881c040)
	/root/go.master/src/runtime/mgcscavenge.go:260 +0xc7 fp=0x8829fe8 sp=0x8829fac pc=0x8065fc7
runtime.goexit()
	/root/go.master/src/runtime/asm_386.s:1333 +0x1 fp=0x8829fec sp=0x8829fe8 pc=0x80a3da1
created by runtime.gcenable
	/root/go.master/src/runtime/mgc.go:218 +0x6b

Reason

I dive this issue then find this issue #38922 and this commit

The function named acquireLockRank at src/runtime/lockrank_on.go with go:nosplit , but at src/runtime/lockrank_off.go without go:nosplit. (GOEXPERIMENT="" in default )

Similarly, function lockWithRank/unlockWithRank/releaseLockRank/lockWithRankMayAcquire.

@chainhelen chainhelen changed the title runtime: Panic if newstack at runtime.acquireLockRank with `gcflags="all=-N -l"` runtime: Panic if newstack at runtime.acquireLockRank Aug 17, 2020
chainhelen added a commit to chainhelen/go that referenced this issue Aug 17, 2020
The function named `acquireLockRank` at `src/runtime/lockrank_on.go`
with `go:nosplit`, but at `src/runtime/lockrank_off.go` without
`go:nosplit`. GOEXPERIMENT="" in default, process will panic if
newstack at runtime.acquireLockRank with `gcflags="all=-N -l"`.
Similarly, function `lockWithRank`, `unlockWithRank`, `releaseLockRank`
and `lockWithRankMayAcquire`.

Fixes golang#40843
@gopherbot
Copy link

@gopherbot gopherbot commented Aug 17, 2020

Change https://golang.org/cl/248878 mentions this issue: runtime: Fix panic if newstack at runtime.acquireLockRank

@prattmic
Copy link
Member

@prattmic prattmic commented Aug 17, 2020

@prattmic prattmic added this to the Go1.15.1 milestone Aug 17, 2020
@ALTree ALTree modified the milestones: Go1.15.1, Go1.16 Aug 17, 2020
@ALTree
Copy link
Member

@ALTree ALTree commented Aug 17, 2020

@gopherbot, please backport to Go 1.15, as requested by prattmic.

@gopherbot
Copy link

@gopherbot gopherbot commented Aug 17, 2020

Backport issue(s) opened: #40845 (for 1.15).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

@chainhelen
Copy link
Contributor Author

@chainhelen chainhelen commented Aug 17, 2020

I have commited a pull request #40844 for solving this issue, please check it . Thx.

chainhelen added a commit to chainhelen/go that referenced this issue Aug 18, 2020
Process may crash becaues acquireLockRank and releaseLockRank may
be called in nosplit context. With optimization s and inlining
disabled, these functions won't get inlined or have their morestack
calls eliminated.
Nosplit is not strictly required for lockWithRank, unlockWithRank
and lockWithRankMayAcquire, just keep consistency with lockrank_on.go
here.

Fixes golang#40843
chainhelen added a commit to chainhelen/go that referenced this issue Aug 18, 2020
Process may crash becaues acquireLockRank and releaseLockRank may
be called in nosplit context. With optimization s and inlining
disabled, these functions won't get inlined or have their morestack
calls eliminated.
Nosplit is not strictly required for lockWithRank, unlockWithRank
and lockWithRankMayAcquire, just keep consistency with lockrank_on.go
here.

Fixes golang#40843
chainhelen added a commit to chainhelen/go that referenced this issue Aug 21, 2020
Process may crash becaues acquireLockRank and releaseLockRank may
be called in nosplit context. With optimizations and inlining
disabled, these functions won't get inlined or have their morestack
calls eliminated.
Nosplit is not strictly required for lockWithRank, unlockWithRank
and lockWithRankMayAcquire, just keep consistency with lockrank_on.go
here.

Fixes golang#40843
chainhelen added a commit to chainhelen/go that referenced this issue Aug 21, 2020
Process may crash becaues acquireLockRank and releaseLockRank may
be called in nosplit context. With optimizations and inlining
disabled, these functions won't get inlined or have their morestack
calls eliminated.
Nosplit is not strictly required for lockWithRank, unlockWithRank
and lockWithRankMayAcquire, just keep consistency with lockrank_on.go
here.

Fixes golang#40843
@gopherbot gopherbot closed this in b246c0e Sep 1, 2020
@gopherbot
Copy link

@gopherbot gopherbot commented Sep 1, 2020

Change https://golang.org/cl/252339 mentions this issue: [release-branch.go1.15] runtime: fix panic if newstack at runtime.acquireLockRank

gopherbot pushed a commit that referenced this issue Sep 2, 2020
…uireLockRank

Process may crash becaues acquireLockRank and releaseLockRank may
be called in nosplit context. With optimizations and inlining
disabled, these functions won't get inlined or have their morestack
calls eliminated.
Nosplit is not strictly required for lockWithRank, unlockWithRank
and lockWithRankMayAcquire, just keep consistency with lockrank_on.go
here.

Updates #40843.
Fixes #40845.

Change-Id: I5824119f98a1da66d767cdb9a60dffe768f13c81
GitHub-Last-Rev: 38fd3cc
GitHub-Pull-Request: #40844
Reviewed-on: https://go-review.googlesource.com/c/go/+/248878
Reviewed-by: Dan Scales <danscales@google.com>
Run-TryBot: Emmanuel Odeke <emm.odeke@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
(cherry picked from commit b246c0e)
Reviewed-on: https://go-review.googlesource.com/c/go/+/252339
Run-TryBot: Dmitri Shuralyov <dmitshur@golang.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

4 participants
You can’t perform that action at this time.