Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

runtime: "panic during panic" on plan9 builders #49684

Open
bcmills opened this issue Nov 19, 2021 · 9 comments
Open

runtime: "panic during panic" on plan9 builders #49684

bcmills opened this issue Nov 19, 2021 · 9 comments
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Plan9
Milestone
Backlog

Comments

@bcmills
Copy link
Contributor

bcmills commented Nov 19, 2021 •
edited
Loading 

#!watchflakes
post <- builder ~ `^plan9.*` && `panic during panic`

plan9-386-0intro at e8cda0a6c925668972ada40602ada08468fa90dc

:: Running /tmp/workdir-gnot/go/src/make.rc with args ["/tmp/workdir-gnot/go/src/make.rc"] and env ["home=/usr/glenda" "path=.\x00/bin" "*=" "type=host-plan9-386-0intro" "rcname=gobuildlet" "pid=169" "fn#term%=fn term% {$*}\n" "fn#newworkdir=fn newworkdir {if(test -d $workdir){for(i in `{du -a $workdir|awk '{print $2}'})chmod u+w $i;rm -rf $workdir};mkdir $workdir}\n" "status=" "fn#cd=fn cd {builtin cd $* && awd}\n" "GOROOT_BOOTSTRAP=/tmp/workdir-gnot/go1.4" "workdir=/tmp/workdir-gnot" "objtype=386" "fn#sigexit=" "cflag=" "0=/bin/gobuildlet" "WORKDIR=/tmp/workdir-gnot" "GO_BUILDER_NAME=plan9-386-0intro" "GOBIN=" "GOROOT_BOOTSTRAP=/usr/go-plan9-386-bootstrap"] in dir /tmp/workdir-gnot/go/src

Building Go cmd/dist using /usr/go-plan9-386-bootstrap
Building Go toolchain1 using /usr/go-plan9-386-bootstrap.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
Building Go toolchain3 using go_bootstrap and Go toolchain2.
# runtime
fatal error: unexpected signal during runtime execution
[signal sys: trap: fault read code=0x0 addr=0x0 pc=0x57001]

runtime stack:
fatal error: unexpected signal during runtime execution
panic during panic
[signal sys: trap: fault read code=0x0 addr=0x0 pc=0x4dc6a]

runtime stack:
fatal error: unexpected signal during runtime execution
stack trace unavailable
go tool dist: FAILED: /tmp/workdir-gnot/go/pkg/tool/plan9_386/go_bootstrap install -gcflags=all= -ldflags=all= -a -i cmd/asm cmd/cgo cmd/compile cmd/link: exit status: 'go_bootstrap 293035: 2'

greplogs --dashboard -md -l -e '(?ms)plan9-.*^panic during panic' --since=2021-01-01

2021-11-19T00:05:59-e8cda0a/plan9-386-0intro
2021-11-11T20:20:50-46b2fc0/plan9-arm
2021-09-07T19:39:04-d92101f/plan9-arm
2021-09-02T16:26:26-08588e6/plan9-arm
2021-08-24T22:10:11-5d863f8/plan9-arm
2021-04-29T13:24:00-858fdbe-eb3fe28/plan9-arm
2021-04-26T18:54:39-0d1280c/plan9-arm
2021-02-23T06:01:29-e521498/plan9-arm

@0intro, @millerresearch, @fhs: is this possibly a duplicate of #35456 and/or #22227?
@bcmills bcmills added OS-Plan9 NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Nov 19, 2021
@bcmills bcmills added this to the Backlog milestone Nov 19, 2021
@bcmills bcmills mentioned this issue Nov 19, 2021
Open
@millerresearch
Copy link
Contributor

These all seem to be glitches deep in the internals of the runtime, all different except that fatal error: bad g->status in ready occurs in two of them.

I have no theories.

@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Jul 7, 2022
@bcmills
Copy link
Contributor Author

bcmills commented Dec 7, 2022

greplogs -l -e '(?ms)plan9-.*^panic during panic' --since=2022-01-01
2022-11-09T17:33:07-7a92c4f/plan9-arm
2022-09-30T05:48:38-d7dbe01/plan9-arm
2022-08-30T21:30:15-248c34b-bd56cb9/plan9-386-0intro
2022-08-29T20:04:56-dbd0ce8/plan9-386-0intro
2022-08-15T18:40:54-1d66144/plan9-arm
2022-07-08T12:57:49-1dfab61-14abe8a/plan9-386-0intro
2022-03-27T18:38:56-018b78c/plan9-arm
2022-03-24T09:17:34-0ed31eb/plan9-arm
2022-03-21T19:10:35-6be52ab/plan9-386-0intro
2022-02-23T20:51:28-e94f7df/plan9-386-0intro
2022-01-12T01:49:33-d55c255-3d3f5d9/plan9-arm

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `^plan9.*` && `panic during panic`
2022-12-13 14:05 plan9-arm go@61e2b8ec os/exec.TestExtraFilesRace (log) 
runtime: signal received on thread not created by Go.
runtime: g 111: unexpected return pc for syscall.Syscall6 called from 0x27272030
stack: frame={sp:0x921fd58, fp:0x921fdec} stack=[0x9200000,0x9220000)
0x0921fcd8:  0x00000000  0x0006cc88 <runtime.exitsyscall+0x00000144>  0x00572fb8  0x00000000 
0x0921fce8:  0x00000000  0x00000001  0x00000000  0x00000000 
0x0921fcf8:  0x00000000  0x00000000  0x00000000  0x0007f880 <syscall.Await+0x00000050> 
0x0921fd08:  0x0000002f  0x0921fd4c  0x00000200  0x00000000 
0x0921fd18:  0x00000010  0x00000000  0x0006cb98 <runtime.exitsyscall+0x00000054>  0x09032600 
0x0921fd28:  0x00000002  0x0921fd4c  0x0007ddec <syscall.startProcess.func1+0x000001ec>  0x0921fd4c 
0x0921fd38:  0x00000200  0x00000200  0x091ce318  0x0006cb40 <runtime.entersyscall+0x00000018> 
...
fatal error: unknown caller pc

runtime stack:
runtime.throw({0x34f84f, 0x11})
	/boot/workdir/go/src/runtime/panic.go:1047 +0x4c fp=0x3ffff884 sp=0x3ffff870 pc=0x3c564
runtime.gentraceback(0xffffffff, 0xffffffff, 0x0, 0x91ce2d0, 0x0, 0x0, 0x7fffffff, 0x3ffffc14, 0x0, 0x0)
	/boot/workdir/go/src/runtime/traceback.go:270 +0x18d0 fp=0x3ffffac8 sp=0x3ffff884 pc=0x65460
runtime.scanstack(0x91ce2d0, 0x9031c5c)
	/boot/workdir/go/src/runtime/mgcmark.go:804 +0x1fc fp=0x3ffffc20 sp=0x3ffffac8 pc=0x20184
runtime.markroot.func1()
...

runtime.gopark(0x368f98, 0x92c2130, 0xe, 0x17, 0x2)
	/boot/workdir/go/src/runtime/proc.go:381 +0x100 fp=0x91aae8c sp=0x91aae78 pc=0x3f474
runtime.chanrecv(0x92c2100, 0x91aaf68, 0x1)
	/boot/workdir/go/src/runtime/chan.go:583 +0x530 fp=0x91aaed8 sp=0x91aae8c pc=0x5e04
runtime.chanrecv1(0x92c2100, 0x91aaf68)
	/boot/workdir/go/src/runtime/chan.go:442 +0x20 fp=0x91aaeec sp=0x91aaed8 pc=0x58a4
os/exec_test.TestExtraFilesRace(0x9113c20)
	/boot/workdir/go/src/os/exec/exec_test.go:849 +0x530 fp=0x91aaf98 sp=0x91aaeec pc=0x2e2960
testing.tRunner(0x9113c20, 0x368e68)

— watchflakes

@rsc rsc mentioned this issue Mar 15, 2023
Closed
@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `^plan9.*` && `panic during panic`
2023-09-25 14:43 plan9-arm go@de3bae19 (log) 
fatal error: unexpected signal during runtime execution
fatal error: unexpected signal during runtime execution
panic during panic
[signal sys: trap: fault read code=0x0 addr=0x0 pc=0x3f554]

runtime stack:
runtime.throw({0x61f923, 0x2a})
	/tmp/genbootstrap-3479644045/go-plan9-arm-bootstrap/src/runtime/panic.go:1047 +0x4c fp=0x3ffffbdc sp=0x3ffffbc8 pc=0x3aca0
runtime.sigpanic()
	/tmp/genbootstrap-3479644045/go-plan9-arm-bootstrap/src/runtime/os_plan9.go:80 +0x4e8 fp=0x3ffffc1c sp=0x3ffffbdc pc=0x36aa4
runtime.preemptall(...)
	/tmp/genbootstrap-3479644045/go-plan9-arm-bootstrap/src/runtime/proc.go:5531
runtime.freezetheworld()
	/tmp/genbootstrap-3479644045/go-plan9-arm-bootstrap/src/runtime/proc.go:916 +0x170 fp=0x3ffffc30 sp=0x3ffffc20 pc=0x3f554
runtime.startpanic_m()
	/tmp/genbootstrap-3479644045/go-plan9-arm-bootstrap/src/runtime/panic.go:1220 +0x15c fp=0x3ffffc44 sp=0x3ffffc30 pc=0x3b2d0
runtime.fatalthrow.func1()
	/tmp/genbootstrap-3479644045/go-plan9-arm-bootstrap/src/runtime/panic.go:1125 +0x2c fp=0x3ffffc64 sp=0x3ffffc44 pc=0x3afc8
runtime.fatalthrow(0x2)
	/tmp/genbootstrap-3479644045/go-plan9-arm-bootstrap/src/runtime/panic.go:1120 +0x64 fp=0x3ffffc80 sp=0x3ffffc64 pc=0x3af84
runtime.throw({0x61f923, 0x2a})
	/tmp/genbootstrap-3479644045/go-plan9-arm-bootstrap/src/runtime/panic.go:1047 +0x4c fp=0x3ffffc94 sp=0x3ffffc80 pc=0x3aca0
runtime.sigpanic()
	/tmp/genbootstrap-3479644045/go-plan9-arm-bootstrap/src/runtime/os_plan9.go:80 +0x4e8 fp=0x3ffffcd4 sp=0x3ffffc94 pc=0x36aa4
runtime.forEachP(0x6565d0)
	/tmp/genbootstrap-3479644045/go-plan9-arm-bootstrap/src/runtime/proc.go:1673 +0xe4 fp=0x3ffffd10 sp=0x3ffffcd8 pc=0x41094
runtime.gcMarkDone.func1()
	/tmp/genbootstrap-3479644045/go-plan9-arm-bootstrap/src/runtime/mgc.go:814 +0x4c fp=0x3ffffd24 sp=0x3ffffd10 pc=0x663e4
runtime.systemstack()
	/tmp/genbootstrap-3479644045/go-plan9-arm-bootstrap/src/runtime/asm_arm.s:317 +0x60 fp=0x3ffffd28 sp=0x3ffffd24 pc=0x69950

— watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `^plan9.*` && `panic during panic`
2023-09-22 21:51 plan9-arm oauth2@14b275c9 go@5152be39 (log) 
runtime: program exceeds 0-thread limit
fatal error: thread exhaustion

runtime stack:
runtime.throw({0x670c18, 0x11})
	runtime/panic.go:1018 +0x4c fp=0x3ffffd74 sp=0x3ffffd60 pc=0x41354
runtime.checkmcount()
	runtime/proc.go:805 +0xd4 fp=0x3ffffd90 sp=0x3ffffd74 pc=0x45874
runtime.mReserveID()
	runtime/proc.go:821 +0x74 fp=0x3ffffda4 sp=0x3ffffd90 pc=0x4591c
...
runtime.retake(0x1788ab41d31b6a80)
	runtime/proc.go:5745 +0x2f8 fp=0x3ffffe20 sp=0x3ffffdf8 pc=0x513a4
runtime.sysmon()
	runtime/proc.go:5653 +0x5d0 fp=0x3ffffe8c sp=0x3ffffe20 pc=0x50eb8
runtime.mstart1()
	runtime/proc.go:1601 +0x7c fp=0x3ffffe9c sp=0x3ffffe8c pc=0x47604
runtime.mstart0()
	runtime/proc.go:1558 +0x64 fp=0x3ffffeac sp=0x3ffffe9c pc=0x47578
runtime.mstart()
	runtime/asm_arm.s:210 +0x8 fp=0x3ffffeb0 sp=0x3ffffeac pc=0x754d4

— watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `^plan9.*` && `panic during panic`
2023-11-21 20:13 plan9-arm go@f7b27790 (log) 
fatal error: unexpected signal during runtime execution
fatal error: unexpected signal during runtime execution
panic during panic
[signal sys: trap: fault read code=0x0 addr=0x0 pc=0x45ed4]

runtime stack:
runtime.throw({0x6947cb, 0x2a})
	runtime/panic.go:1023 +0x4c fp=0x3ffffa80 sp=0x3ffffa6c pc=0x41338
runtime.sigpanic()
	runtime/os_plan9.go:75 +0x4d0 fp=0x3ffffac0 sp=0x3ffffa80 pc=0x3d5c0
...
	os/exec/exec.go:577 +0x50 fp=0x1263fb0 sp=0x1263f74 pc=0x126e88
os/exec.(*Cmd).Start.func2(0x137e150)
	os/exec/exec.go:724 +0x30 fp=0x1263fe4 sp=0x1263fb0 pc=0x127ac0
os/exec.(*Cmd).Start.gowrap1()
	os/exec/exec.go:736 +0x30 fp=0x1263fec sp=0x1263fe4 pc=0x127a64
runtime.goexit({})
	runtime/asm_arm.s:859 +0x4 fp=0x1263fec sp=0x1263fec pc=0x771c0
created by os/exec.(*Cmd).Start in goroutine 416
	os/exec/exec.go:723 +0x968
go tool dist: Failed: exit status: 'go 30131: 2'

— watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `^plan9.*` && `panic during panic`
2024-01-08 19:14 plan9-arm perf@4ad5199a go@f19f31f2 (log) 
runtime:   gp: gp=0x11f5e68, goid=0, gp->atomicstatus=0
runtime: getg:  g=0x131bd48, goid=0,  g->atomicstatus=0
fatal error: bad g->status in ready

runtime stack:
runtime.throw({0x68c5ec, 0x16})
	runtime/panic.go:1023 +0x4c fp=0x3ffffd18 sp=0x3ffffd04 pc=0x42784
runtime.ready(0x11f5e68, 0x4, 0x1)
	runtime/proc.go:959 +0x13c fp=0x3ffffd38 sp=0x3ffffd18 pc=0x47138
runtime.send.goready.func1()
...
fatal error: unexpected signal during runtime execution
panic during panic
[signal sys: trap: fault read code=0x0 addr=0x0 pc=0x6e290]

runtime stack:
runtime.throw({0x69c9f2, 0x2a})
	runtime/panic.go:1023 +0x4c fp=0x3ffff9c0 sp=0x3ffff9ac pc=0x42784
runtime.sigpanic()
	runtime/os_plan9.go:75 +0x4d0 fp=0x3ffffa00 sp=0x3ffff9c0 pc=0x3ec58
runtime.(*unwinder).resolveInternal(0x3ffffb44, 0x1, 0x1)
...
runtime.fatalthrow(0x2)
	runtime/panic.go:1192 +0x64 fp=0x3ffffd04 sp=0x3ffffce8 pc=0x42c08
runtime.throw({0x68c5ec, 0x16})
	runtime/panic.go:1023 +0x4c fp=0x3ffffd18 sp=0x3ffffd04 pc=0x42784
runtime.ready(0x11f5e68, 0x4, 0x1)
	runtime/proc.go:959 +0x13c fp=0x3ffffd38 sp=0x3ffffd18 pc=0x47138
runtime.send.goready.func1()
	runtime/proc.go:413 +0x2c fp=0x3ffffd48 sp=0x3ffffd38 pc=0x8a90
runtime.systemstack(0x49901)
	runtime/asm_arm.s:317 +0x60 fp=0x3ffffd4c sp=0x3ffffd48 pc=0x7a650

— watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `^plan9.*` && `panic during panic`
2024-03-08 22:17 plan9-arm go@854dea06 go/types.TestSelf (log) 
--- FAIL: TestSelf (2.72s)
    self_test.go:33: api.go:32:2: could not import go/constant (can't find import: "go/constant": runtime:   gp: gp=0x117cd88, goid=157, gp->atomicstatus=2
        runtime: getg:  g=0x1202488, goid=0,  g->atomicstatus=0
        fatal error: bad g->status in ready

        runtime stack:
        runtime.throw({0x69c4e9, 0x16})
        	runtime/panic.go:1021 +0x4c fp=0x3ffffcd4 sp=0x3ffffcc0 pc=0x42c7c
        runtime.ready(0x117cd88, 0x4, 0x1)
        	runtime/proc.go:960 +0x15c fp=0x3ffffcf4 sp=0x3ffffcd4 pc=0x47854
...
        fatal error: unexpected signal during runtime execution
        panic during panic
        [signal sys: trap: fault read code=0x0 addr=0x0 pc=0x6e338]

        runtime stack:
        runtime.throw({0x6ad029, 0x2a})
        	runtime/panic.go:1021 +0x4c fp=0x3ffff8ac sp=0x3ffff898 pc=0x42c7c
        runtime.sigpanic()
        	runtime/os_plan9.go:75 +0x4d0 fp=0x3ffff8ec sp=0x3ffff8ac pc=0x3f1b4
        runtime.(*unwinder).next(0x3ffffb00)
...
        	runtime/panic.go:1190 +0x64 fp=0x3ffffcc0 sp=0x3ffffca4 pc=0x43100
        runtime.throw({0x69c4e9, 0x16})
        	runtime/panic.go:1021 +0x4c fp=0x3ffffcd4 sp=0x3ffffcc0 pc=0x42c7c
        runtime.ready(0x117cd88, 0x4, 0x1)
        	runtime/proc.go:960 +0x15c fp=0x3ffffcf4 sp=0x3ffffcd4 pc=0x47854
        runtime.send.goready.func1()
        	runtime/proc.go:413 +0x2c fp=0x3ffffd04 sp=0x3ffffcf4 pc=0x8b58
        runtime.systemstack(0x4a101)
        	runtime/asm_arm.s:317 +0x60 fp=0x3ffffd08 sp=0x3ffffd04 pc=0x7a910
        )

— watchflakes

@gopherbot
Copy link
Contributor

Found new dashboard test flakes for:

#!watchflakes
post <- builder ~ `^plan9.*` && `panic during panic`
2024-04-12 18:04 plan9-arm go@28b5d80d (log) 
fatal error: unexpected signal during runtime execution
fatal error: unexpected signal during runtime execution
panic during panic
[signal sys: trap: fault read code=0x0 addr=0x0 pc=0x46ca4]

runtime stack:
runtime.throw({0x4b3b97, 0x2a})
	/boot/workdir/go/src/runtime/panic.go:1021 +0x4c fp=0x3ffffb6c sp=0x3ffffb58 pc=0x41f44
runtime.sigpanic()
	/boot/workdir/go/src/runtime/os_plan9.go:75 +0x4d0 fp=0x3ffffbac sp=0x3ffffb6c pc=0x3e55c
...
	/boot/workdir/go/src/syscall/zsyscall_plan9_arm.go:45 +0x48 fp=0x1269d24 sp=0x1269cf8 pc=0xcc1c0
syscall.Await(0x113c0a0)
	/boot/workdir/go/src/syscall/syscall_plan9.go:274 +0x50 fp=0x1269f84 sp=0x1269d24 pc=0xcb7a8
syscall.startProcess.func1()
	/boot/workdir/go/src/syscall/exec_plan9.go:517 +0x210 fp=0x1269fec sp=0x1269f84 pc=0xc9720
runtime.goexit({})
	/boot/workdir/go/src/runtime/asm_arm.s:884 +0x4 fp=0x1269fec sp=0x1269fec pc=0x78b18
created by syscall.startProcess in goroutine 1511
	/boot/workdir/go/src/syscall/exec_plan9.go:492 +0xbc
go tool dist: FAILED: /boot/workdir/go/pkg/tool/plan9_arm/go_bootstrap install -a cmd/asm cmd/cgo cmd/compile cmd/link: exit status: 'go_bootstrap 5030: 2'

— watchflakes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Plan9
Projects
Go Compiler / Runtime
Status: Triage Backlog
Test Flakes
Status: No status
Milestone
Backlog
Development

No branches or pull requests
3 participants
@bcmills @gopherbot @millerresearch