runtime: failures with `fatal error: traceback did not unwind completely` on `openbsd/arm64` #62182

bcmills · 2023-08-21T15:15:15Z

#!watchflakes
post <- goos == "openbsd" && goarch == "arm64" && log ~ `fatal error: traceback did not unwind completely`

(Pulled out from #55167 (comment), #62105 (comment), and #62101 (comment).)

The text was updated successfully, but these errors were encountered:

gopherbot · 2023-08-21T15:33:35Z

Found new dashboard test flakes for:

#!watchflakes
post <- log ~ `fatal error: traceback did not unwind completely`

2023-08-07 19:31 linux-arm64-longtest go@d367ec6a runtime.TestVDSO (log)

--- FAIL: TestVDSO (0.65s)
    crash_test.go:158: /tmp/workdir/tmp/go-build1177904178/testprog.exe SignalInVDSO: exit status 2
    crash_test.go:161: output:
        runtime: g1: frame.sp=0x400029af2f top=0x400029afd0
        	stack=[0x400029a000-0x400029b000
        fatal error: traceback did not unwind completely

        runtime stack:
        runtime.throw({0x10ccf6?, 0x1?})
        	/tmp/workdir/go/src/runtime/panic.go:859 +0x40 fp=0xfffff3b14370 sp=0xfffff3b14340 pc=0x4a230
        runtime.(*unwinder).finishInternal(0xfffff3b14488?)
        	/tmp/workdir/go/src/runtime/traceback.go:561 +0x138 fp=0xfffff3b143b0 sp=0xfffff3b14370 pc=0x710e8
        runtime.(*unwinder).next(0xfffff3b14488?)
        	/tmp/workdir/go/src/runtime/traceback.go:442 +0x2cc fp=0xfffff3b14440 sp=0xfffff3b143b0 pc=0x70efc
        runtime.copystack(0x40000021a0, 0x1000)
        	/tmp/workdir/go/src/runtime/stack.go:931 +0x30c fp=0xfffff3b14840 sp=0xfffff3b14440 pc=0x65c2c
        runtime.newstack()
        	/tmp/workdir/go/src/runtime/stack.go:1113 +0x384 fp=0xfffff3b149f0 sp=0xfffff3b14840 pc=0x660f4
        runtime.morestack()
        	/tmp/workdir/go/src/runtime/asm_arm64.s:316 +0x70 fp=0xfffff3b149f0 sp=0xfffff3b149f0 pc=0x7d5d0


        wanted:
        success

2023-08-07 19:31 linux-arm64-longtest go@aca65771 runtime.TestCgoCCodeSIGPROF (log)

--- FAIL: TestCgoCCodeSIGPROF (0.28s)
    crash_cgo_test.go:217: /tmp/workdir/tmp/go-build2962622675/testprogcgo.exe CgoCCodeSIGPROF: exit status 2
    crash_cgo_test.go:220: expected "OK\n" got runtime: g6: frame.sp=0x4000284782 top=0x40002847d0
        	stack=[0x4000284000-0x4000284800
        fatal error: traceback did not unwind completely

        runtime stack:
        runtime.throw({0x572985?, 0x1?})
        	/tmp/workdir/go/src/runtime/panic.go:859 +0x40 fp=0xf8d5df7fb400 sp=0xf8d5df7fb3d0 pc=0x43dc00
        runtime.(*unwinder).finishInternal(0xf8d5df7fb518?)
        	/tmp/workdir/go/src/runtime/traceback.go:558 +0x138 fp=0xf8d5df7fb440 sp=0xf8d5df7fb400 pc=0x468c98
        runtime.(*unwinder).next(0xf8d5df7fb518?)
        	/tmp/workdir/go/src/runtime/traceback.go:439 +0x2c8 fp=0xf8d5df7fb4d0 sp=0xf8d5df7fb440 pc=0x468aa8
        runtime.copystack(0x4000003d40, 0x800)
        	/tmp/workdir/go/src/runtime/stack.go:930 +0x2fc fp=0xf8d5df7fb5d0 sp=0xf8d5df7fb4d0 pc=0x45b5bc
        runtime.newstack()
        	/tmp/workdir/go/src/runtime/stack.go:1112 +0x384 fp=0xf8d5df7fb780 sp=0xf8d5df7fb5d0 pc=0x45ba84
        runtime.morestack()
        	/tmp/workdir/go/src/runtime/asm_arm64.s:316 +0x70 fp=0xf8d5df7fb780 sp=0xf8d5df7fb780 pc=0x476ab0

2023-08-15 03:40 plan9-arm go@60506f4d (log)

runtime: signal received on thread not created by Go.
runtime: g820: frame.sp=0x9a37e24 top=0x9a37fec
	stack=[0x9a28000-0x9a38000
fatal error: traceback did not unwind completely

runtime stack:
go tool dist: Failed: exit status: 'go 43241: runtime: signal received on thread not created by Go.
'
runtime.throw({0x68c649, 0x23})
	runtime/panic.go:859 +0x4c fp=0x3ffff78c sp=0x3ffff778 pc=0x40fb0
runtime.(*unwinder).finishInternal(0x3ffff824)
	runtime/traceback.go:561 +0x164 fp=0x3ffff7ac sp=0x3ffff78c pc=0x691bc
runtime.(*unwinder).next(0x3ffff824)
	runtime/traceback.go:442 +0x270 fp=0x3ffff804 sp=0x3ffff7ac pc=0x68e40
runtime.copystack(0x9576500, 0x10000)
	runtime/stack.go:931 +0x2c0 fp=0x3ffffa78 sp=0x3ffff804 pc=0x5aab8
runtime.shrinkstack(0x9576500)
	runtime/stack.go:1215 +0x118 fp=0x3ffffa88 sp=0x3ffffa78 pc=0x5bb0c
runtime.scanstack(0x9576500, 0x943295c)
	runtime/mgcmark.go:776 +0x128 fp=0x3ffffd30 sp=0x3ffffa88 pc=0x26144
runtime.markroot.func1()
	runtime/mgcmark.go:240 +0xf4 fp=0x3ffffd5c sp=0x3ffffd30 pc=0x24a44
runtime.markroot(0x943295c, 0x3d, 0x1)
	runtime/mgcmark.go:214 +0x234 fp=0x3ffffdb8 sp=0x3ffffd5c pc=0x24620
runtime.gcDrain(0x943295c, 0x3)
	runtime/mgcmark.go:1069 +0x5ac fp=0x3ffffe04 sp=0x3ffffdb8 pc=0x27088
runtime.gcBgMarkWorker.func2()
	runtime/mgc.go:1386 +0xb8 fp=0x3ffffe30 sp=0x3ffffe04 pc=0x21cd0
runtime.systemstack()
	runtime/asm_arm.s:317 +0x60 fp=0x3ffffe34 sp=0x3ffffe30 pc=0x754e0

2023-08-20 05:56 openbsd-arm64-jsing go@a2ec545e os/signal.test [build] (log)

go.go(/home/gopher/build/tmp/go-link-917651527/go.o:(syscall.libc_syscall_trampoline.abi0)): warning: syscall() may go away, please rewrite code to use direct calls

2023-08-20 05:56 openbsd-arm64-jsing go@a2ec545e strings (log)

runtime: g1: frame.sp=0x400011fe00 top=0x400013ffd0
	stack=[0x4000100000-0x4000140000
fatal error: traceback did not unwind completely

runtime stack:
runtime.throw({0x171b13?, 0x400013ffb0?})
	/home/gopher/build/go/src/runtime/panic.go:1018 +0x40 fp=0x4dff20360 sp=0x4dff20330 pc=0x452a0
runtime.(*unwinder).finishInternal(0x202f0?)
	/home/gopher/build/go/src/runtime/traceback.go:561 +0x110 fp=0x4dff203a0 sp=0x4dff20360 pc=0x6b7a0
runtime.(*unwinder).next(0x4dff20478?)
	/home/gopher/build/go/src/runtime/traceback.go:442 +0x2a4 fp=0x4dff20430 sp=0x4dff203a0 pc=0x6b5e4
runtime.scanstack(0x40000021a0, 0x7?)
	/home/gopher/build/go/src/runtime/mgcmark.go:802 +0x23c fp=0x4dff20870 sp=0x4dff20430 pc=0x2d8dc
runtime.markroot.func1()
	/home/gopher/build/go/src/runtime/mgcmark.go:240 +0xb4 fp=0x4dff208c0 sp=0x4dff20870 pc=0x2c664
runtime.markroot(0x400002a140, 0x14, 0x1)
	/home/gopher/build/go/src/runtime/mgcmark.go:214 +0x1cc fp=0x4dff20970 sp=0x4dff208c0 pc=0x2c32c
runtime.gcDrain(0x400002a140, 0x7)
	/home/gopher/build/go/src/runtime/mgcmark.go:1069 +0x3f8 fp=0x4dff209d0 sp=0x4dff20970 pc=0x2e308
runtime.gcBgMarkWorker.func2()
	/home/gopher/build/go/src/runtime/mgc.go:1405 +0x74 fp=0x4dff20a20 sp=0x4dff209d0 pc=0x2a504
runtime.systemstack()
	/home/gopher/build/go/src/runtime/asm_arm64.s:243 +0x6c fp=0x4dff20a30 sp=0x4dff20a20 pc=0x765bc

— watchflakes

bcmills · 2023-08-21T15:51:05Z

Hrm. The failure on linux-arm64-longtest may have been fixed by https://go.dev/cl/516856.

If so, that may indicate that the openbsd/arm64 failure had a different root cause.
(attn @4a6f656c @golang/openbsd)

aclements · 2023-08-21T16:11:37Z

Yes, the two linux-arm64-longtest failures were fixed by https://go.dev/cl/516856.

gopherbot · 2023-08-21T23:08:17Z

Found new dashboard test flakes for:

#!watchflakes
post <- goos == "openbsd" && goarch == "arm64" && log ~ `fatal error: traceback did not unwind completely`

2023-08-21 18:00 openbsd-arm64-jsing go@c7383441 os/signal.test [build] (log)

go.go(/home/gopher/build/tmp/go-link-3327794622/go.o:(syscall.libc_syscall_trampoline.abi0)): warning: syscall() may go away, please rewrite code to use direct calls

2023-08-21 18:00 openbsd-arm64-jsing go@c7383441 strings (log)

runtime: g1: frame.sp=0x400019fe00 top=0x40001bffd0
	stack=[0x4000180000-0x40001c0000
fatal error: traceback did not unwind completely

runtime stack:
runtime.throw({0x171b13?, 0x40001bffb0?})
	/home/gopher/build/go/src/runtime/panic.go:1018 +0x40 fp=0x4564aba10 sp=0x4564ab9e0 pc=0x452a0
runtime.(*unwinder).finishInternal(0x202f0?)
	/home/gopher/build/go/src/runtime/traceback.go:561 +0x110 fp=0x4564aba50 sp=0x4564aba10 pc=0x6b7a0
runtime.(*unwinder).next(0x4564abb28?)
	/home/gopher/build/go/src/runtime/traceback.go:442 +0x2a4 fp=0x4564abae0 sp=0x4564aba50 pc=0x6b5e4
runtime.scanstack(0x40000021a0, 0x10?)
	/home/gopher/build/go/src/runtime/mgcmark.go:802 +0x23c fp=0x4564abf20 sp=0x4564abae0 pc=0x2d8dc
runtime.markroot.func1()
	/home/gopher/build/go/src/runtime/mgcmark.go:240 +0xb4 fp=0x4564abf70 sp=0x4564abf20 pc=0x2c664
runtime.markroot(0x400002a140, 0x14, 0x1)
	/home/gopher/build/go/src/runtime/mgcmark.go:214 +0x1cc fp=0x4564ac020 sp=0x4564abf70 pc=0x2c32c
runtime.gcDrain(0x400002a140, 0x7)
	/home/gopher/build/go/src/runtime/mgcmark.go:1069 +0x3f8 fp=0x4564ac080 sp=0x4564ac020 pc=0x2e308
runtime.gcBgMarkWorker.func2()
	/home/gopher/build/go/src/runtime/mgc.go:1405 +0x74 fp=0x4564ac0d0 sp=0x4564ac080 pc=0x2a504
runtime.systemstack()
	/home/gopher/build/go/src/runtime/asm_arm64.s:243 +0x6c fp=0x4564ac0e0 sp=0x4564ac0d0 pc=0x765bc

— watchflakes

Giulio2002 · 2023-09-23T20:35:22Z

I get this error with go1.21.1 as well, is it fixed already somewhere?

potuz · 2023-09-24T00:27:27Z

I get this error with go1.21.1 as well, is it fixed already somewhere?

It's fixed here #62326

aclements · 2023-09-25T15:50:36Z

@Giulio2002 , are you getting this error on OpenBSD or a different platform? The fix for #62326 will appear in Go 1.21.2, but it appears there's still some separate issue on OpenBSD, which is what the current issue is still tracking.

gopherbot · 2023-11-24T22:11:29Z

Found new dashboard test flakes for:

#!watchflakes
post <- goos == "openbsd" && goarch == "arm64" && log ~ `fatal error: traceback did not unwind completely`

2023-11-22 15:50 openbsd-arm64-jsing go@37971b31 os/signal.test [build] (log)

go.go(/home/gopher/build/tmp/go-link-1478659808/go.o:(syscall.libc_syscall_trampoline.abi0)): warning: syscall() may go away, please rewrite code to use direct calls

2023-11-22 15:50 openbsd-arm64-jsing go@37971b31 strings (log)

runtime: g1: frame.sp=0x400019fe10 top=0x40001bffd0
	stack=[0x4000180000-0x40001c0000
fatal error: traceback did not unwind completely

runtime stack:
runtime.throw({0x18397d?, 0x4105ea6b8?})
	/home/gopher/build/go/src/runtime/panic.go:1023 +0x40 fp=0x4105ea510 sp=0x4105ea4e0 pc=0x47000
runtime.(*unwinder).finishInternal(0x4105ea578?)
	/home/gopher/build/go/src/runtime/traceback.go:565 +0x110 fp=0x4105ea550 sp=0x4105ea510 pc=0x71fe0
runtime.(*unwinder).next(0x4105ea630?)
...

goroutine 7 gp=0x4000003860 m=nil [GC worker (idle)]:
runtime.gopark(0x400003efb0?, 0x0?, 0x0?, 0x0?, 0x0?)
	/home/gopher/build/go/src/runtime/proc.go:402 +0xc8 fp=0x400003ef30 sp=0x400003ef10 pc=0x49e08
runtime.gcBgMarkWorker()
	/home/gopher/build/go/src/runtime/mgc.go:1310 +0xd8 fp=0x400003efd0 sp=0x400003ef30 pc=0x2ae58
runtime.goexit({})
	/home/gopher/build/go/src/runtime/asm_arm64.s:1222 +0x4 fp=0x400003efd0 sp=0x400003efd0 pc=0x7efd4
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/home/gopher/build/go/src/runtime/mgc.go:1234 +0x28

— watchflakes

gopherbot · 2023-11-30T00:37:23Z

Found new dashboard test flakes for:

#!watchflakes
post <- goos == "openbsd" && goarch == "arm64" && log ~ `fatal error: traceback did not unwind completely`

2023-11-29 19:13 openbsd-arm64-jsing go@636c6e35 os/signal.test [build] (log)

go.go(/home/gopher/build/tmp/go-link-1780026275/go.o:(syscall.libc_syscall_trampoline.abi0)): warning: syscall() may go away, please rewrite code to use direct calls

2023-11-29 19:13 openbsd-arm64-jsing go@636c6e35 strings (log)

runtime: g1: frame.sp=0x400011fe10 top=0x400013ffd0
	stack=[0x4000100000-0x4000140000
fatal error: traceback did not unwind completely

runtime stack:
runtime.throw({0x183782?, 0x45f87ccd8?})
	/home/gopher/build/go/src/runtime/panic.go:1023 +0x40 fp=0x45f87cb30 sp=0x45f87cb00 pc=0x47070
runtime.(*unwinder).finishInternal(0x45f87cb98?)
	/home/gopher/build/go/src/runtime/traceback.go:565 +0x110 fp=0x45f87cb70 sp=0x45f87cb30 pc=0x72110
runtime.(*unwinder).next(0x45f87cc50?)
...

goroutine 6 gp=0x4000003380 m=nil [GC worker (idle)]:
runtime.gopark(0x400003efb0?, 0x400026c000?, 0x1a?, 0xa?, 0x0?)
	/home/gopher/build/go/src/runtime/proc.go:402 +0xc8 fp=0x400003ef30 sp=0x400003ef10 pc=0x49e78
runtime.gcBgMarkWorker()
	/home/gopher/build/go/src/runtime/mgc.go:1310 +0xd8 fp=0x400003efd0 sp=0x400003ef30 pc=0x2ae58
runtime.goexit({})
	/home/gopher/build/go/src/runtime/asm_arm64.s:1222 +0x4 fp=0x400003efd0 sp=0x400003efd0 pc=0x7f134
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/home/gopher/build/go/src/runtime/mgc.go:1234 +0x28

— watchflakes

gopherbot · 2023-12-04T12:22:42Z

Found new dashboard test flakes for:

#!watchflakes
post <- goos == "openbsd" && goarch == "arm64" && log ~ `fatal error: traceback did not unwind completely`

2023-11-29 15:49 openbsd-arm64-jsing go@1908d435 os/signal.test [build] (log)

go.go(/home/gopher/build/tmp/go-link-1313590292/go.o:(syscall.libc_syscall_trampoline.abi0)): warning: syscall() may go away, please rewrite code to use direct calls

2023-11-29 15:49 openbsd-arm64-jsing go@1908d435 strings (log)

runtime: g1: frame.sp=0x400019fe10 top=0x40001bffd0
	stack=[0x4000180000-0x40001c0000
fatal error: traceback did not unwind completely

runtime stack:
runtime.throw({0x183782?, 0x482cbdb18?})
	/home/gopher/build/go/src/runtime/panic.go:1023 +0x40 fp=0x482cbd970 sp=0x482cbd940 pc=0x47070
runtime.(*unwinder).finishInternal(0x482cbd9d8?)
	/home/gopher/build/go/src/runtime/traceback.go:565 +0x110 fp=0x482cbd9b0 sp=0x482cbd970 pc=0x72110
runtime.(*unwinder).next(0x482cbda90?)
...

goroutine 7 gp=0x4000003860 m=nil [GC worker (idle)]:
runtime.gopark(0x400003efb0?, 0x3?, 0x82?, 0x62?, 0x0?)
	/home/gopher/build/go/src/runtime/proc.go:402 +0xc8 fp=0x400003ef30 sp=0x400003ef10 pc=0x49e78
runtime.gcBgMarkWorker()
	/home/gopher/build/go/src/runtime/mgc.go:1310 +0xd8 fp=0x400003efd0 sp=0x400003ef30 pc=0x2ae58
runtime.goexit({})
	/home/gopher/build/go/src/runtime/asm_arm64.s:1222 +0x4 fp=0x400003efd0 sp=0x400003efd0 pc=0x7f134
created by runtime.gcBgMarkStartWorkers in goroutine 1
	/home/gopher/build/go/src/runtime/mgc.go:1234 +0x28

— watchflakes

bcmills added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. compiler/runtime Issues related to the Go compiler and/or runtime. labels Aug 21, 2023

bcmills mentioned this issue Aug 21, 2023

runtime: corruption in tests #55167

Open

bcmills changed the title ~~runtime: failures with fatal error: traceback did not unwind completely~~ runtime: failures with fatal error: traceback did not unwind completely on openbsd/arm64 Aug 21, 2023

bcmills added OS-OpenBSD arch-arm64 labels Aug 21, 2023

This comment was marked as off-topic.

Sign in to view

bcmills mentioned this issue Aug 28, 2023

runtime: "traceback did not unwind completely" #62326

Closed

mknyszek added this to the Backlog milestone Aug 30, 2023

mknyszek added the help wanted label Aug 30, 2023

Giulio2002 mentioned this issue Sep 29, 2023

Use go 1.21 in workflows ledgerwatch/erigon#8008

Closed

AskAlexSharov mentioned this issue Oct 24, 2023

erigon sepolia crashes with "fatal error: traceback did not unwind completely" ledgerwatch/erigon#8517

Closed

yeya24 mentioned this issue Nov 29, 2023

Thanos Receive (RouteOnly mode) Panic thanos-io/thanos#6942

Open

runtime: failures with `fatal error: traceback did not unwind completely` on `openbsd/arm64` #62182

runtime: failures with `fatal error: traceback did not unwind completely` on `openbsd/arm64` #62182

bcmills commented Aug 21, 2023 •

edited

gopherbot commented Aug 21, 2023

bcmills commented Aug 21, 2023

aclements commented Aug 21, 2023

gopherbot commented Aug 21, 2023

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

Giulio2002 commented Sep 23, 2023 •

edited

potuz commented Sep 24, 2023

aclements commented Sep 25, 2023

gopherbot commented Nov 24, 2023

gopherbot commented Nov 30, 2023

gopherbot commented Dec 4, 2023

runtime: failures with fatal error: traceback did not unwind completely on openbsd/arm64 #62182

runtime: failures with fatal error: traceback did not unwind completely on openbsd/arm64 #62182

Comments

bcmills commented Aug 21, 2023 • edited

gopherbot commented Aug 21, 2023

bcmills commented Aug 21, 2023

aclements commented Aug 21, 2023

gopherbot commented Aug 21, 2023

This comment was marked as off-topic.

This comment was marked as off-topic.

This comment was marked as off-topic.

Giulio2002 commented Sep 23, 2023 • edited

potuz commented Sep 24, 2023

aclements commented Sep 25, 2023

gopherbot commented Nov 24, 2023

gopherbot commented Nov 30, 2023

gopherbot commented Dec 4, 2023

runtime: failures with `fatal error: traceback did not unwind completely` on `openbsd/arm64` #62182

runtime: failures with `fatal error: traceback did not unwind completely` on `openbsd/arm64` #62182

bcmills commented Aug 21, 2023 •

edited

Giulio2002 commented Sep 23, 2023 •

edited