runtime: unexpected return pc for runtime cacheSpan [...]

[karalabe]

Go version

go1.23.2

Output of go env in your module/workspace:

Docker alpine go1.23.2 on amd64

What did you do?

Ran out project as usual (?)

What did you see happen?

	fatal error: unexpected signal during runtime execution
	[signal SIGSEGV: segmentation violation code=0x1 addr=0x5f pc=0x41847f]
	 goroutine 5340061 gp=0xc000df7180 m=25 mp=0xc000583108 [running]:
	 runtime.throw({0x1b1bd01?, 0x20?})
	 	runtime/panic.go:1067 +0x48 fp=0xc005efa5a8 sp=0xc005efa578 pc=0x476e28
	 runtime.sigpanic()
	 	runtime/signal_unix.go:884 +0x3c9 fp=0xc005efa608 sp=0xc005efa5a8 pc=0x4790c9
	 runtime.(*mcentral).cacheSpan(0x417d53)
	 	runtime/mcentral.go:184 +0x15f fp=0xc005efa680 sp=0xc005efa608 pc=0x41847f
	 runtime: g 5340061: unexpected return pc for runtime.(*mcentral).cacheSpan called from 0xc005efa6b8
	 stack: frame={sp:0xc005efa608, fp:0xc005efa680} stack=[0xc005ef8000,0xc005efc000)
	 0x000000c005efa508:  0x00007fb023b0d688  0x0000000000000020
	 0x000000c005efa518:  0x0000000000000017  0x000000000047d4d2 <runtime.systemstack+0x0000000000000032>
	 0x000000c005efa528:  0x000000c005efa568  0x000000000043d1d8 <runtime.fatalthrow+0x0000000000000058>
	 0x000000c005efa538:  0x000000c005efa548  0x000000c000df7180
	 0x000000c005efa548:  0x000000000043d200 <runtime.fatalthrow.func1+0x0000000000000000>  0x000000c000df7180
	 0x000000c005efa558:  0x0000000000476e28 <runtime.throw+0x0000000000000048>  0x000000c005efa578
	 0x000000c005efa568:  0x000000c005efa598  0x0000000000476e28 <runtime.throw+0x0000000000000048>
	 0x000000c005efa578:  0x000000c005efa580  0x000000000043cda0 <runtime.throw.func1+0x0000000000000000>
	 0x000000c005efa588:  0x0000000001b1bd01  0x000000000000002a
	 0x000000c005efa598:  0x000000c005efa5f8  0x00000000004790c9 <runtime.sigpanic+0x00000000000003c9>
	 0x000000c005efa5a8:  0x0000000001b1bd01  0x0000000000000020
	 0x000000c005efa5b8:  0x0000000000000020  0x000000000178e740
	 0x000000c005efa5c8:  0x000000c0125879a0  0x000000c005efa648
	 0x000000c005efa5d8:  0x000000c000df7180  0x000000000178e740
	 0x000000c005efa5e8:  0x000000c005efa650  0x0000000000436d2a <runtime.(*spanSet).push+0x000000000000002a>
	 0x000000c005efa5f8:  0x000000c005efa678  0x000000000041847f <runtime.(*mcentral).cacheSpan+0x000000000000015f>
	 0x000000c005efa608: <0x000000000041843f <runtime.(*mcentral).cacheSpan+0x000000000000011f>  0x0000060405efa648
	 0x000000c005efa618:  0x00000000005d04e5 <github.com/ethereum/go-ethereum/rlp.makeListDecoder.func3+0x0000000000000025>  0x0000000000000197
	 0x000000c005efa628:  0x0001c90900000000  0x000000c012587988
	 0x000000c005efa638:  0x000000000178e740  0x000000c0125879a0
	 0x000000c005efa648:  0x000000c005efa7e8  0x00007fb0c8602cf8
	 0x000000c005efa658:  0x0000000000000005  0x0000000002eee528
	 0x000000c005efa668:  0x00007fb0a5f4e440  0x000000000178f9c0
	 0x000000c005efa678:  0x000000c005efa6b8 >0x0000000000417d53 <runtime.(*mcache).refill+0x0000000000000153>
	 0x000000c005efa688:  0x0000000002eee4a8  0x0000000000000038
	 0x000000c005efa698:  0x000000c005efa6f0  0x0000000000000008
	 0x000000c005efa6a8:  0x00007fb0a5f4e440  0x000000000000001a
	 0x000000c005efa6b8:  0x000000c005efa6f0  0x00000000004119e5 <runtime.(*mcache).nextFree+0x0000000000000085>
	 0x000000c005efa6c8:  0x00007fb113c911d8  0x00000000004be91a <reflect.NewAt+0x000000000000003a>
	 0x000000c005efa6d8:  0x002d0000002d0d80  0x00007fb0a5f4e440
	 0x000000c005efa6e8:  0x000000000000001a  0x000000c005efa790
	 0x000000c005efa6f8:  0x000000000047176d <runtime.mallocgc+0x00000000000004cd>  0x00007fb113c911d8
	 0x000000c005efa708:  0x000000000000001a  0x000000c005efa730
	 0x000000c005efa718:  0x0000000000477ff3 <reflect.resolveTypeOff+0x0000000000000013>  0x0000000000000199
	 0x000000c005efa728:  0x00000000001f0d80  0x000000c005efa7b8
	 0x000000c005efa738:  0x00000000004c3c7f <reflect.(*rtype).ptrTo+0x000000000000009f>  0x0000000000000016
	 0x000000c005efa748:  0x00007fb113c911d8  0x00000000000000b0
	 0x000000c005efa758:  0x00000000018b0d80  0x000000c005efa758
	 0x000000c005efa768:  0x000000c000583108  0x000000c005efa7e8
	 0x000000c005efa778:  0x00000000004bab72 <reflect.Value.Set+0x0000000000000132>

What did you expect to see?

No big boom :)

[gabyhelp]

Related Issues and Documentation

• runtime: segmentation violation runtime.(*unwinder).next #66151 (closed)
• runtime: misc/cgo/test.TestSetgid panic with `unexpected return pc for runtime.sigpanic` on linux-arm64-packet since 2022-06-04 #53374 (closed)
• runtime: "schedule: holding locks" #44544 (closed)
• runtime: panic on memclrNoHeapPointers, unexpected fault address #59382 (closed)
• runtime: unexpected signal during runtime execution on Debian #29377
• runtime: fatal error: unexpected signal during runtime execution #60783 (closed)
• runtime, cmd/compile: SIGILL during fixedbugs/issue11656.go on aix-ppc64 #44583 (closed)
• fatal error: unexpected signal during runtime execution #17824 (closed)
• runtime: fatal error: unknown caller pc #44096 (closed)
• runtime: stack grow panic tracing back through sigpanic from signal handler #23484

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[ianlancetaylor]

CC @golang/runtime

[mknyszek]

The span pointer in the variable is definitely bogus, but I don't see how this is possible. This code path has not changed recently. This looks like memory corruption of the stack, especially given that the unwinder crashes trying to print a stack trace.

I don't think we'll be able to make much progress without a reproducer. But here are some thoughts/questions.

• Maybe give GODEBUG=asyncpreemptoff=1 a try. It's gotten far less buggy over the years but who knows.
• Are you using unsafe or cgo widely? Have you made any changes to your codebase related to these recently?
• I see you're running on Alpine; have you upgraded the image recently? There could be some incompatibility with a new version of musl. If it's a pure Go program, you can possibly try building with CGO_ENABLED=0 to avoid using libc on Linux to rule this out.

Thanks.

[gopherbot]

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)