runtime: checkdead throws with lock held results in deadlock

[lucoffe]

What version of Go are you using (go version)?

1.20.3

Does this issue reproduce with the latest release?

yes

What did you do?

• Set schedtrace and/or sheddetail in GODEBUG
• Provoked a throw in checkdead on runnable G's with no running M's (see runtime: checkdead fires due to suspected race in the Go runtime when GOMAXPROCS=1 on AWS #59600)

What did you expect to see?

• The process crashes successfully

What did you see instead?

• The process does idle forever

Additional details

In the checkdead function context, the lock is held by the caller. Up the stack, following a throw, schedtrace will try to acquire the scheduler lock, resulting in a double lock.

 0  0x00000000004751c3 in runtime.futex
    at /Users/luciencoffe/sdk/gotip/src/runtime/sys_linux_amd64.s:558
 1  0x00000000004390b0 in runtime.futexsleep
    at /Users/luciencoffe/sdk/gotip/src/runtime/os_linux.go:69
 2  0x000000000040bb45 in runtime.lock2
    at /Users/luciencoffe/sdk/gotip/src/runtime/lock_futex.go:107
 3  0x000000000040c373 in runtime.lockWithRank
    at /Users/luciencoffe/sdk/gotip/src/runtime/lockrank_off.go:22
 4  0x000000000040baa5 in runtime.lock
    at /Users/luciencoffe/sdk/gotip/src/runtime/lock_futex.go:48
 5  0x000000000044a6e8 in runtime.schedtrace
    at /Users/luciencoffe/sdk/gotip/src/runtime/proc.go:5645
 6  0x000000000043d9f0 in runtime.startpanic_m
    at /Users/luciencoffe/sdk/gotip/src/runtime/panic.go:1243
 7  0x000000000043d6ee in runtime.fatalthrow.func1
    at /Users/luciencoffe/sdk/gotip/src/runtime/panic.go:1150
 8  0x000000000043d685 in runtime.fatalthrow
    at /Users/luciencoffe/sdk/gotip/src/runtime/panic.go:1149
 9  0x000000000043d37c in runtime.throw
    at /Users/luciencoffe/sdk/gotip/src/runtime/panic.go:1076
10  0x0000000000449ee5 in runtime.checkdead.func1
    at /Users/luciencoffe/sdk/gotip/src/runtime/proc.go:5294
11  0x0000000000440811 in runtime.forEachG
    at /Users/luciencoffe/sdk/gotip/src/runtime/proc.go:607
12  0x0000000000449bd7 in runtime.checkdead
    at /Users/luciencoffe/sdk/gotip/src/runtime/proc.go:5281
13  0x000000000044b169 in runtime.mput
    at /Users/luciencoffe/sdk/gotip/src/runtime/proc.go:5774
14  0x000000000044391c in runtime.stopm
    at /Users/luciencoffe/sdk/gotip/src/runtime/proc.go:2410
15  0x0000000000444430 in runtime.findRunnable
15  0x0000000000444430 in runtime.findRunnable
    at /Users/luciencoffe/sdk/gotip/src/runtime/proc.go:3085
16  0x0000000000445afd in runtime.schedule
    at /Users/luciencoffe/sdk/gotip/src/runtime/proc.go:3438
17  0x0000000000446019 in runtime.park_m
    at /Users/luciencoffe/sdk/gotip/src/runtime/proc.go:3589
18  0x00000000004714ee in runtime.mcall
    at /Users/luciencoffe/sdk/gotip/src/runtime/asm_amd64.s:458

[gopherbot]

Change https://go.dev/cl/487375 mentions this issue: runtime: prevent double lock in checkdead by unlocking before throws