runtime: mayMoreStackPreempt failures [1.20 backport]

[gopherbot]

@prattmic requested issue #55160 to be considered for backport to the next 1.20 minor release.

I believe this is a bug in https://go.dev/cl/7314062. (Yes, a 10 year old bug).

The "delicate dance" when a spinning M transitions to non-spinning is supposed to check all sources of work, but it doesn't check the global run queue: https://cs.opensource.google/go/go/+/master:src/runtime/proc.go;l=3084;drc=5b6e6d2b3d4d6877c86471209e249f1b6e36d0ca

gosched / mayMoreStack put the preempted goroutine on the global run queue, increasing the chance of failure (in addition to the extra stress it puts on the scheduler in general). Still, I am surprised this bug could go unnoticed for so long.

With that issue fixed, I can no longer reproduce this hang.

@gopherbot Please open backports to 1.20 and 1.19. This is a hang in the scheduler with no work around.

[prattmic]

Upon further investigation, it seems that this can only cause hangs if asynchronous preemption is disabled and a goroutine is in an otherwise non-preemptible loop. If those don't apply, this bug will still drop parallelism (1 fewer P will run than should) but otherwise the program will continue to work.

Thus my current thinking is that we should not backport the fix, but I will revisit once we finish evaluating and merging the fix for tip.

[prattmic]

We have decided to wait for 1.22 for this: #55160 (comment)