runtime: fatal error: throwOnGCWork (and general builder instability) #29124
Dup of #27993. (Definitely a release-blocker. I'm actively working on this one.)
Dup of #25519. Only affects debug call injection, so not high-priority. This isn't technically Android-specific, though it seems to happen a lot there, so maybe we should just disable that test on Android.
Stuck doing a syscall.Open?
I believe the vet process used ~100% cpu when I sent it SIGQUIT. So an infinite loop somehow.
This fixes a few different issues that led to hangs and general flakiness in the TestDebugCall* tests. 1. This fixes missing wake-ups in two error paths of the SIGTRAP signal handler. If the goroutine was in an unknown state, or if there was an unknown debug call status, we currently don't wake the injection coordinator. These are terminal states, so this resulted in a hang. 2. This adds a retry if the target goroutine is in a transient state that prevents us from injecting a call. The most common failure mode here is that the target goroutine is in _Grunnable, but this was previously masked because it deadlocked the test. 3. Related to 2, this switches the "ready" signal from the target goroutine from a blocking channel send to a non-blocking channel send. This makes it much less likely that we'll catch this goroutine while it's in the runtime performing that send. 4. This increases GOMAXPROCS from 2 to 8 during these tests. With the current setting of 2, we can have at most the non-preemptible goroutine we're injecting a call in to and the goroutine that's trying to make it exit. If anything else comes along, it can deadlock. One particular case I observed was in TestDebugCallGC, where runtime.GC() returns before the forEachP that prepares sweeping on all goroutines has finished. When this happens, the forEachP blocks on the non-preemptible loop, which means we now have at least three goroutines that need to run. Fixes #25519. Updates #29124. Change-Id: I7bc41dc0b865b7d0bb379cb654f9a1218bc37428 Reviewed-on: https://go-review.googlesource.com/c/154112 Run-TryBot: Austin Clements <firstname.lastname@example.org> Reviewed-by: Michael Knyszek <email@example.com>