runtime/trace: TestFlightRecorderGenerationCount/MaxBytes failures

[gopherbot]

#!watchflakes
default <- pkg == "runtime/trace" && test == "TestFlightRecorderGenerationCount/MaxBytes"

Issue created automatically to collect these failures.

Example (log):

=== RUN   TestFlightRecorderGenerationCount/MaxBytes
    flightrecorder_test.go:289: must have thread, but thread ID is none
    helpers.go:38: Trace Go1.26
        EventBatch gen=27 m=18446744073709551615 time=13259767214631 size=26
        Sync
        Frequency freq=15625000
        ClockSnapshot dt=7 mono=848625101736854 sec=1771699533 nsec=946155833
        EventBatch gen=27 m=1040869 time=13259767214670 size=78
        ProcStatus dt=1 p=2 pstatus=1
        GoStatus dt=15 g=107 m=1040869 gstatus=2
...
        String id=68
        	data="runtime/trace_test.testFlightRecorder"
        String id=69
        	data="runtime/trace_test.TestFlightRecorderGenerationCount.func1"
        String id=70
        	data="runtime/trace_test.TestFlightRecorderGenerationCount.func3"
        EndOfGeneration
        
    helpers.go:39: Convert this to a raw trace with `go test internal/trace/testtrace -convert in.tracetxt -out out.trace`
--- FAIL: TestFlightRecorderGenerationCount/MaxBytes (3.04s)

— watchflakes

[gopherbot]

Found new dashboard test flakes for:

#!watchflakes
default <- pkg == "runtime/trace" && test == "TestFlightRecorderGenerationCount/MaxBytes"

2026-02-21 18:38 gotip-linux-ppc64le_power10 go@6435bf46 runtime/trace.TestFlightRecorderGenerationCount/MaxBytes (log)

=== RUN   TestFlightRecorderGenerationCount/MaxBytes
    flightrecorder_test.go:289: must have thread, but thread ID is none
    helpers.go:38: Trace Go1.26
        EventBatch gen=27 m=18446744073709551615 time=13259767214631 size=26
        Sync
        Frequency freq=15625000
        ClockSnapshot dt=7 mono=848625101736854 sec=1771699533 nsec=946155833
        EventBatch gen=27 m=1040869 time=13259767214670 size=78
        ProcStatus dt=1 p=2 pstatus=1
        GoStatus dt=15 g=107 m=1040869 gstatus=2
...
        String id=68
        	data="runtime/trace_test.testFlightRecorder"
        String id=69
        	data="runtime/trace_test.TestFlightRecorderGenerationCount.func1"
        String id=70
        	data="runtime/trace_test.TestFlightRecorderGenerationCount.func3"
        EndOfGeneration
        
    helpers.go:39: Convert this to a raw trace with `go test internal/trace/testtrace -convert in.tracetxt -out out.trace`
--- FAIL: TestFlightRecorderGenerationCount/MaxBytes (3.04s)

— watchflakes

[mknyszek]

I figured out what's wrong here. There's a corner case where the only way a running goroutine is emitted into the trace is via a GoStatusStack event, and it specifically happens to the goroutine that's advancing the trace. (This makes sense. For it to happen to another goroutine, it would have to be preempted. But this goroutine puts itself temporarily in _Gwaiting to suspend other goroutines, and later we know from the waitreason that we were actually running, so GoRunning ends up in the trace.) The parser isn't really prepared to deal with that, though in theory it can. We can do one of two things here. 1. Prevent this possibility by having the goroutine entering traceAdvance name itself (and its P) in the trace explicitly. 2. Teach the parser how to obtain and set up all the relevant context (and know that it has to order the ProcStatus update before the GoStatus event). (2) seems way harder. I'm inclined to just have this goroutine log itself into the trace instead of trying to suspend itself. This case is the only one that matters for the trace, so it will fully resolve the issue. I am inclined to do (1).

…

On Wed, Feb 25, 2026 at 1:29 PM cherrymui ***@***.***> wrote: Assigned #77742 <#77742> to @mknyszek <https://github.com/mknyszek>. — Reply to this email directly, view it on GitHub <#77742 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJQ3HA3SWBP3GJWVSAXZDT4NYH3DAVCNFSM6AAAAACV4FNLSKVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMRTGA4DENRYGQ4TOOA> . You are receiving this because you were assigned.Message ID: ***@***.***>

[mknyszek]

Oops, no, that's not totally right. It matters in the other cases too. If we suspend another goroutine that hasn't emitted any events but is in this funky state, we will emit a GoStatus event for it with status GoRunning. So we can get into this situation with other goroutines (like GC workers), too. I have an idea. On Thu, Feb 26, 2026 at 2:37 PM Michael Knyszek ***@***.***> wrote:

…

*mknyszek* left a comment (golang/go#77742) <#77742 (comment)> I figured out what's wrong here. There's a corner case where the only way a running goroutine is emitted into the trace is via a GoStatusStack event, and it specifically happens to the goroutine that's advancing the trace. (This makes sense. For it to happen to another goroutine, it would have to be preempted. But this goroutine puts itself temporarily in _Gwaiting to suspend other goroutines, and later we know from the waitreason that we were actually running, so GoRunning ends up in the trace.) The parser isn't really prepared to deal with that, though in theory it can. We can do one of two things here. 1. Prevent this possibility by having the goroutine entering traceAdvance name itself (and its P) in the trace explicitly. 2. Teach the parser how to obtain and set up all the relevant context (and know that it has to order the ProcStatus update before the GoStatus event). (2) seems way harder. I'm inclined to just have this goroutine log itself into the trace instead of trying to suspend itself. This case is the only one that matters for the trace, so it will fully resolve the issue. I am inclined to do (1). On Wed, Feb 25, 2026 at 1:29 PM cherrymui ***@***.***> wrote: > Assigned #77742 <#77742> to @mknyszek > <https://github.com/mknyszek>. > > — > Reply to this email directly, view it on GitHub > <#77742 (comment)>, or > unsubscribe > < https://github.com/notifications/unsubscribe-auth/AAJQ3HA3SWBP3GJWVSAXZDT4NYH3DAVCNFSM6AAAAACV4FNLSKVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMRTGA4DENRYGQ4TOOA> > . > You are receiving this because you were assigned.Message ID: > ***@***.***> > — Reply to this email directly, view it on GitHub <#77742 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJQ3HHZ53MFXZK4LSEP32D4N5YTBAVCNFSM6AAAAACV4FNLSKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTSNRZGY2DENZYHE> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[mknyszek]

No, I was right the first time. What saves us for other goroutines is the call to forEachP, which makes sure that all the Ps are preempted once. This ensures running goroutines will have a status emitted *except* for exactly the one in traceAdvance (since that forEachP will trivially call the callback directly). On Thu, Feb 26, 2026 at 2:57 PM Michael Knyszek ***@***.***> wrote:

…

*mknyszek* left a comment (golang/go#77742) <#77742 (comment)> Oops, no, that's not totally right. It matters in the other cases too. If we suspend another goroutine that hasn't emitted any events but is in this funky state, we will emit a GoStatus event for it with status GoRunning. So we can get into this situation with other goroutines (like GC workers), too. I have an idea. On Thu, Feb 26, 2026 at 2:37 PM Michael Knyszek ***@***.***> wrote: > *mknyszek* left a comment (golang/go#77742) > <#77742 (comment)> > I figured out what's wrong here. There's a corner case where the only way > a > running goroutine is emitted into the trace is via a GoStatusStack event, > and it specifically happens to the goroutine that's advancing the trace. > (This makes sense. For it to happen to another goroutine, it would have to > be preempted. But this goroutine puts itself temporarily in _Gwaiting to > suspend other goroutines, and later we know from the waitreason that we > were actually running, so GoRunning ends up in the trace.) The parser > isn't > really prepared to deal with that, though in theory it can. > > We can do one of two things here. > 1. Prevent this possibility by having the goroutine entering traceAdvance > name itself (and its P) in the trace explicitly. > 2. Teach the parser how to obtain and set up all the relevant context (and > know that it has to order the ProcStatus update before the GoStatus > event). > > (2) seems way harder. I'm inclined to just have this goroutine log itself > into the trace instead of trying to suspend itself. This case is the only > one that matters for the trace, so it will fully resolve the issue. I am > inclined to do (1). > > On Wed, Feb 25, 2026 at 1:29 PM cherrymui ***@***.***> wrote: > > > Assigned #77742 <#77742> to > @mknyszek > > <https://github.com/mknyszek>. > > > > — > > Reply to this email directly, view it on GitHub > > <#77742 (comment)>, or > > unsubscribe > > < > https://github.com/notifications/unsubscribe-auth/AAJQ3HA3SWBP3GJWVSAXZDT4NYH3DAVCNFSM6AAAAACV4FNLSKVHI2DSMVQWIX3LMV45UABCJFZXG5LFIV3GK3TUJZXXI2LGNFRWC5DJN5XDWMRTGA4DENRYGQ4TOOA> > > > . > > You are receiving this because you were assigned.Message ID: > > ***@***.***> > > > > — > Reply to this email directly, view it on GitHub > <#77742 (comment)>, or > unsubscribe > < https://github.com/notifications/unsubscribe-auth/AAJQ3HHZ53MFXZK4LSEP32D4N5YTBAVCNFSM6AAAAACV4FNLSKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTSNRZGY2DENZYHE> > . > You are receiving this because you are subscribed to this thread.Message > ID: ***@***.***> > — Reply to this email directly, view it on GitHub <#77742 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAJQ3HBYZQLLL25ZMIOVQW34N524LAVCNFSM6AAAAACV4FNLSKVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTSNRZG4ZDAOJVHE> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[gopherbot]

Change https://go.dev/cl/750940 mentions this issue: runtime: explicitly trace status of traceAdvance goroutine