/ go Public
runtime: some runtime throws print all goroutine stack dumps by default #46995
Issues related to the Go compiler and/or runtime.
Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
I'm going to be using
go version devel go1.17-d19a53338f Wed Jun 30 02:00:49 2021 +0000 linux/amd64as an example here, but this also reproduces on 1.16.5.
From https://golang.org/pkg/runtime/, emphasis mine:
So it seems like, by default, a panic should only print the stack trace for the goroutine that panicked, and nothing else.
However, that doesn't seem correct in some cases: https://play.golang.org/p/zWw2TUS4YzZ
As per the docs, I would expect to just see one goroutine stack dump - the "running" one that hit the panic via
runtime.mapassign_fast64, but I get all of them.
This seems to be on purpose in the code:
Lines 1385 to 1399 in d19a533
This is problematic for two reasons:
A: It's confusing. This had me puzzled when looking at a Go process crash output along with the runtime docs.
B: It can lead to large amounts of panic output, depending on what kind of panic one runs into. At work, someone was very surprised that a panic dumped ~50MiB of goroutine stacks, while most other panics dumped just one goroutine stack. That actually caused infra issues, because it appears our log ingestion didn't like that sudden dump of output very much, and we never found out before given that most panics don't behave that way.
I see three possible resolutions to this problem:
The code is working as intended. We should update the docs to clarify the nuance that the default does print all goroutine stack traces in some cases, and what those cases are.
We should amend the implementation to match the docs. Concurrent map writes should result in a panic that only prints one stack trace, i.e. the "running" goroutine from above.
Something in between, such as "print the stack traces for all goroutines that are involved in writes to the map in question". I can imagine that the current behavior exists for debugging purposes: if you get concurrent map writes, you might want to know what those two or more goroutines were doing, not just one of them. However, you presumably don't need all goroutines for that; just the ones involved in the race.
cc @aclements @randall77 @mknyszek @prattmic as per https://dev.golang.org/owners
The text was updated successfully, but these errors were encountered: