runtime: printing multiple fatal errors may interleave messages

[grongor]

Go version

master and latest (1.23.1)

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN=''
GOCACHE='/home/grongor/.cache/go-build'
GOENV='/home/grongor/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/grongor/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/grongor/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.23.1'
GODEBUG=''
GOTELEMETRY='local'
GOTELEMETRYDIR='/home/grongor/.config/go/telemetry'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/home/grongor/projects/grongor/go-fatal-race/go.mod'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build2941128308=/tmp/go-build -gno-record-gcc-switches'

What did you do?

Runnable test: https://github.com/grongor/go-fatal-race
Playground: https://go.dev/play/p/YicJmPOpJ6E

What did you see happen?

fatal error: concurrent map writes
fatal error: concurrent map writes

goroutine 5 [running]:
main.main.func1()

Sometimes different malformations like this:

fatal error: fatal error: concurrent map writes
concurrent map writes

goroutine 7 [running]: ...

What did you expect to see?

single fatal error: concurrent map writes followed by a stacktrace

[gabyhelp]

Related Issues and Documentation

• runtime?: "concurrent map read and write" crashes no longer put the 2 offending goroutines at the top of the output #63172 (closed)
• testing: don't overwrite race detector exit code #43001
• runtime: fatal error stack traces are swallowed for binaries with elevated privileges #68103
• runtime: concurrent map iter+write fatal doesn't respect GOTRACEBACK #68019
• runtime: fatal ERROR: concurrent map read and map write #26010 (closed)
• runtime/race: tracebacks missing symbol/file names #60245 (closed)
• testing: data race between parallel panicking and normal subtest #37551 (closed)
• cmd/go: "fatal error: concurrent map writes" during go get #35317 (closed)
• cmd/compile/internal/types: concurrent map read and map write while compiling #21352 (closed)
• does len(map) race with map writes? #20615 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[gopherbot]

Change https://go.dev/cl/613016 mentions this issue: runtime: fix fatal()/throw() print race

[zigo101]

Looks some similar to #23097

[grongor]

Not really related...the issue here is that the individual print() calls from two (or more) different fatal() function calls can intertwine "arbitrarily", causing somewhat random output.

[timothy-king]

CC @golang/runtime

[randall77]

Probably we need printlock/printunlock around the prints inside throw and fatal.

[prattmic]

fatalthrow -> startpanic_m takes paniclk to serialize the traceback output, but the top-level fatal/throw message is outside the lock.

I don't think it is a good idea to take an additional lock before printing in throw. If we're throwing, then the runtime is in a very bad state and I want to minimize the chance that something goes wrong like hanging in throw before we print anything at all.

fatal, on the other hand, is for user errors and the runtime should still be in an OK state, so I think a printlock there would be OK.

[gopherbot]

Change https://go.dev/cl/615655 mentions this issue: runtime: print fatal messages without interleaving