runtime: segfault on tracestack.go:267 while running flight recorder #75478
Description
Go version
go version go1.25.1 linux/amd64
Output of go env in your module/workspace:
AR='ar'
CC='gcc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='g++'
GCCGO='gccgo'
GO111MODULE=''
GOAMD64='v1'
GOARCH='amd64'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/home/runner/.cache/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/home/runner/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build1173010075=/tmp/go-build -gno-record-gcc-switches'
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMOD='/home/runner/actions-runner/_work/[snip]/[snip]/go.mod'
GOMODCACHE='/home/runner/go/pkg/mod'
GONOPROXY='github.com/[snip]/[private repo]'
GONOSUMDB='github.com/[snip]/[private repo]'
GOOS='linux'
GOPATH='/home/runner/go'
GOPRIVATE='github.com/[snip]/[private repo]'
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.1.linux-amd64'
GOSUMDB='sum.golang.org'
GOTELEMETRY='local'
GOTELEMETRYDIR='/home/runner/.config/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/home/runner/go/pkg/mod/golang.org/toolchain@v0.0.1-go1.25.1.linux-amd64/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.25.1'
GOWORK='/home/runner/actions-runner/_work/[snip]/[snip]/go.work'
PKG_CONFIG='pkg-config'What did you do?
While running the new flight recorder in production, we're getting ~5 segfaults per 24-hours on one out of our four workloads.
What did you see happen?
SIGSEGV: segmentation violation
PC=0x475d8a m=5 sigcode=128 addr=0x0
goroutine 0 gp=0xc0005821c0 m=5 mp=0xc000580008 [idle]:
runtime.fpTracebackPCs(...)
runtime/tracestack.go:267
runtime.traceStack(0xc142152540?, 0x200000004?, 0x6846a50)
runtime/tracestack.go:124 +0x2ca fp=0xc000111f70 sp=0xc000111b08 pc=0x475d8a
runtime.traceAdvance.func1.1()
runtime/trace.go:399 +0xd6 fp=0xc000111fc8 sp=0xc000111f70 pc=0x46c916
runtime.systemstack(0x0)
runtime/asm_amd64.s:513 +0x4a fp=0xc000111fd8 sp=0xc000111fc8 pc=0x487baa
goroutine 233 gp=0xc001558540 m=5 mp=0xc000580008 [trace goroutine status]:
runtime.systemstack_switch()
runtime/asm_amd64.s:478 +0x8 fp=0xc1abf27d70 sp=0xc1abf27d60 pc=0x487b48
runtime.traceAdvance.func1(0xc27eccc430?)
runtime/trace.go:375 +0x107 fp=0xc1abf27e20 sp=0xc1abf27d70 pc=0x46c727
runtime.forEachGRace(0xc1abf27f70)
runtime/proc.go:745 +0x45 fp=0xc1abf27e50 sp=0xc1abf27e20 pc=0x44bbc5
runtime.traceAdvance(0x0)
runtime/trace.go:361 +0x9b fp=0xc1abf27fb8 sp=0xc1abf27e50 pc=0x48589b
runtime.(*traceAdvancerState).start.func1()
runtime/trace.go:996 +0x2f fp=0xc1abf27fe0 sp=0xc1abf27fb8 pc=0x46d06f
runtime.goexit(
{}
)
runtime/asm_amd64.s:1693 +0x1 fp=0xc1abf27fe8 sp=0xc1abf27fe0 pc=0x4899e1
created by runtime.(*traceAdvancerState).start in goroutine 1
runtime/trace.go:990 +0x15f
What did you expect to see?
No segfault. I'm not able to reproduce this locally; the go version and go env above are the ones from the github action that builds and deploys this binary. Not sure what data is important to debug this.