Repository navigation
runtime: darwin CPU profiles omit systemstack callers during syscalls #80921
Description
Go version
go version go1.28-devel_82065aa34d Mon Aug 17 10:35:04 2026 -0700 darwin/arm64
Output of go env in your module/workspace:
AR='ar'
CC='clang'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='clang++'
GCCGO='gccgo'
GO111MODULE=''
GOARCH='arm64'
GOARM64='v8.0'
GOAUTH='netrc'
GOBIN='/Users/rhysh/go/bin'
GOCACHE='/Users/rhysh/Library/Caches/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/Users/rhysh/Library/Application Support/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -arch arm64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -ffile-prefix-map=/var/folders/pw/d_qmtcrd3vs0890gvmrq8qx80000gn/T/go-build3165146723=/tmp/go-build -gno-record-gcc-switches -fno-common'
GOHOSTARCH='arm64'
GOHOSTOS='darwin'
GOINSECURE=''
GOMOD='/dev/null'
GOMODCACHE='/Users/rhysh/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='darwin'
GOPACKAGESDRIVER=''
GOPATH='/Users/rhysh/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTELEMETRY='on'
GOTELEMETRYDIR='/Users/rhysh/Library/Application Support/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/local/go/pkg/tool/darwin_arm64'
GOVCS=''
GOVERSION='go1.28-devel_82065aa34d Mon Aug 17 10:35:04 2026 -0700'
GOWORK=''
PKG_CONFIG='pkg-config'What did you do?
I took a CPU profile on darwin/arm64: a platform that uses SIGPROF for threads to profile themselves, and which also uses libcall for some syscalls.
I looked at the CPU profile samples that make particular syscalls and attempted to find their root, past a systemstack call.
$ go test runtime -bench=BenchmarkChanCreation -run=NONE -cpuprofile ./cpu.pb.gz
goos: darwin
goarch: arm64
pkg: runtime
cpu: Apple M1
BenchmarkChanCreation-8 40789101 28.87 ns/op
PASS
ok runtime 2.533s
$ go tool pprof -focus=runtime.usleep -peek=runtime.systemstack ./cpu.pb.gz
File: runtime.test
Type: cpu
Time: 2026-08-17 11:02:52 PDT
Duration: 2.11s, Total samples = 8.26s (391.66%)
Active filters:
focus=runtime.usleep
Showing nodes accounting for 2.22s, 26.88% of 8.26s total
----------------------------------------------------------+-------------
flat flat% sum% cum cum% calls calls% + context
----------------------------------------------------------+-------------
0 0% 0% 0.45s 5.45% | runtime.systemstack
0.40s 88.89% | runtime.(*mheap).freeSpan.func1
0.02s 4.44% | runtime.(*mheap).alloc.func1
0.02s 4.44% | runtime.(*pageAlloc).scavenge.func1
0.01s 2.22% | runtime.forEachP.func1
----------------------------------------------------------+-------------
What did you see happen?
The CPU profile samples that end in libcall syscalls don't follow the call stack past runtime.systemstack; note the lack of callers in the go tool pprof -peek=runtime.systemstack output above.
What did you expect to see?
I expected CPU profile samples to trace back past systemstack just as well when in syscalls as they do when executing Go code.
It looks like there's an unwindJumpStack flag missing in the usesLibcall branch of the SIGPROF handler: https://github.com/golang/go/blob/82065aa34d/src/runtime/proc.go#L5862-L5872
I see that @prattmic ran into some trouble in that area in #50936, leading to the full revert of CL 358900. That appears to have been permanently fixed by CL 428656.
Activity
gabyhelp commentedon Aug 17, 2026
Related Issues
Related Code Changes
(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)