runtime: unknown GOEXPERIMENT spinbitmutex

[itsalvinchris]

Go version

go version go1.25.0 linux/arm64

Output of go env in your module/workspace:

AR='ar'
CC='gcc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='g++'
GCCGO='gccgo'
GO111MODULE=''
GOARCH='arm64'
GOARM64='v8.0'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/root/.cache/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT='nospinbitmutex'
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build767931946=/tmp/go-build -gno-record-gcc-switches'
GOHOSTARCH='arm64'
GOHOSTOS='linux'
GOINSECURE=''
GOMOD='/go/src/xxxx.xxxx.xxxx/y/z/tools/abc/go.mod'
GOMODCACHE='/go/pkg/mod'
GONOPROXY='none'
GONOSUMDB='xxxx.xxxx.xxxx'
GOOS='linux'
GOPATH='/go'
GOPRIVATE='xxxx.xxxx.xxxx'
GOPROXY='xxxx.xxxx.xxxx|proxy.golang.org|direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTELEMETRY='local'
GOTELEMETRYDIR='/root/.config/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_arm64'
GOVCS=''
GOVERSION='go1.25.0'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

/workspace # go version
go version go1.25.0 linux/arm64
/workspace # export GOEXPERIMENT=nospinbitmutex
/workspace # go version
go: unknown GOEXPERIMENT spinbitmutex

What did you see happen?

go: unknown GOEXPERIMENT spinbitmutex

What did you expect to see?

Should be fine, as the 1.24 release notes mention the new GOEXPERIMENT=nospinbitmutex. In 1.25, there are no release notes that mention the removal of this experiment. However, in the goexperiment standard library, it has been removed - comparing 1.25 to 1.24. We are using this flag in our production builds for some services due to changes in latency behavior since the mutex implementation changes in 1.24.

[gabyhelp]

Related Issues

• runtime: spinbitmutex performance differs between 1.24.0 and 1.24.1, what changed? #72117 (closed)
• testing: flags from different packages interact with GOEXPERIMENT=nocoverageredesign #65763 (closed)

Related Code Changes

• runtime,cmd/link: include GOEXPERIMENTs in runtime.Version(), "go version X"

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[ALTree]

Looks like the GOEXPERIMENT spinbitmutex flag was removed in 7a17711 but the change is indeed not mentioned in the 1.25 release notes.

due to changes in latency behavior since the mutex implementation changes in 1.24

Did you open an issue for this?

[itsalvinchris]

I haven't raised any issues yet, but based on my observations and benchmarks, the problem is most likely related to tail latency. Our HTTP service is highly sensitive to the p99 maximum latency (p95 is simillar when using nospinbit versus spinbit). The delays primarily occur during network calls, such as those to Redis, MySQL, or external HTTP services.

[itsalvinchris]

I conducted the benchmark myself on an arm64 system using an AWS c7g.4xlarge instance, which has 16 cores and 32 GB of memory. The results showed that overall performance is similar when using nospinbit versus spinbit. However, while the average latency is comparable, the variance in latency is noticeably higher.

[cherrymui]

Generally, GOEXPERIMENTs are removed without notice, when they have been enabled by default for a release or longer without major issues. So we don't need to document the removal on the release notes. However, we probably should document GOEXPERIMENT better in general.

Perhaps you want to file a new issue for the performance regression with spinbitmutex, with more details about your workload and how the issue could be reproduced. Thanks.