append() grows backing array to odd-sized, unexpected capacity

[ti-mo]

Go version

1.22.0

Output of go env in your module/workspace:

GO111MODULE=''
GOARCH='amd64'
GOBIN='/home/timo/.go/bin'
GOCACHE='/home/timo/.cache/go-build'
GOENV='/home/timo/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFLAGS=''
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMODCACHE='/home/timo/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/home/timo/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/lib/go'
GOSUMDB='sum.golang.org'
GOTMPDIR=''
GOTOOLCHAIN='auto'
GOTOOLDIR='/usr/lib/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.22.0'
GCCGO='gccgo'
GOAMD64='v1'
AR='ar'
CC='gcc'
CXX='g++'
CGO_ENABLED='1'
GOMOD='/dev/null'
GOWORK=''
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
PKG_CONFIG='pkg-config'
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build3026741144=/tmp/go-build -gno-record-gcc-switches'

What did you do?

I wrote a test once that reassigns a base slice to a few other variables and (accidentally) relied on append() to unshare the backing array between those variables. The base slice was exactly 32 elements long, appending an element would grow the new slice to cap 64, doing a copy() behind the scenes.

What did you see happen?

Once the slice exceeds 16 elements, the slice is grown to a cap of 37. This seems like an odd number, since typically the cap is doubled. Up until Go 1.21.7, this is still the case, but no longer in 1.22.0. I'm flagging this issue just in case a bug slipped into the calculation in 1.22.0.

This is a minimal reproducer: https://go.dev/play/p/AGFSQJ7wBsV. So far this only seems to happen when the slice element is larger than 128 bytes and contains a pointer. This reproduces locally as well as on the playground.

What did you expect to see?

The slice is grown to a cap of 32. Again, just making this issue out of an abundance caution because it broke a test that relied on unspecified behaviour. This is not part of the spec as far as I can see, and no one should rely on this property.

[randall77]

This is a consequence of the new allocation headers.

Previously, a 32 entry backing array of 24 byte elements just fits in the 768 byte size class.
Now, we need room for an object header. That's an extra 8 bytes, which bumps up to the next size class. That is the 896 byte size class, which can fit 37 24-byte elements.

It does not happen for the smaller sizes because below 512 bytes there are no allocation headers.

So this is not a bug, but an expected (mildly unfortunate) consequence of allocation headers.

Thanks for reporting it though. Good to know people are watching this kind of thing.

[ti-mo]

@randall77 Cool, good to know, thanks!