runtime: CPU limit detection not working for non-root cgroup mount

[huww98]

Go version

go version go1.25.4 linux/amd64

Output of go env in your module/workspace:

AR='ar'
CC='gcc'
CGO_CFLAGS='-O2 -g'
CGO_CPPFLAGS=''
CGO_CXXFLAGS='-O2 -g'
CGO_ENABLED='1'
CGO_FFLAGS='-O2 -g'
CGO_LDFLAGS='-O2 -g'
CXX='g++'
GCCGO='gccgo'
GO111MODULE=''
GOAMD64='v1'
GOARCH='amd64'
GOAUTH='netrc'
GOBIN=''
GOCACHE='/root/.cache/go-build'
GOCACHEPROG=''
GODEBUG=''
GOENV='/root/.config/go/env'
GOEXE=''
GOEXPERIMENT=''
GOFIPS140='off'
GOFLAGS=''
GOGCCFLAGS='-fPIC -m64 -pthread -Wl,--no-gc-sections -fmessage-length=0 -ffile-prefix-map=/tmp/go-build657523016=/tmp/go-build -gno-record-gcc-switches'
GOHOSTARCH='amd64'
GOHOSTOS='linux'
GOINSECURE=''
GOMOD='/dev/null'
GOMODCACHE='/go/pkg/mod'
GONOPROXY=''
GONOSUMDB=''
GOOS='linux'
GOPATH='/go'
GOPRIVATE=''
GOPROXY='https://proxy.golang.org,direct'
GOROOT='/usr/local/go'
GOSUMDB='sum.golang.org'
GOTELEMETRY='local'
GOTELEMETRYDIR='/root/.config/go/telemetry'
GOTMPDIR=''
GOTOOLCHAIN='local'
GOTOOLDIR='/usr/local/go/pkg/tool/linux_amd64'
GOVCS=''
GOVERSION='go1.25.4'
GOWORK=''
PKG_CONFIG='pkg-config'

What did you do?

I started a container on a host with cgroup v1, and the new CPU limit-aware GOMAXPROCS default is not working for me. I have CPU limit set to 1, but I still got GOMAXPROCS set to 104 (number of CPU cores in my host).

I will use the go command itself to demo the issue:

# strace go version
openat(AT_FDCWD, "/proc/self/mountinfo", O_RDONLY|O_CLOEXEC) = 3
read(3, "1384 1311 0:152 / / rw,relatime "..., 65536) = 4072
close(3)                                = 0
openat(AT_FDCWD, "/proc/self/cgroup", O_RDONLY|O_CLOEXEC) = 3
read(3, "13:perf_event:/kubepods.slice/ku"..., 65536) = 2031
close(3)                                = 0
openat(AT_FDCWD, "/sys/fs/cgroup/cpu,cpuacct/kubepods.slice/kubepods-pod9aca97da_927e_44b8_8f19_c9d64ae70e63.slice/cricontainerd-83498b51f86c442b5d587c2ef918add0cf67b937f85667ad07b2a4053d6c3472.scope/cpu.cfs_quota_us", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)

# cat /proc/self/mountinfo | grep cgroup | grep cpu
1393 1391 0:30 /kubepods.slice/kubepods-pod9aca97da_927e_44b8_8f19_c9d64ae70e63.slice/cri-containerd-83498b51f86c442b5d587c2ef918add0cf67b937f85667ad07b2a4053d6c3472.scope /sys/fs/cgroup/cpu,cpuacct ro,nosuid,nodev,noexec,relatime master:6 - cgroup cgroup rw,cpu,cpuacct
1400 1391 0:37 /kubepods.slice/kubepods-pod9aca97da_927e_44b8_8f19_c9d64ae70e63.slice/cri-containerd-83498b51f86c442b5d587c2ef918add0cf67b937f85667ad07b2a4053d6c3472.scope /sys/fs/cgroup/cpuset ro,nosuid,nodev,noexec,relatime master:13 - cgroup cgroup rw,cpuset

root@simple:/go# cat /proc/self/cgroup | grep cpu
9:cpuset:/kubepods.slice/kubepods-pod9aca97da_927e_44b8_8f19_c9d64ae70e63.slice/cri-containerd-83498b51f86c442b5d587c2ef918add0cf67b937f85667ad07b2a4053d6c3472.scope
2:cpu,cpuacct:/kubepods.slice/kubepods-pod9aca97da_927e_44b8_8f19_c9d64ae70e63.slice/cri-containerd-83498b51f86c442b5d587c2ef918add0cf67b937f85667ad07b2a4053d6c3472.scope

What did you see happen?

Go runtime is reading /sys/fs/cgroup/cpu,cpuacct/kubepods.slice/kubepods-pod9aca97da_927e_44b8_8f19_c9d64ae70e63.slice/cricontainerd-83498b51f86c442b5d587c2ef918add0cf67b937f85667ad07b2a4053d6c3472.scope/cpu.cfs_quota_us

And it failed with ENOENT

What did you expect to see?

Go runtime should read /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us

# cat /sys/fs/cgroup/cpu,cpuacct/cpu.cfs_quota_us
100000

We should cut off the prefix (as read from /proc/self/mountinfo) from /proc/self/cgroup.

This issue exists on any env where the 4th field of mountinfo is not /. And this is the default with cgroup v1, where group namespace is usually not used. I can propose a fix for this.

[gabyhelp]

Related Issues

• runtime: cgroup GOMAXPROCS set to 1 instead of 2 #74283
• runtime: CPU limit-aware GOMAXPROCS default #73193 (closed)
• runtime: make `GOMAXPROCS` cfs-aware on `GOOS=linux` #33803 (closed)

_{(Emoji vote if this was helpful or unhelpful; more detailed feedback welcome in this discussion.)}

[prattmic]

Thanks for the report. I agree this sounds like a bug. I had not seen cgroups mounted like this. If you would like to send the fix, please do!

[gopherbot]

Change https://go.dev/cl/723241 mentions this issue: internal/runtime/cgroup: fix path on non-root mount point

[prattmic]

We did not (and still do not) have a test for this case. Our testing for container-aware GOMAXPROCS in general is more rudimentary than I'd like because (a) we're not going to be able to run full large tools like Kubernetes in tests to directly verify their functionality, and (b) our CI all runs without supervisor privileges, so we are mostly limited to testing cgroup v2 on hosts that allow unprivileged cgroup v2 creation.

On the other hand, I assume the Kubernetes project has good support for spinning up arbitrary pods with various cgroup configurations in tests, so perhaps Kubernetes itself could add some tests that verify this functionality works as expected in various Kubernetes configurations? The test itself is straightforward: start a pod with a CPU limit and verify that runtime.GOMAXPROCS(0) matches the limit. That is essentially what https://cs.opensource.google/go/go/+/master:src/runtime/cgroup_linux_test.go does, but with manual cgroup manipulation.

[huww98]

Kubernetes is also removing support for cgroup v1 now :( https://kubernetes.io/blog/2025/11/26/kubernetes-v1-35-sneak-peek/