Skip to content

runtime: zero cgroup CPU period yields NaN GOMAXPROCS and fatal "procresize: invalid arg" at startup #80783

Description

@alex-fall-bindplane

Go version

go1.25.7, go1.25.9, go1.26.4, and current master (the unguarded division is still present at master).

go1.23.8 and go1.24.2 are unaffected, since cgroup-aware GOMAXPROCS was added in Go 1.25.

Output of go env in your module/workspace

Cross-compiled GOOS=linux GOARCH=amd64 CGO_ENABLED=0 from a darwin/arm64 host. The defect is in the target runtime rather than the build host; a natively built linux/amd64 binary behaves the same.

What did you do?

Ran a trivial linux/amd64 Go binary on a RHEL/CentOS 6 host (kernel 2.6.32-754.el6.x86_64) with the cgroup v1 cpu controller mounted.

Red Hat backported CFS bandwidth control into their 2.6.32 kernel (CONFIG_CFS_BANDWIDTH=y), but the root cgroup reports:

cpu.cfs_quota_us  = 0
cpu.cfs_period_us = 0

Upstream kernels and EL7+ report -1 / 100000 here, so this does not reproduce on a normal modern system.

The program is just:

package main

import (
	"fmt"
	"runtime"
)

func main() {
	fmt.Printf("OK %s NumCPU=%d GOMAXPROCS=%d\n",
		runtime.Version(), runtime.NumCPU(), runtime.GOMAXPROCS(0))
}

What did you see happen?

It aborts before main():

fatal error: procresize: invalid arg

runtime stack:
runtime.throw({0x4c0bf2?, 0x0?})
	/usr/local/go/src/runtime/panic.go:1094 +0x48 fp=0x7ffd1d631b40 sp=0x7ffd1d631b10 pc=0x46dd28
runtime.procresize(0x0?)
	/usr/local/go/src/runtime/proc.go:5872 +0x9b6 fp=0x7ffd1d631c10 sp=0x7ffd1d631b40 pc=0x44a9b6
runtime.schedinit()
	/usr/local/go/src/runtime/proc.go:935 +0x3b4 fp=0x7ffd1d631cb0 sp=0x7ffd1d631c10 pc=0x43f974
runtime.rt0_go()
	/usr/local/go/src/runtime/asm_amd64.s:349 +0x11c fp=0x7ffd1d631cb8 sp=0x7ffd1d631cb0 pc=0x47209c

The same kernel with the cpu cgroup controller not mounted runs the binary fine, which is what isolates this to the cgroup path.

What did you expect to see?

The program to run. A zero (or otherwise unusable) period should be treated as "no limit", not turned into a negative GOMAXPROCS.

Analysis

src/internal/runtime/cgroup/cgroup_linux.go, ReadCPULimit, v1 branch:

if quota < 0 {
	// No limit.
	return 0, false, nil
}

period, err := readV1Number(c.periodFD)
if err != nil {
	return 0, false, errMalformedFile
}

return float64(quota) / float64(period), true, nil

quota == 0 is not < 0, so the no-limit early return is skipped, and there is no guard on period == 0. The division is 0/0, which is NaN.

That NaN then flows through adjustCgroupGOMAXPROCS in src/runtime/cgroup_linux.go:

limit = ceil(limit)      // NaN
limit = max(limit, 2)    // NaN — the builtin max propagates NaN
if int32(limit) < procs {
	procs = int32(limit)
}

Converting NaN to int32 on amd64 gives -2147483648, which compares less than the CPU count, so procs becomes negative. schedinit passes it to procresize, which throws on nprocs <= 0.

Measured on linux/amd64:

quota/period      = NaN
ceil()            = NaN
max(limit, 2)     = NaN
int32(limit)      = -2147483648

The cgroup v2 path in src/internal/runtime/cgroup/cgroup.go (parseV2Limit) ends in the same unguarded float64(quota) / float64(period), so a 0 period there would produce the same NaN. I have not observed a kernel writing that to cpu.max, but the guard is missing in both paths.

Suggested fix

Treat a non-positive period as "no limit" in both ReadCPULimit (v1) and parseV2Limit (v2), returning (0, false, nil). Defending adjustCgroupGOMAXPROCS against a non-finite limit would also stop any future parse bug from turning into a startup crash rather than a degraded GOMAXPROCS.

Workarounds

Either of these avoids the crash (both verified on go1.26.4 on the affected kernel):

  • GODEBUG=containermaxprocs=0 — the NaN is still computed for the GODEBUG counter but is discarded, and the real CPU count is used.
  • Setting GOMAXPROCS to a positive integer, which makes schedinit take the env branch and never call defaultGOMAXPROCS.

Possibly related

#77374 is the same panic and stack trace on the same kernel version. It was closed by the reporter without a diagnosis, so the underlying cause was never recorded.

Reproduction environment

QEMU qemu-system-x86_64, the stock kernel-2.6.32-754.el6.x86_64 from vault.centos.org, and a busybox initramfs that mounts /proc, /sys, and mount -t cgroup -o cpu cgroup /cgroup/cpu. Happy to share the exact rig if useful.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

NeedsInvestigationSomeone must examine and confirm this is a valid issue and not a duplicate of an existing one.compiler/runtimeIssues related to the Go compiler and/or runtime.

Type

No type

Projects

Relationships

None yet

Development

No branches or pull requests

Issue actions