/ go Public
runtime: make the proportion of CPU the GC uses based on actual available CPU time and not GOMAXPROCS #59715
Issues related to the Go compiler and/or runtime.
Someone must examine and confirm this is a valid issue and not a duplicate of an existing one.
Currently the Go GC sets its parallelism as 25% of
GOMAXPROCS. In doing so, it effectively assumes the Go runtime has access to
GOMAXPROCS*Nseconds of CPU time in
Nseconds of wall time, which is true in many circumstances, but not all.
For example, if a container is configured to have 100 ms of CPU time available for each 1 second window of wall time and
GOMAXPROCS=4(because that's how much parallelism is available on the machine), then the Go runtime is effectively assuming it can use 1 second of CPU time very 1 second of wall time on GC alone.
In practice, this is not true, resulting in significant CPU throttling that can hurt latency-sensitive applications. To be more specific, throttling in general is not really a Go runtime issue. The issue is that the GC on its own is using more CPU over a given window of time than the container is actually allowed.
Another example where this can go awry is with
GOMEMLIMIT. If a container with a small CPU reservation runs up against the memory limit and the GC starts to execute frequently enough to then subsequently hit the 50% GC CPU limit, that 50% GC CPU limit is going to be based on
GOMAXPROCS. Once again, this results in significant throttling that can make an already bad latency situation worse.
One possible fix for this is to make the Go GC container-aware. Rather than set the GC CPU limit, or the 25% parallelism from
GOMAXPROCS, the Go GC could derive these numbers from the container.
On Linux, that means using the CPU cgroups parameters
cpu.cfs_quota_us. More specifically, the GC CPU limit is set to 50% of
cpu.cfs_quota_usin any given window of
cpu.cfs_period_us, while the GC's default mark-phase parallelism is 25% of
cpu.cfs_quota_us / cpu.cfs_period_us. (Disclaimer: this second part I'm less sure about.)
The Go runtime would re-read these parameters regularly, for example every few seconds, to stay up-to-date with its current environment.
The text was updated successfully, but these errors were encountered: