cgroups v2 Support

[crosbymichael]

Work needs to be done to the cgroups lib and containerd metrics interfaces to support cgroups v2 support. With much of the work in 5.3 kernels this should be reasonable to start supporting as a first class feature and can be a replacement for v1 for some users.

runc recently gained support for v2 as well as crun.

---

(below are edited by maintainers)

runc checklist

See opencontainers/runc#2315

containerd checklist

• base support cgroup2 #3799
• PID metrics support cgroup2 #3799
• CPU metrics
• Memory metrics
• OOM handler: cgroup2: implement containerd.events.TaskOOM event #4273
• IO metrics: Added IO metrics for cgroup v2 #3919

containerd/CRI checklist

• metrics: Cgroupv2: Added CPU, Memory metrics  cri#1376
• enable cgroup namespace by default: cgroup2: unshare cgroup namespace for containers cri#1371
• enable cgroup namespace for pod sandboxes as well as for containers? (To be determined)
• disable cgroup namespace for privileged pods (discussed in add KEP for cgroups v2 support kubernetes/enhancements#1370): cgroup2: do not unshare cgroup namespace for privileged cri#1415
• systemd cgroup driver for rootless (seems working without additional work: https://github.com/AkihiroSuda/critest-rootless-cgroup2)

CI

• CI (crun): cgroup2 CI #4279
• CI (runc): cgroup2 CI: add RUNC_FLAVOR=runc #4408
• CI for rootless mode with systemd (will be a follow-up PR)

[dnoliver]

Fedora 31 ships with CGroups v2 by default, and Docker (moby-engine, docker-ce) does not work by default (you need to add this kernel command line arguments systemd.unified_cgroup_hierarchy=0)

Workaround seems to work fine (for now)

[crosbymichael]

The majority of the work will have to be done in runc to get v2 support working fully. It as the start for cgroupv2 support but still lacks devices support via eBPF

[crosbymichael]

I'm looking into this today and trying to compile a list of all the tasks that need to be completed.

[dnoliver]

For reference, the Docker/CGroupV2 issue for Fedora 31 is tracked in https://bugzilla.redhat.com/show_bug.cgi?id=1746355

[AkihiroSuda]

attempting to write eBPF device controller
https://github.com/AkihiroSuda/runc/commits/ebpf

(EDIT: now PR is ready: opencontainers/runc#2145)

[AkihiroSuda]

Do we support cgroup2 only for shim v2 runc v2?

If Moby cannot migrate to shim v2 (moby/moby#38943) soon, we also need to support shim v1?

[AkihiroSuda]

RFC for containerd/cgroups package: containerd/cgroups#102

[crosbymichael]

I think v2 shims. I'll work on docker to use the v2 shims by default as enough time has past and we are not supporting v1 shims as much anymore.

[AkihiroSuda]

PR #3799

[AkihiroSuda]

added the checklist to the top comment of this issue
cc @Zyqsempai

[Zyqsempai]

I will take CPU and Memory metrics.

[AkihiroSuda]

The base parts are merged now. (#3799)

We need to add CPU/Memory/IO metrics and OOM handler.
https://github.com/containerd/containerd/tree/master/metrics/cgroups/v2
Help wanted.