pod and qos level cgroup support

[derekwaynecarr]

[Kubelet] Add alpha support for `--cgroups-per-qos` using the configured `--cgroup-driver`. Disabled by default.

---

This change is

[googlebot]

We found a Contributor License Agreement for you (the sender of this pull request) and all commit authors, but as best as we can tell these commits were authored by someone else. If that's the case, please add them to this pull request and have them confirm that they're okay with these commits being contributed to Google. If we're mistaken and you did author these commits, just reply here to confirm.

[derekwaynecarr]

Picking this back up now for 1.5 release. Will start sending discrete PRs.

[derekwaynecarr]

This PR is gated on #29049

This PR is replacing the referenced item as it abstracts across cgroup drivers which required significant changes in the interim.

[derekwaynecarr]

opencontainers/runc pr changes here: opencontainers/runc#1084

[derekwaynecarr]

Recording so I do not forget:

There are two issues that need resolution around naming.

• When/how to convert the "internal" cgroup name (BestEffort/pod-xyz) to the "concrete" cgroup name that may be modified by the driver (i.e. "/BestEffort.slice/BestEffort-pod_xyz.slice")?

When constructing the sandbox config, we need to have a way to convert internal names easily to concrete names, but always keeping the linux cgroupfs style syntax. Exposing GetConcreteContainerName on PodContainerManager that does that conversion seems practical to me.

• How individual shims need to handle conversion of the concrete names to their driver specific syntax?

In this case, the dockershim would need to run docker info to see its configured cgroup driver, and then call a utility to convert that to the proper slice form in the case of systemd (i.e /BestEffort.slice/BestEffort-pod_xyz.slice) becomes BestEffort-pod_xyz.slice. That conversion can be shared in libcontainer utility.

/cc @vishh as I think this captures what we agreed upon.

[derekwaynecarr]

@dubstack @vishh -- i can manually verify pod level cgroup creation AND deletion works as expected on systemd driver now, need to do a lot more testing and cleanup, but this will be the basis for breaking this uber PR into a number of smaller pulls. either way, i was happy to see everything is getting cleaned up on the cgroupfs filesystem as expected.

[googlebot]

CLAs look good, thanks!

[derekwaynecarr]

@vishh -- this should be good.

[vishh]

---

Reviewed 9 of 30 files at r13, 1 of 10 files at r14, 4 of 11 files at r15, 20 of 20 files at r18.
Review status: all files reviewed at latest revision, 31 unresolved discussions.

---

Comments from Reviewable

[vishh]

Bumping priority since @derekwaynecarr cannot shepherd this PR beyond today until the code freeze and this PR is necessary for v1.5.

[vishh]

@derekwaynecarr Given that this feature is alpha, why not use the feature gates flag instead of adding a separate flag for cgroups-per-qos?

[vishh]

Chatted with @derekwaynecarr offline. We decided to prefix the cgroup related flags with an experimental string to better convey the support for this feature to users.

[k8s-github-robot]

@k8s-bot test this [submit-queue is verifying that this PR is safe to merge]

[k8s-github-robot]

Automatic merge from submit-queue