docker-container: place build containers in a separate cgroup

[djs55]

Previously build containers created by the docker-container driver were in the default parent cgroup, along with non-build containers. This made it hard to apply resource limits to all the builds on a machine.

This PR adds a --driver-opt cgroup-parent=CGROUP to allow the cgroup to be customised. A default value of /docker/buildx is set.

[thaJeztah]

I noticed that the buildkit code embedded in dockerd also does some setting up of a parent cgroup (originally added (extracted to a function later) in moby/moby@d52485c);

https://github.com/moby/moby/blob/c4040417b6fe21911dc7ab5e57db27519dd44a6a/cmd/dockerd/daemon.go#L293

	cgroupParent := newCgroupParent(config)

https://github.com/moby/moby/blob/2b70006e3bfa492b8641ff443493983d832955f4/cmd/dockerd/daemon_unix.go#L125

func newCgroupParent(config *config.Config) string {
	cgroupParent := "docker"
	useSystemd := daemon.UsingSystemd(config)
	if useSystemd {
		cgroupParent = "system.slice"
	}
	if config.CgroupParent != "" {
		cgroupParent = config.CgroupParent
	}
	if useSystemd {
		cgroupParent = cgroupParent + ":" + "docker" + ":"
	}
	return cgroupParent
}

• Are those used for the same purpose? (or is the above only used for containers used during build? (I think it may be only for build-containers, and therefore is set to match dockerd --cgroup-parent option)
• The code above looks to be taking systemd cgroups into account; does that mean we need to set a different default depending on wether the daemon that runs the container uses systemd or not?

For the last bullet, I'm looking at the daemon code that handles (custom) parent cgroups for containers, which seems to confirm that; https://github.com/moby/moby/blob/306fa44b7ca59282dc8695e6d169c5b25698d0cb/daemon/daemon_unix.go#L709-L714

if hostConfig.CgroupParent != "" && UsingSystemd(daemon.configStore) {
	// CgroupParent for systemd cgroup should be named as "xxx.slice"
	if len(hostConfig.CgroupParent) <= 6 || !strings.HasSuffix(hostConfig.CgroupParent, ".slice") {
		return warnings, fmt.Errorf("cgroup-parent for systemd cgroup should be a valid slice named as \"xxx.slice\"")
	}
}

Wondering how to solve that

• buildx could inspect the daemon's configuration before creating the container (and use format "A" or "B")
• some special value (specific for the buildx builder case) to pass, and have the daemon perform the check and pick the right format to use
• (not sure if possible) some "uniform" format that the client can use, and the daemon to convert it to the right format for the situation (this would benefit "any" container, but A) not sure if possible, and B) not sure if it's useful; setting a custom parent cgroup usually would mean that that cgroup has to be created up front?)

[thaJeztah]

/cc @AkihiroSuda (perhaps you have some thoughts on the above)

[AkihiroSuda]

Probably, this should be set only when d.DockerAPI.Info().CgroupDriver == “cgroupfs”

[djs55]

Thanks, I've pushed an update which gates the setting on CgroupDriver == "cgroupfs" so hopefully it doesn't break anything if using systemd. I must admit I'm not very familiar with systemd, I need to spend some time playing with it.