Enabling cgroup v2 mode on initial boot with Alpine image in Lima

[na0x2c6]

Hello, thank you always for the fantastic development.

I am setting up a rootless podman environment using the alpine image. To check stats information through podman stats, it's necessary to utilize cgroup v2. However, the default mode for alpine is hybrid, and thus it cannot be utilized as is.

I have figured out that in alpine using OpenRC, by writing rc_cgroup_mode=unified in /etc/conf.d/cgroups, we can utilize cgroup v2. However, to apply the cgroup mode to the machine, a reboot is necessary.

Currently, it seems that when utilizing alpine with lima, on the initial boot rc_cgroup_mode=hybrid mode is enforced. This is because the cgroup service is started in 10-alpine-prep.sh, and also, it seems that operations such as updating /etc/* before this boot process are not permitted.

Although there is no issue if the machine is rebooted now, is there no way to apply cgroup v2 mode from the initial boot? If not, would it be reasonable to request this feature?

I tried setting a file with provision.[n].mode: "boot" in the lima template, but as far as I understand, this is supported only in cloud-init. Is it not possible to use such a mode in alpine too?

Thanks.

[jandubois]

The only way I know how to do this without the reboot is to include the modified /etc/* files in the apkovl tarball and have it extracted during boot. I did this once for Rancher Desktop, but reverted it because it caused various other problem. This is still under investigation (although currently on hold due to time constraints).

It may be possible to make it configurable via cloud-init, but that will also require additional work (we use lima-init instead of cloud-init, and only support a small subset of the cloud-init functionality). Right now you would need to build a separate Alpine image with these changes.

[na0x2c6]

@jandubois
Thank you for the prompt response.

I understand the situation. I've tried to read through the Lima code as much as possible and noticed that the code utilizing provision.[n].mode: "boot" is specialized for updating user-data. Even if work to support Alpine is carried out, considering the alignment with the code for cloud-init, it seems like the benefits might not justify the amount of work involved, so I will consider resolving this on my own.

Right now you would need to build a separate Alpine image with these changes.

Thank you. I have attempted a build before, so it seems doable. I will give it a try.

[jandubois]

You can see the change I made at rancher-sandbox/alpine-lima#18.

It modifies /etc/rc.conf, but your idea of creating /etc/conf.d/cgroups would be better.

Anyways, you can use the images from https://github.com/rancher-sandbox/alpine-lima/releases/tag/v0.2.31.rd5 to test it:

$ l start --tty=false --set '.images=[{"location":"https://github.com/rancher-sandbox/alpine-lima/releases/download/v0.2.31.rd5/alpine-lima-rd-3.18.0-x86_64.iso", "arch": "x86_64"}]' template://alpine
...
$ l shell alpine ls /sys/fs/cgroup
acpid                   cgroup.procs            cpu.stat                lima-guestagent
cgroup.controllers      cgroup.stat             cpuset.cpus.effective   memory.reclaim
cgroup.max.depth        cgroup.subtree_control  cpuset.mems.effective   memory.stat
cgroup.max.descendants  cgroup.threads          io.stat                 sshd

This is the rd edition though; you may want to build your own std edition to avoid including the additional Rancher Desktop requirements, but at least you can try out the image to see if that change works for you or not.

Note that this change did not fix kind, broke k3d, and also was incompatible with older versions of k3s, which is why we reverted it for Rancher Desktop. I believe WSL2 went through a similar cycle of attempting to move to the unified layout, and had to (temporarily) revert to hybrid due to various issues.

[na0x2c6]

@jandubois
Thank you very much for the detailed response.

With your help, I was able to build a custom image and now can use unified mode from the initial boot.

Diff: na0x2c6/alpine-lima@v0.2.32...v0.2.32-std-cgroup-experimental
Custom Image: https://github.com/na0x2c6/alpine-lima/releases/tag/v0.2.32-std-cgroup-experimental
PoC Template: https://github.com/na0x2c6/lima-podman/blob/397849f53be696d74ff1fc75b3d6a17b7da58fec/podman.yaml

The above allows for changing the mode using the environment variable LIMA_ALPINE_CGROUP_MODE at machine startup. If I were to create a Pull Request for this towards the alpine-lima main body, would it be reasonable?