Running `createContainer` hooks with readonly rootfs

[elezar]

In the NVIDIA Container Toolkit we use a number of createContainer hooks to perform required modifications on the container so that it can have access to requested GPUs. These include creating symlinks based on the driver libraries injected from the host, or updating the ldcache in the container.

It has recently been pointed out that these hooks fail when running a container with a readonly rootfs. This is note the case when runc is used and this seems to come from the fact that runc remounts the rootfs as readonly AFTER the createContainer hooks are run, whearas crun does this BEFORE the hooks are run.

In crun, the rootfs is set as readonly here (or more specifically in libcrun_set_mounts) which is before the createContainer hooks are run. In runc this is not the case, the rootfs is remounted readonly after the create container hooks are run in the prepareRoot call.

Since the OCI runtime spec only specifies:

readonly (bool, OPTIONAL) If true then the root filesystem MUST be read-only inside the container, defaults to false.

it is not clear on the timing w.r.t to the various hooks.

Is there a specific reason for the difference in ordering of these operations?

[giuseppe]

no, there is no real reason for that, opened a PR:

• linux: run createContainer hooks before making root RO #1925