Draft: Create host directories when mounting container volumes

[olemorud]

Relying on Docker to create hostpath when using the flag -v hostpath:containerpath is undocumented behavior and is not compatible with other containerization tools that people might want to use (even if they aren't explicitly supported).

[ChristopherHX]

This is not a review, I'm not part of this repository

But it is documented here by docker: https://docs.docker.com/engine/reference/commandline/run/#volume

When the host directory of a bind-mounted volume doesn’t exist, Docker automatically creates this directory on the host for you. In the example above, Docker creates the /doesnt/exist folder before starting your container.

You can use https://github.com/actions/runner-container-hooks to customize how containers are managed by the actions/runner while keeping actions/runner unchanged.

For example you have to already provide a symlink/wrapper or it wouldn't even try to use your unspecified container runtime.

[olemorud]

@ChristopherHX

But it is documented here by docker: https://docs.docker.com/engine/reference/commandline/run/#volume

When the host directory of a bind-mounted volume doesn’t exist, Docker automatically creates this directory on the host for you. In the example above, Docker creates the /doesnt/exist folder before starting your container.

You are right 😅 Thanks for pointing this out.

Creating directories is not supported by Podman however (by design), making the setup of self-hosted runners on any RHEL-derivative more painful. Podman is also rootless by default and is a much safer option for most self-hosted runners. This PR fixes the only annoying compatibility issue without introducing any breaking changes for Docker users (directories will still be created as usual)

You can use https://github.com/actions/runner-container-hooks to customize how containers are managed by the actions/runner while keeping actions/runner unchanged.

For example you have to already provide a symlink/wrapper or it wouldn't even try to use your unspecified container runtime.

You can just symlink /usr/bin/docker to whatever container runtime you want as long as the interface is compatible and not touch hooks at all. In this case you need to symlink it to a wrapper script that creates the missing directories before passing the arguments onto Podman, which is a trivial but annoying workaround (around 15 lines of python).

I am not sure how this would be easier to implement in a run_container_step hook, where in the examples it seems that you have to write and transpile several hundred lines of typescript to modify the container behavior.

[ChristopherHX]

I am not sure how this would be easier to implement in a run_container_step hook, where in the examples it seems that you have to write and transpile several hundred lines of typescript to modify the container behavior.

Easier to maintain (as long this pr is not merged), you can keep runner auto updates enabled and you don't have to recompile the typescript code every month. Other way around you have 1 month time to recompile an actions/runner fork using --disableupdate, if you don't use the flag all changes are reverted.
For example I import my self signed cerificates to every job container in runner container hooks.

You only need to write some lines for runner container hooks, a loop which does mkdir

a wrapper script that creates the missing directories before passing the arguments onto Podman, which is a trivial but annoying workaround (around 15 lines of python).

Well this is another way, but container hooks don't need to parse cli parameters. They get structured data, again if you prefer python over nodejs then is python the more natural way to create a wrapper instead of a hook. Hooks use nodejs, because node is an existing runtime dependency.

[nathanweeks]

I've temporarily coded around this in the runner by overwriting the /usr/bin/docker script (installed by podman-docker, which internally calls podman) with a version that adds the relevant mkdir commands, though ideally each development team that uses rootless podman runners for enhanced security wouldn't have to independently discover and code around this issue. Creating the non-existent host directories beforehand as proposed by this PR seems like a reasonable solution that shouldn't cause issues with runners that use real Docker.

The docker -v semantics of creating any non-existent host directories were recognized as a misfeature and briefly deprecated, but subsequently un-deprecated to preserve backwards compatibility, though it was at least not carried forward to the newer--mount type=bind syntax (moby/moby#13121).

The podman project maintainers have declared their intent not to emulate docker -v just because GitHub Actions expects it (containers/podman#6234 (comment)).