Using Ignition, files created under /run and /tmp are not present when boot completes

[dlipovetsky]

Let's say that I create the files /etc/test, /tmp/test, and /run/test in my ignition config. When the boot completes, the only file present is /etc/test.

I think I understand why /tmp/test is not present; systemd mounts /tmp after ignition completes, so whatever was in the directory /tmp is not visible when the mount is created.

However, I really don't understand why /run/test is not present. By comparison, ignition's own cache in the file /run/ignition.json is present when the boot completes.

Does anyone have some insight into this? Thanks!

I'm using flatcar 3033.3.15, the latest LTS release at this time. For reference, here's the config.ign. It's valid, according to ignition v0.36.2:

{
    "ignition": {
        "config": {},
        "security": {},
        "timeouts": {},
        "version": "2.3.0"
    },
    "networkd": {},
    "passwd": {},
    "storage": {
        "files": [
            {
                "filesystem": "root",
                "path": "/run/test",
                "contents": {
                    "source": "data:,test%0A",
                    "verification": {}
                },
                "mode": 448
            },
            {
                "filesystem": "root",
                "path": "/tmp/test",
                "contents": {
                    "source": "data:,test%0A",
                    "verification": {}
                },
                "mode": 448
            },
            {
                "filesystem": "root",
                "path": "/etc/test",
                "contents": {
                    "source": "data:,test%0A",
                    "verification": {}
                },
                "mode": 448
            }
        ]
    }
}

[pothos]

Hi, I think it doesn't work because these folders are not bind-mounted under /sysroot/ in that stage. You can create the files by declaring systemd-tmpfiles configs: https://www.freedesktop.org/software/systemd/man/tmpfiles.d.html#

[dlipovetsky]

I'll try to make my previous comment a little more clear:

Ignition writes the file /run/ignition.json whenever it runs:
https://github.com/flatcar/ignition/blob/6d8ad3f39728bcab4b982e21594043bf3bb52d2b/internal/exec/engine.go#L174

The files I'm looking for are written during by Ignition's files stage, which runs before systemd "switches root": https://github.com/flatcar/bootengine//blob/589e6bec78718a25eaccc46f25d2c59763b91327/dracut/30ignition/ignition-files.service#L30

If the file /run/ignition.json is present after boot, why are the other files not present? 🤔

[pothos]

Your files are not created under the initrd's /run because the mount point is different.

[dlipovetsky]

Your files are not created under the initrd's /run because the mount point is different.

Thanks.

(I hope I've understood this correctly: Ignition writes its config cache directly to /run, but writes a file specified in its config, e.g. /run/foo to /sysroot/run. Afterward, when the switch_root operation runs, it switches /sysroot to /. The various bind mounts, among them /run, effectively hide any files that were present under the directory that is a mount point. So, a file that is at /sysroot/run/foo is not at /run/foo after the switch_root.)

Looks like the manpage for switch_root states this best:

switch_root moves already mounted /proc, /dev, /sys and /run to newroot and makes newroot the new root filesystem and starts init process.

The user-defined files in the ignition config are written to newroot, and therefore any files under newroot/run are not accessible after switch_root moves the /run mount to newroot.

The config cache is written directly to /run/ignition.json, and so it remains available when the /run mount is moved to newroot.

[dlipovetsky]

I feel like it's too easy to make the mistake of creating files that won't be accessible when the boot finishes.

Which, if any, of these would be help?

(a) Note in the Ignition docs that a file written to a path that becomes a mount point, e.g., /run/foo, is not going to be present at boot.

(b) Modify ignition-validate to issue a warning or error for affected file paths.

[dlipovetsky]

(a) I filed coreos/ignition#1697.

(b) Looks like ignition-validate validates the config structure, not the values, so I'm not sure if this check belongs 🤔