Enable veth and bridges on diskless boots.

[eldondev]

Enable veth and bridges on diskless boots.

Previously a rootless boot would allow systemd-networkd to manage unexpected interfaces, and might interfere with the expected operation of bridges, flannel, and veth interfaces.

How to use

Boot flatcar via ipxe or via qemu directly with kernel + initrd, but no root kcmdline

Testing done

Install k3s and observe services to start successfully.

[eldondev]

Relevant mention of the possibility of this future work here

[eldondev]

Thoughts @dongsupark ?

[tormath1]

Hey, sorry this has slipped. I am not following the change here, what was your issue? Because veth is ignored here. Sorry again about the delay.

[eldondev]

Hey, sorry this has slipped. I am not following the change here, what was your issue? Because veth is ignored here. Sorry again about the delay.

Hi, by default it doesn't seem to me that yy-pxe.network ignores veth devices. I have booted a machine via ipxe (in this case, no root= parameter on the kernel command line). When I try to start a docker container, I get the following set of log messages, and no container is created:

Jul 23 11:12:12 localhost kernel: docker0: port 1(vethe354305) entered blocking state
Jul 23 11:12:12 localhost kernel: docker0: port 1(vethe354305) entered disabled state
Jul 23 11:12:12 localhost kernel: device vethe354305 entered promiscuous mode
Jul 23 11:12:12 localhost systemd[1]: var-lib-docker-overlay2-ce13c7ef102e01262194df0827ee0ffefbfdecd209e5a148799c2c18f837e0b9\x2dinit-merged.mount: Deactivated successfully.
Jul 23 11:12:12 localhost systemd-networkd[1273]: vethe354305: Link UP
Jul 23 11:12:12 localhost systemd-networkd[1273]: veth2d6501c: Configuring with /usr/lib/systemd/network/yy-pxe.network.
Jul 23 11:12:12 localhost systemd-networkd[1273]: veth2d6501c: Link UP
Jul 23 11:12:12 localhost kernel: IPv6: ADDRCONF(NETDEV_CHANGE): veth2d6501c: link becomes ready
Jul 23 11:12:12 localhost kernel: IPv6: ADDRCONF(NETDEV_CHANGE): vethe354305: link becomes ready
Jul 23 11:12:12 localhost kernel: docker0: port 1(vethe354305) entered blocking state
Jul 23 11:12:12 localhost kernel: docker0: port 1(vethe354305) entered forwarding state
Jul 23 11:12:12 localhost systemd-networkd[1273]: veth2d6501c: Gained carrier
Jul 23 11:12:12 localhost systemd-networkd[1273]: vethe354305: Gained carrier
Jul 23 11:12:12 localhost systemd-networkd[1273]: vethe354305: Configuring with /usr/lib/systemd/network/yy-pxe.network.
Jul 23 11:12:12 localhost containerd[1496]: time="2024-07-23T11:12:12.940796182Z" level=info msg="loading plugin \"io.containerd.internal.v1.shutdown\"..." runtime=io.containerd.runc.v2 type=io.containerd.internal.v1
Jul 23 11:12:12 localhost containerd[1496]: time="2024-07-23T11:12:12.940894103Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.pause\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Jul 23 11:12:12 localhost containerd[1496]: time="2024-07-23T11:12:12.940930042Z" level=info msg="loading plugin \"io.containerd.event.v1.publisher\"..." runtime=io.containerd.runc.v2 type=io.containerd.event.v1
Jul 23 11:12:12 localhost containerd[1496]: time="2024-07-23T11:12:12.940951536Z" level=info msg="loading plugin \"io.containerd.ttrpc.v1.task\"..." runtime=io.containerd.runc.v2 type=io.containerd.ttrpc.v1
Jul 23 11:12:12 localhost kernel: device vethe354305 left promiscuous mode
Jul 23 11:12:12 localhost kernel: docker0: port 1(vethe354305) entered disabled state
Jul 23 11:12:12 localhost systemd-networkd[1273]: docker0: Gained carrier
Jul 23 11:12:12 localhost systemd-networkd[1273]: vethe354305: Link UP
Jul 23 11:12:12 localhost systemd-networkd[1273]: vethe354305: Gained carrier
Jul 23 11:12:12 localhost systemd[1]: Started docker-a9715b603497de7f3a7bebf2c8bd1a889ad13666c321ae8718c51df0554a3445.scope - libcontainer container a9715b603497de7f3a7bebf2c8bd1a889ad13666c321ae8718c51df0554a3445.
Jul 23 11:12:12 localhost systemd-networkd[1273]: vethe354305: Configuring with /usr/lib/systemd/network/yy-pxe.network.
Jul 23 11:12:13 localhost systemd-networkd[1273]: veth2d6501c: Link DOWN
Jul 23 11:12:13 localhost systemd-networkd[1273]: veth2d6501c: Lost carrier
Jul 23 11:12:13 localhost kernel: eth0: renamed from veth2d6501c

As you can see, docker attempts to configure the network, then NetworkManager also attempts to configure the network, and the container fails to start. I think yy-ipxe is intended to cast a broad net to configure network devices via DHCP, but the bycatch is wireguard, veth, and bridge interfaces which are extremely unlikely to be the things we want to get dhcp from.

[eldondev]

If you want me to make a qemu invocation without root= kernel arg to help reproduce, or other questions, please let me know, thanks!