Evaluate kexec

[thebsdbox]

At the moment we run the tink-docker image with all the privileges to allow reboot. We need to determine the path for allowing kexec to a provisioned OS.

cc/ @mmlb

[thebsdbox]

-> https://github.com/gianarb/tinkie/blob/5585e18bfbc62a059fd8a06433b185d471e44a28/kernel/config-5.10.x-x86_64#L432

Boom,

The Tink docker image also has the kexec-tools, tested with:

 alias d="ctr -n services.linuxkit t exec -t --exec-id test docker"
d bash
mount /dev/sda3 /mnt
kexec -l /mnt/boot/vmlinuz --initramfs /mnt/boot/initramfs --append "rootfs=/dev/sda3 ro"
kexec -e

[gianarb]

I think overall, we have to figure out a clear implementation for requirements like:

• How do we reboot?
• How do we kexec?

A big concern I have is how do we define this feature to keep well-defined boundaries between components? Is this feature something that has to be implemented in the installation environment and the template definition? Does it require tink-worker to be aware of it? We should do our best to be sure that it is self-contained and easy to re-implement if somebody wants to write its own OSIE or even its own tink-worker.

I keep mentioning that somebody will want to write its own tink-worker mainly because as soon as more people pick up the project, they will want to use something that is not Docker, they will want to use Vault to authenticate to, or things like that... And it will be way easier to say: "this is the worker interface, have a look and write your own one in Rust."

[mmlb]

We were discussing this yesterday and spit balling some ideas. We thought about pipes or running cmds from tink-worker, but the one we like the most was to extend the template/workflows to provide for "super containers" or "uncontained containers" :D. tink-docker is an example of that here, we could expand the privileges/capabilities we can tweak in the template so that for example the kexec action would just be able to do it itself. No need to coordinate the feature between osie/tinkie/somethingelse since its part of the tink engine.

One thing that we need to think about though is properly finishing the workflow, succesfully if we kexec or reboot.

[thebsdbox]

This is epic !

	err := unix.KexecFileLoad(kernelFD, initrdFD, cmdline, flags)
	if err != nil {
		return err
	}
	unix.Reboot(LINUX_REBOOT_CMD_KEXEC)

[thebsdbox]

One thing that we need to think about though is properly finishing the workflow, successfully if we kexec or reboot.

Which is what was nice about dropping a file to a path.. it completes successfully and something else then reboots.

[jacobweinstock]

i'm in favor of updating the template spec to allow specifying namespaces and seccomps like they do in linuxkit. I like this because gives end-users the flexibility to run a greater amount of actions images to accomplish specific user case goals.

Just dropping this script below in as a note on the bare amount of capabilities and seccomp i was able to find to get kexec working in a container. I believe this would be similar to other things like reboot. 3 namespaces (pid, ipc, net(needed to down interfaces on reboots)), cap-add CAP_SYS_BOOT, and the seccomp profile.

# write the seccomp profile out
cat <<EOF> profile.json
{
  "names": [
    "kexec_file_load",
    "kexec_load"
  ],
  "action": "SCMP_ACT_ALLOW",
  "args": [],
  "comment": "",
  "includes": {
    "caps": [
        "CAP_SYS_BOOT"
    ]
  },
  "excludes": {}
}
EOF

# run the container
docker run -it --pid=host --ipc=host --net=host --cap-add=CAP_SYS_BOOT --security-opt="seccomp=profile.json" --volume /:/host --rm busybox chroot /host

# load and run new kernel
kexec -l /boot/vmlinuz-5.4.0-65-generic --append=root=/dev/sda1 --initrd=/boot/initrd.img-5.4.0-65-generic
kexec -e

[gianarb]

Ok! There is nothing left to do in this repository about kexec support. The action is in tinkerbell/hub as PR right now blocked by some work that has to be done in tink-worker. I am closing this issue.