Adding k3s example

[javydekoning]

Added an k3s example as Truenas Scale is moving away from k3s in favor of Docker. Source: https://forums.truenas.com/t/the-future-of-electric-eel-and-apps

Closes #191

[Jip-Hop]

Thanks! Looks good at first sight. Didn't have time to test this yet. Would you be able to clean up the mentions of docker inside the readme and config file? And I didn't see apt-get update in the initial setup script. Isn't that required? Good to see you documented the reason for adding perf event open syscall. But by default no intel gpu is passed through and Intel GPU top command is also not available by default. Would it make sense to exclude the perf event open syscall by default?

[javydekoning]

I don't think apt update is needed but it doesn't have any drawbacks so I've added it for completeness.

Addressed your other comments as well.

[Jip-Hop]

Thanks! Looks better :)

WARNING : apt does not have a stable CLI interface. Use with caution in scripts

Could you please use apt-get instead of apt?

Would you mind explaining why allowed-unsafe-sysctls=net.ipv4.conf.* is required?

[javydekoning]

Ah I missed that. allowed-unsafe-sysctls=net.ipv4.conf.* is needed only if you run things like WireGuard within k3s. I can remove it. Yes, I can use apt-get instead as well.

[Jip-Hop]

Thanks! I'll try it out this weekend :)

[Jip-Hop]

Cool, the config template works! :D

I still have doubts about the inclusion of kubecolor though. The installed version 0.0.20 is quite some versions behind the latest release v0.3.3. And if it is installed by default I think it would be nice if it were used by default too, by adding the alias kubectl="kubecolor" to the bash profile.

What do you think?

[Jip-Hop]

I noticed shutting down the k3s jail takes very long and found these lines in the journal:

Jun 14 10:09:20 k3s systemd[1]: cri-containerd-c235f3711f3e962b060100a304e5d65cc105e90eeeafddeb5115ea9650602ff7.scope: Stopping timed out. Killing.
Jun 14 10:09:20 k3s systemd[1]: cri-containerd-c235f3711f3e962b060100a304e5d65cc105e90eeeafddeb5115ea9650602ff7.scope: Killing process 2345 (local-path-prov) with signal SIGKILL.
Jun 14 10:09:20 k3s systemd[1]: cri-containerd-c235f3711f3e962b060100a304e5d65cc105e90eeeafddeb5115ea9650602ff7.scope: Killing process 2359 (local-path-prov) with signal SIGKILL.

Do you see the same behavior? I'm using bridge networking instead of macvlan and I had hello world deployment running.

[javydekoning]

Yes, I could add a newer version of Kubecolor (from source) but it would require also installing a bunch of other large packages (like golang) and making changes to PATH. In my install I do add the alias to .zshrc, change the shell and add kubectl completion but all of this will make it a very opinionated install. It may be better to just remove it all together.

With regards to the slow stop, that has to do with Daemon sets needing to be killed. There is no way to gracefully cordon a host if you do a single node control plane/worker mixed install.

[Jip-Hop]

require also installing a bunch of other large packages (like golang) and making changes to PATH

The latest kubecolor binary is available from GitHub releases but I agree it's better not to make this config template too opinionated.

[Jip-Hop]

Thanks!