Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Kubernetes support: To work by default, ship a symlink in /usr/libexec #1193

Closed
pothos opened this issue Sep 29, 2023 · 7 comments
Closed

Kubernetes support: To work by default, ship a symlink in /usr/libexec #1193

pothos opened this issue Sep 29, 2023 · 7 comments
Assignees
@pothos
Labels
kind/feature A feature request

Comments

@pothos
Copy link
Member

pothos commented Sep 29, 2023

Current situation

Kubernetes tries to write files to /usr/libexec/kubernetes/kubelet-plugins/volume/exec/ which fails on Flatcar.
Workarounds were bind mounts or a sysext with a symlink.

Impact

Does not work by default.

Ideal future situation

/usr/libexec/kubernetes/kubelet-plugins/volume/exec is a symlink to /var/kubernetes/kubelet-plugins/volume/exec in the generic image. A tmpfile rule creates the symlink during image build time and also the target at runtime.
@pothos pothos added the kind/feature A feature request label Sep 29, 2023
pothos added a commit to flatcar/sysext-bakery that referenced this issue Sep 29, 2023
@pothos
Kubernetes: Correct symlink creation
bb62ba9 
The symlink for the directory in /usr/libexec has to be created like a
file or otherwise it will land inside the existing directory (which
shouldn't have existed).
Fix the symlink creation. A better solution would be
flatcar/Flatcar#1193 but that's for future
releases.
@pothos pothos mentioned this issue Sep 29, 2023
Merged
@pothos pothos self-assigned this Oct 9, 2023
@pothos
Copy link
Member Author

pothos commented Oct 9, 2023

PR: flatcar/scripts#1238

@tormath1
Copy link
Contributor

tormath1 commented Oct 9, 2023

with this we might be able to get rid of https://github.com/flatcar/mantle/blob/fbb5dee5e8e610cec56e7040c74bf2c6c1e3096a/kola/tests/kubeadm/templates.go#L365

@till
Copy link

till commented Oct 10, 2023

@pothos How would I use your workaround for my setup (now)? I need to move /usr/libexec/k0s somewhere where it's writable.

@pothos
Copy link
Member Author

pothos commented Oct 10, 2023

You can ship the symlink with a systemd-sysext folder (or .raw image) like done here https://github.com/flatcar/sysext-bakery/blob/main/create_kubernetes_sysext.sh#L85
Easiest is to create a folder /etc/extensions/k8s-compat/usr/… with the symlink in it and add that os-release file, too. Then create the target folder under /var and run systemd-sysext refresh.

@pothos pothos closed this as completed Oct 10, 2023
@till
Copy link

till commented Oct 10, 2023

@pothos is there a way without using systemd-sysext? I didn't want to go down that rabbithole 🐇 🕳️ right now

@pothos
Copy link
Member Author

pothos commented Oct 10, 2023

with this we might be able to get rid of https://github.com/flatcar/mantle/blob/fbb5dee5e8e610cec56e7040c74bf2c6c1e3096a/kola/tests/kubeadm/templates.go#L365

This is how you can configure an own path in Kubernetes

@till
Copy link

till commented Oct 10, 2023

Ok, doing something like that. Just figured I could maybe do this easier. Thank you!

@github-actions github-actions bot mentioned this issue Nov 9, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@pothos pothos

Labels
kind/feature A feature request
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@till @pothos @tormath1