Feature Request: Ability to add packages #296
Comments
|
There was a discussion about this in the rancher users slack if I recall correctly and the direction that discussion took was that it could be worth looking into if the new "toolbox" feature could be used for this instead of adding this capability into the default k3os setup or something like that. |
|
Toolbox might not be a great fit for Ansible users, because Ansible is still expecting its login shell over ssh to have a working python interpreter in the path. Toolbox simply allows a logged in user to enter a shell in a container -- we could do things like mount the root fs into the toolbox container and give folks instructions to make toolbox their login shell. Possible, and other container distros have done this, but On the OP: First, the upcoming k3os operator is intended to allow for upgrades and configuration, so hopefully that will meet the need. Ideally it would cover most use cases for fleet management that Ansible would do. Let us know where it falls short. Second, you can get limited functionality from Ansible by using its 'raw' module. This won't allow you to do things like copy files but you can run shell commands. |
|
I don't want to presume to know your use case, but we originally had the same instinct to automate with Ansible. After some discussion, we realized we could accomplish much the same thing using a Kubernetes Job (or CronJob, or DaemonSet, depending on use case), and just let the container be our package manager. Start with an appropriate base image with desired Python, Ansible, etc. packages, or roll your own. Next, add Ansible Roles and Playbooks into the image; alternately mount them from a ConfigMap, or use an init container to pull from source control, etc. for a "Function-as-a-Service" feel. At that point, Ansible may even be a little overkill, since K8s is already taking care of our remoting needs. A simple Bash script or small Go program may be a suitable substitute. |
|
My use case with Ansible is to configure both k3os and k3s. With k3os, I use Ansible to configure the OS, copy deployment files (helm charts, manifests, etc). For k3s I use Ansible to apply all the manifests. I am also planning on using AWX/Tower to manage the Helm deployments when updates are pushed (CI/CD). So for right now, I'm doing this with Alpine as the OS since I know k3os is really just Alpine Linux with k3os customization. I have been using the Ansible raw module to install the python packages and then using the full Ansible module suite from there. I would be open to hearing what others are doing to accomplish the same thing. |
k3OS is indeed based upon Alpine Linux (and hence Busybox also) but please be advised that k3OS does not ship with a package manager and I don't expect that this is something we will take on. See
Be wary of the inherent assumptions here (is what I am trying to say):
I have been noodling about how to do bolt-ons a la RancherOS, with containers, but until such is supported, if you really need ansible (and one assumes apk and other programs) then this is really an use-case for a custom build of k3OS. That said (and this has been said by others elsewhere), you probably don't actually need ansible and could likely achieve what you are trying to do with a relatively unsophisticated set of scripts delivered via container in a daemonset pod that injects relevant parts of the system into the pod containers so that those bits may be mutated. Sharp readers will see that this as an alternative way of bringing ansible to k3OS without the need for a custom build or requiring ansible/python/apk as part of the base build. |
|
I'm not so much hung up on how to automate the OS, but rather adding in flexibility into the OS. I don't know, package management seem like a trivial feature to add (or add back in if you look at it that way). I should also mention I'm doing a bare metal installation of k3os as I do like both k3s and k3os bundled. |
|
I'm currently sat here with a two cluster setup based on k3os. I'm trying to debug why my Cilium ClusherMesh is not load-balancing requests between the two clusters automatically. Cilium provides a Python application that might have been useful for diagnosis (https://docs.cilium.io/en/v1.6/troubleshooting/#automatic-diagnosis I like the philosophy of k3os being a small distribution specifically for running Kubernetes but at the same time I have appreciated the tools available to hand of a larger OS to allow me to debug problems: sometimes problems only occur in production and it can be useful to be able to ssh into servers or containers and attach GDB to a process to debug a dead-lock. I have to now think of what I need to create in order to get this Cilium python tool running and the hurdle to doing that is a rather large context switch away from the problem at hand. Having a command like "k3s debug" that perhaps spawned a container with host networking and OS visibility and downloaded an image that had all the nice debug tools like perf, tcpdump, gdb, python etc. would go a long way to easing the debug-in-production scenario. |
|
What about providing 2 images? One that contains I would much rather use the one with apk, day to day in my lab setup. |
|
if you just want to manage kubernetes resources via ansible, you can use something along these lines: - hosts: k3os
vars:
kubeconfig_file: ".kubeconfig/{{ inventory_hostname }}.yaml"
kube_api_url: "https://{{ inventory_hostname }}:6443"
environment:
K8S_AUTH_HOST: "{{ kube_api_url }}"
K8S_AUTH_KUBECONFIG: "{{ kubeconfig_file }}"
gather_facts: false
tasks:
- name: Gather facts (config)
raw: cat /etc/rancher/k3s/k3s.yaml
register: kubeconfig_out
changed_when: false
- name: Save config to file
copy:
content: "{{ kubeconfig_out.stdout }}"
dest: "{{ kubeconfig_file }}"
delegate_to: localhost
- name: Create a k8s namespace
k8s:
name: testing
api_version: v1
kind: Namespace
state: present
delegate_to: localhost |
|
Is there any updates on this issue? Is there any way to workaround? My use case: I want to add zerotier on the host machine. |
|
+1 i have breaking change with vmware tools after 1.19 (error The guest operations agent is out of date.), i can't upgrade or downgrade the agent |
@chris93111 Just a FYI, I built k3os using alpine 3.17 so that I can get the latest version of open-vm-tools and I wasn't able to solve this problem. Were you ever able to figure it out? (I know this was a long time ago, just hoping you might have some suggestions) |
I know the focus has been a minimal OS to run K3S on, but can we get support for installing packages? Mainly I would like Python installed so I can run Ansible playbooks.
The text was updated successfully, but these errors were encountered: