kata-deploy provides a Dockerfile, which contains all of the binaries and artifacts required to run Kata Containers, as well as reference daemonsets, which can be utilized to install Kata Containers on a running Kubernetes cluster.
Note, installation through daemonsets successfully installs kata-containers.io/kata-runtime
on
a node only if it uses either containerd or CRI-O CRI-shims.
kubectl apply -f kata-rbac.yaml
kubectl apply -f kata-deploy.yaml
Untrusted workloads can node-select based on kata-containers.io/kata-runtime=true
, and are
run through kata-containers.io/kata-runtime
if they are marked with the appropriate CRIO or containerd
annotation:
CRIO: io.kubernetes.cri-o.TrustedSandbox: "false"
containerd: io.kubernetes.cri.untrusted-workload: "true"
The following is a sample workload for running untrusted on a kata-enabled node:
apiVersion: v1
kind: Pod
metadata:
name: nginx
annotations:
io.kubernetes.cri-o.TrustedSandbox: "false"
io.kubernetes.cri.untrusted-workload: "true"
labels:
env: test
spec:
containers:
- name: nginx
image: nginx
imagePullPolicy: IfNotPresent
nodeSelector:
kata-containers.io/kata-runtime: "true"
To run:
kubectl apply -f examples/nginx-untrusted.yaml
Now, you should see the pod start. You can verify that the pod is making use of
kata-containers.io/kata-runtime
by comparing the container ID observed with the following:
/opt/kata/bin/kata-containers.io/kata-runtime list
kubectl describe pod nginx-untrusted
The following removes the test pod:
kubectl delete -f examples/nginx-untrusted.yaml
kubectl delete -f kata-deploy.yaml
kubectl apply -f kata-cleanup.yaml
kubectl delete -f kata-cleanup.yaml
kubectl delete -f kata-rbac.yaml
The Dockerfile used to create the container image deployed in the DaemonSet is provided here. This image contains all the necessary artifacts for running Kata Containers.
Host artifacts:
- kata-containers.io/kata-runtime: pulled from Kata GitHub releases page
- kata-proxy: pulled from Kata GitHub releases page
- kata-shim: pulled from Kata GitHub releases page
- qemu-system-x86_64: statically built and included in this repo, based on Kata's QEMU repo
- qemu/* : supporting binaries required for qemu-system-x86_64
Virtual Machine artifacts:
- kata-containers.img: pulled from Kata github releases page
- vmliuz.container: pulled from Kata github releases page
A few daemonsets are introduced for kata-deploy, as well as an RBAC to facilitate applying labels to the nodes.
This daemonset creates a label on each node in
the cluster identifying the CRI shim in use. For example,
kata-containers.io/container-runtime=crio
or kata-containers.io/container-runtime=containerd.
Depending on the value of kata-containers.io/container-runtime
label on the node, either the CRI-O or
containerd kata installation daemonset executes. These daemonsets install
the necessary kata binaries, configuration files, and virtual machine artifacts on
the node. Once installed, the daemonset adds a node label kata-containers.io/kata-runtime=true
and reconfigures
either CRI-O or containerd to make use of Kata for untrusted workloads. As a final step the daemonset
restarts either CRI-O or containerd and kubelet. Upon deletion, the daemonset removes the kata binaries
and VM artifacts and updates the node label to kata-containers.io/kata-runtime=cleanup.
This daemonset runs of the node has the label kata-containers.io/kata-runtime=cleanup.
These daemonsets removes
the kata-containers.io/container-runtime
and kata-containers.io/kata-runtime
labels as well as restarts either CRI-O or containerd systemctl
daemon and kubelet. You cannot execute these resets during the preStopHook of the Kata installer daemonset,
which necessitated this final cleanup daemonset.