Skip to content
This repository has been archived by the owner on May 12, 2021. It is now read-only.

kata-deploy: add k3s support #823

Merged
merged 2 commits into from
Dec 3, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 8 additions & 2 deletions kata-deploy/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -71,8 +71,14 @@ After completing, the original `daemon.json`, if it existed, is restored and all
### Install Kata on a running Kubernetes cluster

```sh
$ kubectl apply -f kata-rbac.yaml
$ kubectl apply -f kata-deploy.yaml
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/packaging/master/kata-deploy/kata-rbac/base/kata-rbac.yaml
$ kubectl apply -f https://raw.githubusercontent.com/kata-containers/packaging/master/kata-deploy/kata-deploy/base/kata-deploy.yaml
```

or on a [k3s](https://k3s.io/) cluster:

```sh
$ kubectl apply -k github.com/kata-containers/packaging/kata-deploy/kata-deploy/overlays/k3s
```

### Run a sample workload
Expand Down
2 changes: 2 additions & 0 deletions kata-deploy/kata-cleanup/base/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
resources:
- kata-cleanup.yaml
2 changes: 2 additions & 0 deletions kata-deploy/kata-deploy/base/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
resources:
- kata-deploy.yaml
5 changes: 5 additions & 0 deletions kata-deploy/kata-deploy/overlays/k3s/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
bases:
- ../../base

patchesStrategicMerge:
- mount_k3s_conf.yaml
12 changes: 12 additions & 0 deletions kata-deploy/kata-deploy/overlays/k3s/mount_k3s_conf.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
apiVersion: apps/v1
kind: DaemonSet
metadata:
name: kata-deploy
namespace: kube-system
spec:
template:
spec:
volumes:
- name: containerd-conf
hostPath:
path: /var/lib/rancher/k3s/agent/etc/containerd/
2 changes: 2 additions & 0 deletions kata-deploy/kata-rbac/base/kustomization.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
resources:
- kata-rbac.yaml
30 changes: 24 additions & 6 deletions kata-deploy/scripts/kata-deploy.sh
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,15 @@ function get_container_runtime() {
if [ "$?" -ne 0 ]; then
die "invalid node name"
fi
echo "$runtime" | awk -F'[:]' '/Container Runtime Version/ {print $2}' | tr -d ' '
if echo "$runtime" | grep -qE 'Container Runtime Version.*containerd.*-k3s'; then
if systemctl is-active --quiet k3s-agent; then
echo "k3s-agent"
else
echo "k3s"
fi
else
echo "$runtime" | awk -F'[:]' '/Container Runtime Version/ {print $2}' | tr -d ' '
fi
}

function install_artifacts() {
Expand All @@ -50,7 +58,7 @@ function configure_cri_runtime() {
crio)
configure_crio
;;
containerd)
containerd | k3s | k3s-agent)
configure_containerd
;;
esac
Expand Down Expand Up @@ -225,7 +233,7 @@ function cleanup_cri_runtime() {
crio)
cleanup_crio
;;
containerd)
containerd | k3s | k3s-agent)
cleanup_containerd
;;
esac
Expand All @@ -238,7 +246,7 @@ function cleanup_crio() {
}

function cleanup_containerd() {
rm -f /etc/containerd/config.toml
rm -f $containerd_conf_file
if [ -f "$containerd_conf_file_backup" ]; then
mv "$containerd_conf_file_backup" "$containerd_conf_file"
fi
Expand All @@ -265,7 +273,9 @@ function reset_runtime() {
kubectl label node "$NODE_NAME" katacontainers.io/kata-runtime-
systemctl daemon-reload
systemctl restart "$1"
systemctl restart kubelet
if [ "$1" == "crio" ] || [ "$1" == "containerd" ]; then
systemctl restart kubelet
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stepping back, I'm not sure if we need to restart kubelet for either runtime's either.

Out of curiosity, what was the issue with restarting kubelet in the K3s case?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Like k3s's embedded containerd, kubelet isn't installed as a systemd service.
Let me know if the restart is safe to remove. It looks like it's been there since the beginning: e642e32#diff-1f1e29e0ac464e6daaef21f619af1d96R26

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for clarifying. Let me do some testing on my end, but I expect just removing is appropriate.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

(NOTE: we should have CI so I don't have to test and have everyone in the world trust me!)

fi
}

function main() {
Expand All @@ -280,6 +290,14 @@ function main() {
# CRI-O isn't consistent with the naming -- let's use crio to match the service file
if [ "$runtime" == "cri-o" ]; then
runtime="crio"
elif [ "$runtime" == "k3s" ] || [ "$runtime" == "k3s-agent" ]; then
containerd_conf_tmpl_file="${containerd_conf_file}.tmpl"
if [ ! -f "$containerd_conf_tmpl_file" ]; then
cp "$containerd_conf_file" "$containerd_conf_tmpl_file"
fi

containerd_conf_file="${containerd_conf_tmpl_file}"
containerd_conf_file_backup="${containerd_conf_file}.bak"
fi

action=${1:-}
Expand All @@ -289,7 +307,7 @@ function main() {
fi

# only install / remove / update if we are dealing with CRIO or containerd
if [ "$runtime" == "crio" ] || [ "$runtime" == "containerd" ]; then
if [[ "$runtime" =~ ^(crio|containerd|k3s|k3s-agent)$ ]]; then

case "$action" in
install)
Expand Down