Static pod for k3s-agent

[liangyuanpeng]

Is your feature request related to a problem? Please describe.
Static pod on k3s-agent , just like /var/lib/rancher/k3s/server/manifests on k3s server

Describe the solution you'd like

Describe alternatives you've considered

Additional context

Backporting
Not necessary

• Needs backporting to older releases

[ChristianCiach]

/var/lib/rancher/k3s/server/manifests is not used to deploy a static pod on the server. In fact, this directory cannot be used to deploy a static pod. This directory is used to automatically deploy any kind of manifest that could also be deployed by kubectl apply -f (Deployments, DaemonSets, HelmCharts...).

To deploy static pods, just follow the official documentation: https://kubernetes.io/docs/tasks/configure-pod-container/static-pod/

For K3s, this means that you have to configure the kubelet using --kubelet-arg pod-manifest-path=/your/static/pod/directory when installing a K3s agent (and also the servers, because K3s servers also act as agents).

Edit: You don't even have to configure the kubelet. In K3s, the default pod manifest path is /var/lib/rancher/k3s/agent/pod-manifests/ on all agents. See 8c7fbe3

[liangyuanpeng]

Edit: You don't even have to configure the kubelet. In K3s, the default pod manifest path is /var/lib/rancher/k3s/agent/pod-manifests/ on all agents. See 8c7fbe3

Actually, i had try to deploy redis for static pod on /var/lib/rancher/k3s/agent/pod-manifests/ and nothing runing.

redis-svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: redis
  labels:
    app: redis
spec:
  ports:
    - port: 6379
      name: redis
  type: ClusterIP
  selector:
    app: redis

redis.yaml

apiVersion: apps/v1
kind: StatefulSet
metadata:
  labels:
    app: redis
  name: redis
spec:
  selector:
    matchLabels: 
      app: redis
  serviceName: redis-svc
  replicas: 1
  template:
    metadata:
      labels:
        app: redis
    spec:
      containers:
        - image: redis:6.2.4
          name: redis
          imagePullPolicy: IfNotPresent

[brandond]

Static pods have to be... Pods? Not services or statefulsets. Just pods.

[liangyuanpeng]

Static pods have to be... Pods? Not services or statefulsets. Just pods.

Thanks for correct the wrong content. now i just use pods.

cat /var/lib/rancher/k3s/agent/manifests/redis.yaml

apiVersion: v1
kind: Pod
metadata:
  labels:
    app: redis
  name: redis
  namespace: kube-system
spec:
    containers:
      - image: redis:6.2.4
        name: redis
        imagePullPolicy: IfNotPresent

and still not runing.

[root@kube244 manifests]# k get po -A
NAMESPACE     NAME                                      READY   STATUS      RESTARTS   AGE
kube-system   helm-install-traefik-qmv7d                0/1     Completed   1          27d
kube-system   local-path-provisioner-7c458769fb-wrqdp   1/1     Running     1858       27d
kube-system   metrics-server-86cbb8457f-q4j5d           1/1     Running     1768       30d
kube-system   svclb-traefik-lxdvh                       2/2     Running     22         27d
kube-system   coredns-854c77959c-rkg6c                  1/1     Running     31         27d
kube-system   traefik-6f9cbd9bd4-c9bfn                  1/1     Running     20         27d
kube-system   svclb-traefik-x7vqx                       2/2     Running     14         14d

[root@kube244 manifests]# k get node
NAME      STATUS   ROLES                  AGE   VERSION
kube242   Ready    control-plane,master   30d   v1.20.2+k3s1
kube244   Ready    <none>                 14d   v1.18.20+k3s1

Install k3s by default.

---

Update:
Now i upgrade kube244to v1.20.2+k3s1

[root@kube244 manifests]# k get node
NAME      STATUS   ROLES                  AGE   VERSION
kube244   Ready    <none>                 14m   v1.20.2+k3s1
kube242   Ready    control-plane,master   30d   v1.20.2+k3s1

[ChristianCiach]

Try to fix your indentation of spec.containers:

apiVersion: v1
kind: Pod
metadata:
  labels:
    app: redis
  name: redis
  namespace: kube-system
spec:
  containers:
    - image: redis:6.2.4
      name: redis
      imagePullPolicy: IfNotPresent

I've just tried it on one of my agents. I can confirm that it does work!

On one of my agents I've created this file as /ssd/k3s/agent/pod-manifests/redis.yaml (/ssd/k3s is my k3s data-dir). A few seconds later, I can see the pod on my manager:

# kubectl -n kube-systems get pods
NAME                                           READY   STATUS      RESTARTS   AGE
coredns-7448499f4d-b9mnq                       1/1     Running     0          5d2h
helm-install-traefik-b94bp                     0/1     Completed   0          5d1h
helm-install-traefik-crd-49c7g                 0/1     Completed   0          5d2h
local-path-provisioner-5ff76fc89d-wjnv2        1/1     Running     0          5d2h
metrics-server-86cbb8457f-48sb6                1/1     Running     0          5d2h
redis-MY-AGENT-HOSTNAME                        1/1     Running     0          6m20s
traefik-28vdf                                  1/1     Running     0          5d1h
traefik-km8dm                                  1/1     Running     0          5d1h
traefik-vrbjg                                  1/1     Running     0          5d1h

The feature is working fine. Maybe you have some typo or bad indentation inside your YAML file.