What is the proper, talos way to install basic (bridge, macvlan etc.) CNI plugins?

[ajgon]

Default talos installation doesn't have most of the CNI plugins like bridge or macvlan. However I noticed, they are provided here: https://github.com/siderolabs/pkgs/pkgs/container/cni .

The problem is, I can't create extension out of them, as talos complains, that extensions can't write to /opt/cni/bin directory. So what is the proper way to install it? I noticed in the docs, that talosctl should do this automatically - however I assume this is only in QEMU context, or I did something wrong...

[smira]

Most usually the CNI plugins are shipped as part of the CNI installation (e.g. Cilium, Calico).

If they are not shipped with CNI, you have an option to use a Talos-provided init container which contains the plugins.

You can check it out in the default installation mode of Talos with Flannel CNI:

  - command:
    - /install-cni.sh
    image: ghcr.io/siderolabs/install-cni:v1.5.0
    imagePullPolicy: IfNotPresent
    name: install-cni
    resources: {}
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /host/opt/cni/bin/
      name: cni-plugin

You can use this container image or any other similar image.

We would love to make CNI plugins part of the base Talos installation (or via system extensions), and make /opt/cni/bin read-only, but that would make Talos incompatible with popular CNIs like Cilium.

[ajgon]

Ended up with something similar, but with my own image. Wasn't aware, there is a talos-provided one - which is great (one less thing to maintain for me :) ).

Works like a charm, thank you!

[okamidash]

I utilized this as a patch for the kube-router Daemonset; and resulted with the following:

cat << EOF > kube-router-patch.yaml
spec:
  template:
    spec:
      volumes:
      - hostPath:
          path: /opt/cni/bin
          type: ""
        name: cni-plugin
      initContainers:
        - command:
          - /install-cni.sh
          image: ghcr.io/siderolabs/install-cni:v1.5.0
          imagePullPolicy: IfNotPresent
          name: install-cni-sidero
          resources: {}
          terminationMessagePath: /dev/termination-log
          terminationMessagePolicy: File
          volumeMounts:
          - mountPath: /host/opt/cni/bin/
            name: cni-plugin
EOF

cat kube-router-patch.yaml
kubectl patch ds -n kube-system kube-router --type strategic --patch-file patch.yaml
rm kube-router-patch.yaml

Looks like it's working. Would you benefit from instruction being provided to install kube-router?

[frezbo]

docs PR's welcome, https://github.com/siderolabs/talos/tree/main/website/content/v1.6/kubernetes-guides/network probably here