helm: Add pod affinity for cilium-envoy

This is the best effort to avoid cilium-envoy running on the node
without cilium agent. Kind note that the extra cilium-envoy pod
(if any) will be just no-ops except a bunch of warn logs related
to non-existence of shared unix domain socket.

The reason for preferred but not required options is to avoid race
condition for new installation between cilium-agent and cilium-envoy
deamon sets.

Relates: cilium#25081, cilium#30034
Fixes: cilium#31149
Signed-off-by: Tam Mach <tam.mach@cilium.io>

install/kubernetes/cilium/values.yaml.tmpl

@@ -1925,6 +1925,15 @@ envoy:
           labelSelector:
             matchLabels:
               k8s-app: cilium-envoy
+    podAffinity:
+      preferredDuringSchedulingIgnoredDuringExecution:
+        - weight: 1
+          podAffinityTerm:
+            topologyKey: kubernetes.io/hostname
+            labelSelector:
+              matchLabels:
+                k8s-app: cilium
+
   # -- Node selector for cilium-envoy.
   nodeSelector:
     kubernetes.io/os: linux