contrib/kind: set custom DNS resolver for Kind nodes

[ upstream commit 03eeda7 ]

We are using our Kind provisioning script to create K8s clusters when testing
in the CI. Recently, we discovered that on some kernels a default DNS resolver,
which is dockerd, is troublesome for the BPF host routing, which we want to
test in the CI (cilium#23283).

Fix this by patching the coredns configmap after creating a kind cluster to
point to the 8.8.8.8 resolver. Alternative fixes (may still be applied later):

  * Pass a custom /etc/resolv.conf to kubelet via --resolv-conf in the Kind /
    kubeadm config.

  * Override /etc/resolv.conf of Kind nodes after creating a cluster (no race
    condition, as CoreDNS pods won't be started, as a CNI is not ready).

  * Patch Kind to allow users to specify custom DNS entries (i.e., docker run
    --dns="1.1.1.1,8.8.8.8").

Fixes: cilium#23283
Fixes: cilium#23330

Signed-off-by: Anton Protopopov <aspsk@isovalent.com>
Signed-off-by: Martynas Pumputis <m@lambda.lt>
Signed-off-by: Gilberto Bertin <jibi@cilium.io>

contrib/scripts/kind.sh

@@ -169,6 +169,12 @@ for node in $(kind get nodes); do
   kubectl annotate node "${node}" "kind.x-k8s.io/registry=localhost:${reg_port}";
 done
 
+# Replace "forward . /etc/resolv.conf" in the coredns cm with "forward . 8.8.8.8".
+# This is required because in case of BPF Host Routing we bypass iptables thus
+# breaking DNS. See https://github.com/cilium/cilium/issues/23330
+NewCoreFile=$(kubectl get cm -n kube-system coredns -o jsonpath='{.data.Corefile}' | sed 's,forward . /etc/resolv.conf,forward . 8.8.8.8,' | sed -z 's/\n/\\n/g')
+kubectl patch configmap/coredns -n kube-system --type merge -p '{"data":{"Corefile": "'"$NewCoreFile"'"}}'
+
 set +e
 kubectl taint nodes --all node-role.kubernetes.io/master-
 kubectl taint nodes --all node-role.kubernetes.io/control-plane-