Environment
- operator v0.81.1, GKE Standard, Dataplane V2
- cluster created/updated with
--cluster-dns=clouddns --cluster-dns-scope=cluster
profile: all (NetworkPolicy defaults enabled)
Symptom
tekton-results-retention-policy-agent crashloops:
lookup tekton-results-postgres-service.tekton-pipelines.svc.cluster.local
on 169.254.169.254:53: read udp ...->169.254.169.254:53: i/o timeout
fatal: Failed to connect to database: timed out waiting for the condition
Root cause
On GKE clusters with Cloud DNS, kubelet writes nameserver 169.254.169.254 (the node-local metadata server running the Cloud DNS data plane) into every pod's resolv.conf. The CoreDNS/kube-dns pods keep running but are unused.
networkpolicy.DNSEgressRule allows DNS egress only to a pod peer (kube-system / k8s-app=kube-dns on port 53). A podSelector/namespaceSelector peer can never match 169.254.169.254 — it is not a pod. Only an ipBlock peer can allow it. On Dataplane V2 the packets are dropped, causing total DNS failure for any pod whose only DNS allow is that rule.
Most generated policies also include APIServerEgressRule() (allow-all, since NP cannot select host-network endpoints), which incidentally permits the link-local resolver and masks the bug. The two policies without it fail visibly:
results-retention-policy-agent (DNS rule + db_port egress only)results-postgres (DNS rule only; latent — postgres does no runtime lookups)
The same applies to NodeLocal DNSCache (169.254.20.10, Autopilot default / opt-in on Standard): it is a host-network DaemonSet unreachable by pod selectors.
Workaround
Override the generated policy by name via spec.networkPolicy.policies (on TektonConfig) and add an ipBlock: 169.254.169.254/32 peer on UDP+TCP 53. Takes effect on next reconcile; no pod restart needed.
Proposed fix
In DNSEgressRule (Kubernetes platform only), add ipBlock peers 169.254.169.254/32 and 169.254.20.10/32 on port 53 alongside the existing kube-dns pod peer. Additive and zero-risk on clusters not using these resolvers — the extra peers simply never match. OpenShift path unchanged.
Environment
--cluster-dns=clouddns --cluster-dns-scope=clusterprofile: all(NetworkPolicy defaults enabled)Symptom
tekton-results-retention-policy-agentcrashloops:Root cause
On GKE clusters with Cloud DNS, kubelet writes
nameserver 169.254.169.254(the node-local metadata server running the Cloud DNS data plane) into every pod's resolv.conf. The CoreDNS/kube-dns pods keep running but are unused.networkpolicy.DNSEgressRuleallows DNS egress only to a pod peer (kube-system/k8s-app=kube-dnson port 53). A podSelector/namespaceSelector peer can never match 169.254.169.254 — it is not a pod. Only anipBlockpeer can allow it. On Dataplane V2 the packets are dropped, causing total DNS failure for any pod whose only DNS allow is that rule.Most generated policies also include
APIServerEgressRule()(allow-all, since NP cannot select host-network endpoints), which incidentally permits the link-local resolver and masks the bug. The two policies without it fail visibly:results-retention-policy-agent(DNS rule + db_port egress only)results-postgres(DNS rule only; latent — postgres does no runtime lookups)The same applies to NodeLocal DNSCache (
169.254.20.10, Autopilot default / opt-in on Standard): it is a host-network DaemonSet unreachable by pod selectors.Workaround
Override the generated policy by name via
spec.networkPolicy.policies(onTektonConfig) and add anipBlock: 169.254.169.254/32peer on UDP+TCP 53. Takes effect on next reconcile; no pod restart needed.Proposed fix
In
DNSEgressRule(Kubernetes platform only), add ipBlock peers169.254.169.254/32and169.254.20.10/32on port 53 alongside the existing kube-dns pod peer. Additive and zero-risk on clusters not using these resolvers — the extra peers simply never match. OpenShift path unchanged.