Skip to content

v1.5.1 — Colima Docker daemon DNS pinning

Choose a tag to compare

View all tags
@bogdandragosvasile bogdandragosvasile released this 20 Apr 11:23
· 7 commits to main since this release
v1.5.1
44a9035

Fix

Pod restarts on Colima-hosted k3d no longer trigger cascading failures.

Problem

Colima's default DNS (`192.168.5.2` on the lima gateway) is unreachable from inside nested Docker networks. `scripts/04-create-k3d-cluster.sh` already patched `/etc/resolv.conf` on each k3d node at creation — but that patch is container-local and Docker wipes it on any container restart.

If you `docker restart k3d-gitops-local-agent-1` (or Colima restarts the daemon for any reason), every pod on that node fails with `ImagePullBackOff` (can't resolve registry.k8s.io) or "connection refused to 10.43.0.1:443" (kube-proxy iptables go stale). Pods like kube-state-metrics, cert-manager-cainjector, or metallb-controller start crashlooping and generate hundreds of events per hour.

Solution

`scripts/00-prerequisites.sh` now writes `/etc/docker/daemon.json` inside the Colima VM during the prerequisites phase:

```json
{"dns": ["8.8.8.8", "1.1.1.1"]}
```

and reloads the Docker daemon. Every new container (including k3d nodes after a restart) inherits public DNS automatically. The k3d-node-level patch in `04-create-k3d-cluster.sh` stays as belt-and-braces for already-running clusters.

Also

  • New troubleshooting section in the portal user manual (`http://portal.local/manual.html\`) covers the "stale kube-proxy iptables after node restart" failure with the copy-paste recovery recipe.

Recovering an already-affected cluster

```bash

Restart the affected k3d node (non-destructive; pods reschedule)

docker restart k3d-gitops-local-agent-1

Re-patch its DNS (Docker wipes /etc/resolv.conf on restart)

docker exec k3d-gitops-local-agent-1 sh -c \
'printf "nameserver 8.8.8.8\nnameserver 1.1.1.1\noptions ndots:0\n" > /etc/resolv.conf'

Delete pods that crashlooped — they reschedule cleanly

kubectl delete pod -A --field-selector=status.phase!=Running,status.phase!=Succeeded
```

Or, from v1.5.1 onwards, run `bash scripts/00-prerequisites.sh` once and every subsequent container restart will pick up the daemon-level DNS.

Full changelog

See CHANGELOG.md.

Assets 2
Loading