Found by the post-incident installer/chart audit (restart resilience). Related restart theme: #548.
Problem
The always-running control-plane pods render imagePullPolicy: Always on floating :<CLIENT_ENV> tags (e.g. docker.io/tracebloc/jobs-manager:prod) with no digest pin fallback:
client/templates/jobs-manager-deployment.yaml:43client/templates/requests-proxy-deployment.yaml:26 (hardcodes Always regardless of digest)client/templates/resource-monitor-daemonset.yaml:54- rooted at
client/values.yaml:328 / :474-479 (digest: "")
Failure scenario (CONFIRMED)
A Docker Desktop / WSL2 restart (the incident trigger class, routine on Windows) recreates every pod. Always forces a Docker Hub manifest pull at (re)start. If the network isn't back yet, a corporate proxy/TLS-inspection blocks docker.io, or the shared NAT hit Docker Hub's anonymous 100-pull/6h limit, the pull fails → ImagePullBackOff even though the image is already cached in containerd (Always won't fall back to cache). Client shows Offline; self-heals when the registry is reachable, up to ~6h if rate-limited.
Why it matters
ingestor (values.yaml:442 prodDigest) and mysqlClient (:498 digest → IfNotPresent) are already digest-pinned and restart-safe offline; the always-running control-plane pods were left floating. This is also the residual blast radius of R3: image-refresh + Recreate auto-pulling a floating tag onto a single consumer.
Proposed fix
Digest-pin jobsManager / podsMonitor / resourceMonitor / requestsProxy by index-digest (same mechanism as images.ingestor.prodDigest) → IfNotPresent, so a restart survives offline and image-refresh only rolls on a deliberate digest bump.
Severity: Tier 1 — breaks-on-restart (degraded, self-heals). Confidence: CONFIRMED.
Found by the post-incident installer/chart audit (restart resilience). Related restart theme: #548.
Problem
The always-running control-plane pods render
imagePullPolicy: Alwayson floating:<CLIENT_ENV>tags (e.g.docker.io/tracebloc/jobs-manager:prod) with no digest pin fallback:client/templates/jobs-manager-deployment.yaml:43client/templates/requests-proxy-deployment.yaml:26(hardcodesAlwaysregardless of digest)client/templates/resource-monitor-daemonset.yaml:54client/values.yaml:328/:474-479(digest: "")Failure scenario (CONFIRMED)
A Docker Desktop / WSL2 restart (the incident trigger class, routine on Windows) recreates every pod.
Alwaysforces a Docker Hub manifest pull at (re)start. If the network isn't back yet, a corporate proxy/TLS-inspection blocksdocker.io, or the shared NAT hit Docker Hub's anonymous 100-pull/6h limit, the pull fails →ImagePullBackOffeven though the image is already cached in containerd (Alwayswon't fall back to cache). Client shows Offline; self-heals when the registry is reachable, up to ~6h if rate-limited.Why it matters
ingestor(values.yaml:442prodDigest) andmysqlClient(:498digest→IfNotPresent) are already digest-pinned and restart-safe offline; the always-running control-plane pods were left floating. This is also the residual blast radius of R3: image-refresh + Recreate auto-pulling a floating tag onto a single consumer.Proposed fix
Digest-pin
jobsManager/podsMonitor/resourceMonitor/requestsProxyby index-digest (same mechanism asimages.ingestor.prodDigest) →IfNotPresent, so a restart survives offline and image-refresh only rolls on a deliberate digest bump.Severity: Tier 1 — breaks-on-restart (degraded, self-heals). Confidence: CONFIRMED.