k8s: every Deployment uses the same label selector, so each one selects all pods in the deployment

[dboreham]

Summary

Every Deployment generated for a k8s deployment uses the same label selector,
app: <deployment-id>, so each one nominally selects all pods in the deployment.
The service label that distinguishes them is applied to the pods but appears in no
Deployment selector.

The Services get this right; the Deployments do not:

# src/stack/deploy/k8s/cluster_info.py:258  (Service — correct)
selector={"app": self.app_name, "service": service_name},

# src/stack/deploy/k8s/cluster_info.py:615  (Deployment — selects everything)
selector={"matchLabels": {"app": self.app_name}},

Observed

Five services in one pod file:

$ kubectl get deploy -n stack-<id> -o json | ...
  deploy-api       selector={'app': 'stack-<id>'}   pod labels={'app': 'stack-<id>', 'service': 'api'}
  deploy-db        selector={'app': 'stack-<id>'}   pod labels={'app': 'stack-<id>', 'service': 'db'}
  deploy-frontend  selector={'app': 'stack-<id>'}   pod labels={'app': 'stack-<id>', 'service': 'frontend'}
  deploy-mailpit   selector={'app': 'stack-<id>'}   pod labels={'app': 'stack-<id>', 'service': 'mailpit'}
  deploy-worker    selector={'app': 'stack-<id>'}   pod labels={'app': 'stack-<id>', 'service': 'worker'}

$ for each: kubectl get pods -l "<that selector>" | wc -l
  every one matches all 5 pods

Impact — ergonomic, not (currently) a correctness problem

I went looking for damage and did not find any, so this is filed as a hygiene issue
rather than an outage risk. For the record, what I checked on a live single-node k3s
cluster:

• Services are unaffected. Their selectors include service, and each Service has
  exactly one endpoint. No traffic is misrouted.
• ReplicaSets do not fight, because the Deployment controller adds pod-template-hash
  to each RS selector ({'app': 'stack-<id>', 'pod-template-hash': '85cf6476d4'}). The
  container name alone differs between services, so the hashes always differ.
• No stray adoption. I created a bare pod carrying only app=<id> in a scratch
  deployment: nothing adopted it, nothing deleted it, and both ReplicaSets continued to
  report replicas=1. kubectl get deploy shows correct 1/1 readiness throughout.

What it does cost is observability, and that cost is real:

$ kubectl logs -n stack-<id> deploy/deploy-worker --tail=5
Found 5 pods, using pod/deploy-mailpit-57559c67c4-9pmwd

kubectl picks an arbitrary pod from all five. While debugging a stack of mine I read
mailpit's log believing it was the worker's, and drew a wrong conclusion from it before
noticing. kubectl get pods -l app=<id> is similarly undifferentiated (-l service=api
works fine, but that is not what anyone reaches for after kubectl get deploy).

It also leaves the deployment one upstream behaviour change away from a real problem:
overlapping Deployment selectors are documented as unsupported, and today only
pod-template-hash is keeping them apart.

Suggested fix

selector={"matchLabels": {"app": self.app_name, "service": service_name}},

Note spec.selector is immutable on an existing Deployment, so this takes effect for
newly created deployments; existing ones would need to be recreated rather than updated.