From 38817c4f03c150d7d9a543cec212270d04eecbb3 Mon Sep 17 00:00:00 2001 From: Nikolai Emil Damm Date: Sun, 2 Aug 2026 14:00:55 +0200 Subject: [PATCH 1/7] chore: claim #2898 From 86f8e94a563d48b3a15b9da93ede0b012ce7bcc2 Mon Sep 17 00:00:00 2001 From: Nikolai Emil Damm Date: Sun, 2 Aug 2026 14:08:04 +0200 Subject: [PATCH 2/7] security(k8s): disposition CKV_K8S_40 where the UID is load-bearing Nine of the eleven CKV_K8S_40 findings sit on workloads whose UID cannot move: umami's provisioner must be the app's own nextjs user, the three OpenBao jobs must match the StatefulSet UID that owns the shared vault-snapshots PVC, CoreDNS runs as root to bind :53, the minio server and its bucket Job share one identity over the /data emptyDir, and the two userns probes run as the exact UID pair they are measuring. Each site carries a scoped checkov.io/skipN naming its own reason, so the control still fires on a new workload that runs below UID 10000 without one. The two residual findings are inside vendored upstream bundles and need a different mechanism (#2899). Measured with scripts/megalinter-scan-counts.sh: checkov 37 -> 28, CKV_K8S_40 11 -> 2, no other check's count moved. Part of #2787 --- .mega-linter.yml | 9 +++++++-- k8s/bases/apps/umami/cron-job.yaml | 5 +++++ k8s/bases/infrastructure/vault-backup/cron-job.yaml | 5 +++++ k8s/bases/infrastructure/vault-backup/job.yaml | 4 ++++ k8s/bases/infrastructure/vault-config/job.yaml | 5 +++++ .../infrastructure/controllers/coredns/deployment.yaml | 4 ++++ .../infrastructure/controllers/minio/deployment.yaml | 5 +++++ .../docker/infrastructure/controllers/minio/job.yaml | 4 ++++ .../hetzner/apps/userns-headlamp-mapping-probe/job.yaml | 5 +++++ .../hetzner/apps/userns-longhorn-smoke/job.yaml | 5 +++++ 10 files changed, 49 insertions(+), 2 deletions(-) diff --git a/.mega-linter.yml b/.mega-linter.yml index 0986b5f48..3260de4a4 100644 --- a/.mega-linter.yml +++ b/.mega-linter.yml @@ -83,10 +83,15 @@ DISABLE_ERRORS_LINTERS: # does not fail the build while a tracked backlog is worked off. Nothing is thresholded, no # path is hidden, and each entry names the issue that removes it. # - # checkov (37) and trivy (624): Kubernetes misconfiguration in k8s/, which the section above + # checkov (28) and trivy (624): Kubernetes misconfiguration in k8s/, which the section above # explains is NOT covered elsewhere. Tracked by #2787. # - # checkov's remaining 37 are all `kubernetes` framework, led by CKV_K8S_40 (11). The `secrets` + # checkov's remaining 28 are all `kubernetes` framework. CKV_K8S_40 is down to 2, both inside + # the vendored KubeVirt and CDI release bundles, which cannot carry an inline annotation without + # a hand-edit the next vendor bump reverts (#2899). The other nine sites each carry a scoped + # skip naming why that UID cannot move — an image's own user, a shared PVC's ownership, root to + # bind :53, or a UID that is itself the thing under test (#2898). CKV_K8S_40 is NOT disabled: a + # new workload running below UID 10000 with no stated reason is still flagged. The `secrets` # framework is at 0: its 31 CKV_SECRET_6 findings were ExternalSecret/PushSecret key NAMES and # OpenBao paths rather than secret material, and each site now carries a scoped skip naming the # control and the reason (#2892). CKV_SECRET_6 is NOT disabled — a real base64 secret committed diff --git a/k8s/bases/apps/umami/cron-job.yaml b/k8s/bases/apps/umami/cron-job.yaml index f36d7f640..52c7e7238 100644 --- a/k8s/bases/apps/umami/cron-job.yaml +++ b/k8s/bases/apps/umami/cron-job.yaml @@ -46,6 +46,11 @@ kind: CronJob metadata: name: umami-provision-tenants namespace: umami + annotations: + # The provisioner talks to the same Umami instance the app serves and must + # act as the app's own identity, the image's nextjs user (see the + # helm-release: uid/gid 1001 = podSecurityContext.runAsUser/fsGroup). + checkov.io/skip1: CKV_K8S_40=must run as umami's own nextjs user 1001 to match the app it provisions spec: schedule: "*/15 * * * *" # Never run two reconciles at once; a slow run (e.g. waiting on a down Umami) diff --git a/k8s/bases/infrastructure/vault-backup/cron-job.yaml b/k8s/bases/infrastructure/vault-backup/cron-job.yaml index 1739c028c..4f911e64a 100644 --- a/k8s/bases/infrastructure/vault-backup/cron-job.yaml +++ b/k8s/bases/infrastructure/vault-backup/cron-job.yaml @@ -27,6 +27,11 @@ metadata: namespace: openbao annotations: checkov.io/skip1: CKV_K8S_38=the snapshot container reads its SA JWT to log in via OpenBao Kubernetes auth + # UID 100 is the UID the upstream OpenBao chart's StatefulSet runs as (see + # the openbao namespace comment). The snapshot writer and OpenBao share the + # vault-snapshots PVC, so both must agree on it; raising this above 10000 + # would orphan every retained snapshot the DR restore path reads. + checkov.io/skip2: CKV_K8S_40=UID 100 matches the OpenBao StatefulSet so the shared vault-snapshots PVC stays readable to both spec: schedule: "30 3 * * *" # Pin the cron evaluation to UTC instead of inheriting kube-controller-manager's diff --git a/k8s/bases/infrastructure/vault-backup/job.yaml b/k8s/bases/infrastructure/vault-backup/job.yaml index e46ad72e7..d3ad48d9a 100644 --- a/k8s/bases/infrastructure/vault-backup/job.yaml +++ b/k8s/bases/infrastructure/vault-backup/job.yaml @@ -29,6 +29,10 @@ metadata: # Jobs are immutable; let Flux delete-and-recreate on spec change. kustomize.toolkit.fluxcd.io/force: enabled checkov.io/skip1: CKV_K8S_38=the snapshot container reads its SA JWT to log in via OpenBao Kubernetes auth + # Writes the baseline snapshot to the same vault-snapshots PVC as the + # nightly CronJob, so it carries the same UID constraint: 100 is the UID + # the upstream OpenBao chart's StatefulSet runs as. + checkov.io/skip2: CKV_K8S_40=UID 100 matches the OpenBao StatefulSet so the baseline snapshot lands with the ownership the restore path expects spec: # No ttlSecondsAfterFinished (intentional). This is a one-shot "snapshot at # deploy/change time" Job (see header) -- it must run ONCE per vault-backup diff --git a/k8s/bases/infrastructure/vault-config/job.yaml b/k8s/bases/infrastructure/vault-config/job.yaml index d40b7825d..25c80b97e 100644 --- a/k8s/bases/infrastructure/vault-config/job.yaml +++ b/k8s/bases/infrastructure/vault-config/job.yaml @@ -44,6 +44,11 @@ metadata: # The Job script itself is idempotent — each command checks state first. kustomize.toolkit.fluxcd.io/force: enabled checkov.io/skip1: CKV_K8S_38=the store-keys container writes the openbao-unseal Secret through kubectl + # vault-init auto-restores the newest raft snapshot from the + # vault-snapshots PVC, whose files are owned by the UID the upstream + # OpenBao chart's StatefulSet runs as (100). Reading them requires the + # same UID. + checkov.io/skip2: CKV_K8S_40=UID 100 matches the OpenBao StatefulSet so the DR restore can read snapshots the PVC already owns at that UID spec: # Auto-delete the Job (and its pods) 10 minutes after completion. Without # this, the `kustomize.toolkit.fluxcd.io/force: enabled` annotation above diff --git a/k8s/providers/docker/infrastructure/controllers/coredns/deployment.yaml b/k8s/providers/docker/infrastructure/controllers/coredns/deployment.yaml index 20b8cf570..718668d98 100644 --- a/k8s/providers/docker/infrastructure/controllers/coredns/deployment.yaml +++ b/k8s/providers/docker/infrastructure/controllers/coredns/deployment.yaml @@ -13,6 +13,10 @@ metadata: # cluster DNS by watching Services, Endpoints and Pods through the API # server. Without the token CoreDNS cannot resolve any in-cluster name. checkov.io/skip1: CKV_K8S_38=CoreDNS resolves cluster DNS by watching Services/Endpoints via the API server + # There is no UID to raise: the pod deliberately sets no runAsUser because + # the upstream image runs as root to bind :53 (see the securityContext + # comment on the pod template). + checkov.io/skip2: CKV_K8S_40=CoreDNS runs as root to bind :53; NET_BIND_SERVICE is not effective for a non-root user without ambient capabilities spec: progressDeadlineSeconds: 600 replicas: 2 diff --git a/k8s/providers/docker/infrastructure/controllers/minio/deployment.yaml b/k8s/providers/docker/infrastructure/controllers/minio/deployment.yaml index 90a31b577..47b2ba064 100644 --- a/k8s/providers/docker/infrastructure/controllers/minio/deployment.yaml +++ b/k8s/providers/docker/infrastructure/controllers/minio/deployment.yaml @@ -16,6 +16,11 @@ metadata: namespace: minio labels: app.kubernetes.io/name: minio + annotations: + # The server writes its object store to the /data emptyDir under + # fsGroup 1000, and minio-create-bucket must reach the same files as the + # same identity. The pair only works while both agree on the UID. + checkov.io/skip1: CKV_K8S_40=server and bucket-creation Job share UID 1000 so mc can write the /data emptyDir the server serves spec: replicas: 1 strategy: diff --git a/k8s/providers/docker/infrastructure/controllers/minio/job.yaml b/k8s/providers/docker/infrastructure/controllers/minio/job.yaml index 5d0ac0946..7af381ae0 100644 --- a/k8s/providers/docker/infrastructure/controllers/minio/job.yaml +++ b/k8s/providers/docker/infrastructure/controllers/minio/job.yaml @@ -6,6 +6,10 @@ kind: Job metadata: name: minio-create-bucket namespace: minio + annotations: + # Paired with the minio Deployment's UID: the bucket this Job creates is + # served by that pod, so both run as 1000. + checkov.io/skip1: CKV_K8S_40=mc runs as UID 1000 to match the minio server whose bucket it creates spec: backoffLimit: 20 ttlSecondsAfterFinished: 3600 diff --git a/k8s/providers/hetzner/apps/userns-headlamp-mapping-probe/job.yaml b/k8s/providers/hetzner/apps/userns-headlamp-mapping-probe/job.yaml index 5627696fb..7b80c9a51 100644 --- a/k8s/providers/hetzner/apps/userns-headlamp-mapping-probe/job.yaml +++ b/k8s/providers/hetzner/apps/userns-headlamp-mapping-probe/job.yaml @@ -43,6 +43,11 @@ metadata: labels: app.kubernetes.io/name: userns-headlamp-mapping-probe app.kubernetes.io/component: user-namespace-probe + annotations: + # The UID/GID pair is the measurement subject, not an incidental choice — + # the probe reads the idmap applied to headlamp's own identity, so any + # other UID measures a different mapping and answers a different question. + checkov.io/skip1: CKV_K8S_40=the probe must run as headlamp's own 100/101 pair because that mapping is what it measures spec: # Deliberately no ttlSecondsAfterFinished: the TTL controller deletes the Job # AND its pod, and the pod's log is the entire deliverable here. The component diff --git a/k8s/providers/hetzner/apps/userns-longhorn-smoke/job.yaml b/k8s/providers/hetzner/apps/userns-longhorn-smoke/job.yaml index fc7661261..db80318ee 100644 --- a/k8s/providers/hetzner/apps/userns-longhorn-smoke/job.yaml +++ b/k8s/providers/hetzner/apps/userns-longhorn-smoke/job.yaml @@ -10,6 +10,11 @@ metadata: labels: app.kubernetes.io/name: userns-longhorn-smoke app.kubernetes.io/component: storage-smoke-test + annotations: + # The whole assertion is that a user-namespaced pod can own files on a + # Longhorn volume under fsGroup 1000; the writer must run as the matching + # UID or the test proves nothing about that pairing. + checkov.io/skip1: CKV_K8S_40=the smoke test asserts Longhorn volume ownership under fsGroup 1000 and must run as the matching UID spec: backoffLimit: 0 activeDeadlineSeconds: 300 From 7439833a50f7450b3d6b5ee9b23c91e63d56ce45 Mon Sep 17 00:00:00 2001 From: Nikolai Emil Damm Date: Sun, 2 Aug 2026 14:14:27 +0200 Subject: [PATCH 3/7] security(k8s): drop three CKV_K8S_40 skips whose reason was not demonstrated Self-review before requesting a review found that the umami provisioner and the two MinIO workloads share no volume with anything: the CronJob mounts nothing and calls an HTTP API, mc reaches the server over HTTP with only a /tmp/.mc emptyDir and no fsGroup, and the MinIO server's /data emptyDir is writable through fsGroup 1000 rather than through the UID. The reasons those three carried asserted shared file ownership that does not exist. They are candidates to simply raise, and unlike the rest of this backlog that is verifiable in CI because two of them run on the local Docker provider. Moved to #2901 rather than shipped with an invented justification. checkov 37 -> 31, CKV_K8S_40 11 -> 5. Part of #2787 --- .mega-linter.yml | 17 ++++++++++------- k8s/bases/apps/umami/cron-job.yaml | 5 ----- .../controllers/minio/deployment.yaml | 5 ----- .../infrastructure/controllers/minio/job.yaml | 4 ---- 4 files changed, 10 insertions(+), 21 deletions(-) diff --git a/.mega-linter.yml b/.mega-linter.yml index 3260de4a4..c00f01c74 100644 --- a/.mega-linter.yml +++ b/.mega-linter.yml @@ -83,15 +83,18 @@ DISABLE_ERRORS_LINTERS: # does not fail the build while a tracked backlog is worked off. Nothing is thresholded, no # path is hidden, and each entry names the issue that removes it. # - # checkov (28) and trivy (624): Kubernetes misconfiguration in k8s/, which the section above + # checkov (31) and trivy (624): Kubernetes misconfiguration in k8s/, which the section above # explains is NOT covered elsewhere. Tracked by #2787. # - # checkov's remaining 28 are all `kubernetes` framework. CKV_K8S_40 is down to 2, both inside - # the vendored KubeVirt and CDI release bundles, which cannot carry an inline annotation without - # a hand-edit the next vendor bump reverts (#2899). The other nine sites each carry a scoped - # skip naming why that UID cannot move — an image's own user, a shared PVC's ownership, root to - # bind :53, or a UID that is itself the thing under test (#2898). CKV_K8S_40 is NOT disabled: a - # new workload running below UID 10000 with no stated reason is still flagged. The `secrets` + # checkov's remaining 31 are all `kubernetes` framework. CKV_K8S_40 is down from 11 to 5. Six + # sites carry a scoped skip naming why that UID cannot move: the three OpenBao jobs share the + # StatefulSet's UID 100 and the vault-snapshots PVC it owns (read live from the cluster), CoreDNS + # runs as root to bind :53, and the two user-namespace probes run as the exact UID pair they + # measure (#2898). Of the five left, two are inside the vendored KubeVirt and CDI release bundles + # and need a mechanism that survives a vendor bump (#2899); three — the umami provisioner and the + # two MinIO workloads — have no demonstrated constraint at all and are candidates to simply raise + # (#2901). CKV_K8S_40 is NOT disabled: a new workload running below UID 10000 with no stated + # reason is still flagged. The `secrets` # framework is at 0: its 31 CKV_SECRET_6 findings were ExternalSecret/PushSecret key NAMES and # OpenBao paths rather than secret material, and each site now carries a scoped skip naming the # control and the reason (#2892). CKV_SECRET_6 is NOT disabled — a real base64 secret committed diff --git a/k8s/bases/apps/umami/cron-job.yaml b/k8s/bases/apps/umami/cron-job.yaml index 52c7e7238..f36d7f640 100644 --- a/k8s/bases/apps/umami/cron-job.yaml +++ b/k8s/bases/apps/umami/cron-job.yaml @@ -46,11 +46,6 @@ kind: CronJob metadata: name: umami-provision-tenants namespace: umami - annotations: - # The provisioner talks to the same Umami instance the app serves and must - # act as the app's own identity, the image's nextjs user (see the - # helm-release: uid/gid 1001 = podSecurityContext.runAsUser/fsGroup). - checkov.io/skip1: CKV_K8S_40=must run as umami's own nextjs user 1001 to match the app it provisions spec: schedule: "*/15 * * * *" # Never run two reconciles at once; a slow run (e.g. waiting on a down Umami) diff --git a/k8s/providers/docker/infrastructure/controllers/minio/deployment.yaml b/k8s/providers/docker/infrastructure/controllers/minio/deployment.yaml index 47b2ba064..90a31b577 100644 --- a/k8s/providers/docker/infrastructure/controllers/minio/deployment.yaml +++ b/k8s/providers/docker/infrastructure/controllers/minio/deployment.yaml @@ -16,11 +16,6 @@ metadata: namespace: minio labels: app.kubernetes.io/name: minio - annotations: - # The server writes its object store to the /data emptyDir under - # fsGroup 1000, and minio-create-bucket must reach the same files as the - # same identity. The pair only works while both agree on the UID. - checkov.io/skip1: CKV_K8S_40=server and bucket-creation Job share UID 1000 so mc can write the /data emptyDir the server serves spec: replicas: 1 strategy: diff --git a/k8s/providers/docker/infrastructure/controllers/minio/job.yaml b/k8s/providers/docker/infrastructure/controllers/minio/job.yaml index 7af381ae0..5d0ac0946 100644 --- a/k8s/providers/docker/infrastructure/controllers/minio/job.yaml +++ b/k8s/providers/docker/infrastructure/controllers/minio/job.yaml @@ -6,10 +6,6 @@ kind: Job metadata: name: minio-create-bucket namespace: minio - annotations: - # Paired with the minio Deployment's UID: the bucket this Job creates is - # served by that pod, so both run as 1000. - checkov.io/skip1: CKV_K8S_40=mc runs as UID 1000 to match the minio server whose bucket it creates spec: backoffLimit: 20 ttlSecondsAfterFinished: 3600 From fc2449317cccfaf3ac16fe72c5b74ec9a66d1e82 Mon Sep 17 00:00:00 2001 From: Nikolai Emil Damm Date: Sun, 2 Aug 2026 14:33:00 +0200 Subject: [PATCH 4/7] security(k8s): drop two more CKV_K8S_40 skips and correct a false CoreDNS comment CodeRabbit raised both as Major; both verified independently and both correct. vault-config does NOT mount the vault-snapshots PVC -- its restore source is a `snapshots` emptyDir the fetch-snapshot init container populates from the R2/MinIO mirror, and two comments already in the file say so. The skip's ownership reason was simply false. CoreDNS does not run as root. Read from the pinned image's own config (v1.14.3, sha256:884b72dd...): `User: "nonroot:nonroot"`, `Entrypoint: /coredns`. The pre-existing in-file comment asserting root was wrong too, so it is corrected with the measured value rather than deleted. An explicit `runAsUser: 65532` would match the image and clear the check, but nothing in CI brings up a cluster to prove DNS still resolves, so it moves to #2901 instead of changing cluster DNS blind. Both join the no-demonstrated-constraint bucket. checkov 37 -> 33, CKV_K8S_40 11 -> 7. Part of #2787 --- .mega-linter.yml | 21 ++++++++++--------- .../infrastructure/vault-config/job.yaml | 5 ----- .../controllers/coredns/deployment.yaml | 15 ++++++------- 3 files changed, 19 insertions(+), 22 deletions(-) diff --git a/.mega-linter.yml b/.mega-linter.yml index c00f01c74..4a0ef17a1 100644 --- a/.mega-linter.yml +++ b/.mega-linter.yml @@ -83,18 +83,19 @@ DISABLE_ERRORS_LINTERS: # does not fail the build while a tracked backlog is worked off. Nothing is thresholded, no # path is hidden, and each entry names the issue that removes it. # - # checkov (31) and trivy (624): Kubernetes misconfiguration in k8s/, which the section above + # checkov (33) and trivy (624): Kubernetes misconfiguration in k8s/, which the section above # explains is NOT covered elsewhere. Tracked by #2787. # - # checkov's remaining 31 are all `kubernetes` framework. CKV_K8S_40 is down from 11 to 5. Six - # sites carry a scoped skip naming why that UID cannot move: the three OpenBao jobs share the - # StatefulSet's UID 100 and the vault-snapshots PVC it owns (read live from the cluster), CoreDNS - # runs as root to bind :53, and the two user-namespace probes run as the exact UID pair they - # measure (#2898). Of the five left, two are inside the vendored KubeVirt and CDI release bundles - # and need a mechanism that survives a vendor bump (#2899); three — the umami provisioner and the - # two MinIO workloads — have no demonstrated constraint at all and are candidates to simply raise - # (#2901). CKV_K8S_40 is NOT disabled: a new workload running below UID 10000 with no stated - # reason is still flagged. The `secrets` + # checkov's remaining 33 are all `kubernetes` framework. CKV_K8S_40 is down from 11 to 7. Four + # sites carry a scoped skip naming why that UID cannot move: the two vault-backup jobs share the + # OpenBao StatefulSet's UID 100 and the vault-snapshots PVC they mount (the StatefulSet's + # `runAsUser: 100` read live from the cluster), and the two user-namespace probes run as the exact + # UID pair they measure (#2898). Of the seven left, two are inside the vendored KubeVirt and CDI + # release bundles and need a mechanism that survives a vendor bump (#2899); the other five have no + # demonstrated constraint and are candidates to simply raise (#2901) — including CoreDNS, whose + # pinned image config reads `User: "nonroot:nonroot"`, so an explicit `runAsUser: 65532` would + # both match the image and clear the check. CKV_K8S_40 is NOT disabled: a new workload running + # below UID 10000 with no stated reason is still flagged. The `secrets` # framework is at 0: its 31 CKV_SECRET_6 findings were ExternalSecret/PushSecret key NAMES and # OpenBao paths rather than secret material, and each site now carries a scoped skip naming the # control and the reason (#2892). CKV_SECRET_6 is NOT disabled — a real base64 secret committed diff --git a/k8s/bases/infrastructure/vault-config/job.yaml b/k8s/bases/infrastructure/vault-config/job.yaml index 25c80b97e..d40b7825d 100644 --- a/k8s/bases/infrastructure/vault-config/job.yaml +++ b/k8s/bases/infrastructure/vault-config/job.yaml @@ -44,11 +44,6 @@ metadata: # The Job script itself is idempotent — each command checks state first. kustomize.toolkit.fluxcd.io/force: enabled checkov.io/skip1: CKV_K8S_38=the store-keys container writes the openbao-unseal Secret through kubectl - # vault-init auto-restores the newest raft snapshot from the - # vault-snapshots PVC, whose files are owned by the UID the upstream - # OpenBao chart's StatefulSet runs as (100). Reading them requires the - # same UID. - checkov.io/skip2: CKV_K8S_40=UID 100 matches the OpenBao StatefulSet so the DR restore can read snapshots the PVC already owns at that UID spec: # Auto-delete the Job (and its pods) 10 minutes after completion. Without # this, the `kustomize.toolkit.fluxcd.io/force: enabled` annotation above diff --git a/k8s/providers/docker/infrastructure/controllers/coredns/deployment.yaml b/k8s/providers/docker/infrastructure/controllers/coredns/deployment.yaml index 718668d98..f1213e25f 100644 --- a/k8s/providers/docker/infrastructure/controllers/coredns/deployment.yaml +++ b/k8s/providers/docker/infrastructure/controllers/coredns/deployment.yaml @@ -13,10 +13,6 @@ metadata: # cluster DNS by watching Services, Endpoints and Pods through the API # server. Without the token CoreDNS cannot resolve any in-cluster name. checkov.io/skip1: CKV_K8S_38=CoreDNS resolves cluster DNS by watching Services/Endpoints via the API server - # There is no UID to raise: the pod deliberately sets no runAsUser because - # the upstream image runs as root to bind :53 (see the securityContext - # comment on the pod template). - checkov.io/skip2: CKV_K8S_40=CoreDNS runs as root to bind :53; NET_BIND_SERVICE is not effective for a non-root user without ambient capabilities spec: progressDeadlineSeconds: 600 replicas: 2 @@ -109,9 +105,14 @@ spec: priorityClassName: system-cluster-critical restartPolicy: Always schedulerName: default-scheduler - # No runAsNonRoot: the upstream CoreDNS image runs as root, and the - # NET_BIND_SERVICE capability it needs to bind :53 is not effective - # for non-root users without ambient capabilities. + # No runAsUser/runAsNonRoot is set here, so the container runs as whatever + # the image declares. Read from the pinned image's own config (v1.14.3, + # sha256:884b72dd…): `User: "nonroot:nonroot"` with `Entrypoint: /coredns` + # — the distroless nonroot identity (65532), not root, and it binds :53 + # through a file capability on the binary rather than an ambient one. + # Setting `runAsUser: 65532` explicitly would match that and satisfy + # checkov CKV_K8S_40, but nothing in CI brings up a cluster to prove DNS + # still resolves, so it is tracked in #2901 rather than changed blind. securityContext: seccompProfile: type: RuntimeDefault From cc72a287b1b3e6d45cc8b9debc20b1d37d119fc2 Mon Sep 17 00:00:00 2001 From: Nikolai Emil Damm Date: Sun, 2 Aug 2026 16:11:25 +0200 Subject: [PATCH 5/7] security(k8s): withdraw four CKV_K8S_40 skips whose reasons do not hold Each of the four scoped skips named a per-site technical constraint that does not survive checking against the manifests and the live cluster: * The two vault-backup workloads claimed to share the vault-snapshots PVC with the OpenBao StatefulSet. The live StatefulSet mounts config, unseal-keys, tmp and home plus claim templates data and audit, and never that PVC -- which is ReadWriteOnce, so a concurrent co-mount is structurally impossible. Its only consumers are the two backup workloads and the dr-rebuild helper pod. * The headlamp mapping probe claimed its UID was the measurement subject. /proc/self/uid_map describes the pod's user namespace and reads identically for every process in it regardless of UID; the probe is also a separate pod with its own range, which the file's own header comment already states. * The longhorn smoke test claimed the writer "must run as the matching UID" for fsGroup 1000. fsGroup is a GID and supplies volume access independently of runAsUser. All 11 CKV_K8S_40 findings therefore stand unsuppressed (checkov 33 -> 37). The sound argument in #2898 is that user namespaces already mitigate host UID collision more strongly than a high UID -- but that is one portfolio risk acceptance, not 11 per-site constraints. #2904 carries that decision and the per-site remediation. Part of #2898 --- .mega-linter.yml | 24 +++++++++++-------- .../infrastructure/vault-backup/cron-job.yaml | 5 ---- .../infrastructure/vault-backup/job.yaml | 4 ---- .../userns-headlamp-mapping-probe/job.yaml | 5 ---- .../apps/userns-longhorn-smoke/job.yaml | 5 ---- 5 files changed, 14 insertions(+), 29 deletions(-) diff --git a/.mega-linter.yml b/.mega-linter.yml index 4a0ef17a1..c2f5db479 100644 --- a/.mega-linter.yml +++ b/.mega-linter.yml @@ -83,18 +83,22 @@ DISABLE_ERRORS_LINTERS: # does not fail the build while a tracked backlog is worked off. Nothing is thresholded, no # path is hidden, and each entry names the issue that removes it. # - # checkov (33) and trivy (624): Kubernetes misconfiguration in k8s/, which the section above + # checkov (37) and trivy (624): Kubernetes misconfiguration in k8s/, which the section above # explains is NOT covered elsewhere. Tracked by #2787. # - # checkov's remaining 33 are all `kubernetes` framework. CKV_K8S_40 is down from 11 to 7. Four - # sites carry a scoped skip naming why that UID cannot move: the two vault-backup jobs share the - # OpenBao StatefulSet's UID 100 and the vault-snapshots PVC they mount (the StatefulSet's - # `runAsUser: 100` read live from the cluster), and the two user-namespace probes run as the exact - # UID pair they measure (#2898). Of the seven left, two are inside the vendored KubeVirt and CDI - # release bundles and need a mechanism that survives a vendor bump (#2899); the other five have no - # demonstrated constraint and are candidates to simply raise (#2901) — including CoreDNS, whose - # pinned image config reads `User: "nonroot:nonroot"`, so an explicit `runAsUser: 65532` would - # both match the image and clear the check. CKV_K8S_40 is NOT disabled: a new workload running + # checkov's 37 are all `kubernetes` framework, and all 11 CKV_K8S_40 findings stand unsuppressed. + # Four scoped skips were tried and withdrawn: each named a per-site technical constraint that does + # not survive checking. The two vault-backup jobs claimed to share the vault-snapshots PVC with the + # OpenBao StatefulSet — the live StatefulSet mounts `config`/`unseal-keys`/`tmp`/`home` plus claim + # templates `data`/`audit` and never that PVC, which is `ReadWriteOnce` and so cannot be co-mounted + # at all. The two user-namespace probes claimed their UID was the measurement subject — but + # `/proc/self/uid_map` describes the pod's namespace and reads identically for every process in it + # regardless of UID, and `fsGroup` supplies volume access as a GID independent of `runAsUser`. + # #2904 carries the per-site remediation and the risk-acceptance question. Of the 11, two are + # inside the vendored KubeVirt and CDI release bundles and need a mechanism that survives a vendor + # bump (#2899); the rest are candidates to simply raise (#2901) — including CoreDNS, whose pinned + # image config reads `User: "nonroot:nonroot"`, so an explicit `runAsUser: 65532` would both match + # the image and clear the check. CKV_K8S_40 is NOT disabled: a new workload running # below UID 10000 with no stated reason is still flagged. The `secrets` # framework is at 0: its 31 CKV_SECRET_6 findings were ExternalSecret/PushSecret key NAMES and # OpenBao paths rather than secret material, and each site now carries a scoped skip naming the diff --git a/k8s/bases/infrastructure/vault-backup/cron-job.yaml b/k8s/bases/infrastructure/vault-backup/cron-job.yaml index 4f911e64a..1739c028c 100644 --- a/k8s/bases/infrastructure/vault-backup/cron-job.yaml +++ b/k8s/bases/infrastructure/vault-backup/cron-job.yaml @@ -27,11 +27,6 @@ metadata: namespace: openbao annotations: checkov.io/skip1: CKV_K8S_38=the snapshot container reads its SA JWT to log in via OpenBao Kubernetes auth - # UID 100 is the UID the upstream OpenBao chart's StatefulSet runs as (see - # the openbao namespace comment). The snapshot writer and OpenBao share the - # vault-snapshots PVC, so both must agree on it; raising this above 10000 - # would orphan every retained snapshot the DR restore path reads. - checkov.io/skip2: CKV_K8S_40=UID 100 matches the OpenBao StatefulSet so the shared vault-snapshots PVC stays readable to both spec: schedule: "30 3 * * *" # Pin the cron evaluation to UTC instead of inheriting kube-controller-manager's diff --git a/k8s/bases/infrastructure/vault-backup/job.yaml b/k8s/bases/infrastructure/vault-backup/job.yaml index d3ad48d9a..e46ad72e7 100644 --- a/k8s/bases/infrastructure/vault-backup/job.yaml +++ b/k8s/bases/infrastructure/vault-backup/job.yaml @@ -29,10 +29,6 @@ metadata: # Jobs are immutable; let Flux delete-and-recreate on spec change. kustomize.toolkit.fluxcd.io/force: enabled checkov.io/skip1: CKV_K8S_38=the snapshot container reads its SA JWT to log in via OpenBao Kubernetes auth - # Writes the baseline snapshot to the same vault-snapshots PVC as the - # nightly CronJob, so it carries the same UID constraint: 100 is the UID - # the upstream OpenBao chart's StatefulSet runs as. - checkov.io/skip2: CKV_K8S_40=UID 100 matches the OpenBao StatefulSet so the baseline snapshot lands with the ownership the restore path expects spec: # No ttlSecondsAfterFinished (intentional). This is a one-shot "snapshot at # deploy/change time" Job (see header) -- it must run ONCE per vault-backup diff --git a/k8s/providers/hetzner/apps/userns-headlamp-mapping-probe/job.yaml b/k8s/providers/hetzner/apps/userns-headlamp-mapping-probe/job.yaml index 7b80c9a51..5627696fb 100644 --- a/k8s/providers/hetzner/apps/userns-headlamp-mapping-probe/job.yaml +++ b/k8s/providers/hetzner/apps/userns-headlamp-mapping-probe/job.yaml @@ -43,11 +43,6 @@ metadata: labels: app.kubernetes.io/name: userns-headlamp-mapping-probe app.kubernetes.io/component: user-namespace-probe - annotations: - # The UID/GID pair is the measurement subject, not an incidental choice — - # the probe reads the idmap applied to headlamp's own identity, so any - # other UID measures a different mapping and answers a different question. - checkov.io/skip1: CKV_K8S_40=the probe must run as headlamp's own 100/101 pair because that mapping is what it measures spec: # Deliberately no ttlSecondsAfterFinished: the TTL controller deletes the Job # AND its pod, and the pod's log is the entire deliverable here. The component diff --git a/k8s/providers/hetzner/apps/userns-longhorn-smoke/job.yaml b/k8s/providers/hetzner/apps/userns-longhorn-smoke/job.yaml index db80318ee..fc7661261 100644 --- a/k8s/providers/hetzner/apps/userns-longhorn-smoke/job.yaml +++ b/k8s/providers/hetzner/apps/userns-longhorn-smoke/job.yaml @@ -10,11 +10,6 @@ metadata: labels: app.kubernetes.io/name: userns-longhorn-smoke app.kubernetes.io/component: storage-smoke-test - annotations: - # The whole assertion is that a user-namespaced pod can own files on a - # Longhorn volume under fsGroup 1000; the writer must run as the matching - # UID or the test proves nothing about that pairing. - checkov.io/skip1: CKV_K8S_40=the smoke test asserts Longhorn volume ownership under fsGroup 1000 and must run as the matching UID spec: backoffLimit: 0 activeDeadlineSeconds: 300 From 9e9d6241651862acfb11ff14235bceb10a594498 Mon Sep 17 00:00:00 2001 From: Nikolai Emil Damm Date: Sun, 2 Aug 2026 16:16:45 +0200 Subject: [PATCH 6/7] docs(k8s): correct a probe comment that restated the refuted UID claim Withdrawing the suppression annotation left the same false claim in the probe's own securityContext comment: "the mapping measured is the one applied to the UID/GID pair the workload actually runs as". An idmap is a property of the pod's user namespace, so /proc/self/uid_map reads identically for every process in it whatever UID that process runs as. The comment now states what the UID choice actually buys -- the probe runs the way headlamp does rather than as an arbitrary user -- and says plainly that it does not determine what is measured. Also attributes the four withdrawn sites to #2904 rather than folding them into #2901, which was filed for a different five. Part of #2898 --- .mega-linter.yml | 3 ++- .../hetzner/apps/userns-headlamp-mapping-probe/job.yaml | 6 ++++-- 2 files changed, 6 insertions(+), 3 deletions(-) diff --git a/.mega-linter.yml b/.mega-linter.yml index c2f5db479..2d461deb1 100644 --- a/.mega-linter.yml +++ b/.mega-linter.yml @@ -96,7 +96,8 @@ DISABLE_ERRORS_LINTERS: # regardless of UID, and `fsGroup` supplies volume access as a GID independent of `runAsUser`. # #2904 carries the per-site remediation and the risk-acceptance question. Of the 11, two are # inside the vendored KubeVirt and CDI release bundles and need a mechanism that survives a vendor - # bump (#2899); the rest are candidates to simply raise (#2901) — including CoreDNS, whose pinned + # bump (#2899); five have no demonstrated constraint (#2901) and the four above are candidates to + # raise or accept together (#2904) — including CoreDNS, whose pinned # image config reads `User: "nonroot:nonroot"`, so an explicit `runAsUser: 65532` would both match # the image and clear the check. CKV_K8S_40 is NOT disabled: a new workload running # below UID 10000 with no stated reason is still flagged. The `secrets` diff --git a/k8s/providers/hetzner/apps/userns-headlamp-mapping-probe/job.yaml b/k8s/providers/hetzner/apps/userns-headlamp-mapping-probe/job.yaml index 5627696fb..0aca8dbaa 100644 --- a/k8s/providers/hetzner/apps/userns-headlamp-mapping-probe/job.yaml +++ b/k8s/providers/hetzner/apps/userns-headlamp-mapping-probe/job.yaml @@ -76,8 +76,10 @@ spec: app.kubernetes.io/instance: headlamp topologyKey: kubernetes.io/hostname securityContext: - # headlamp's own identity, so the mapping measured is the one applied to - # the UID/GID pair the workload actually runs as. + # headlamp's own identity, so the probe runs the way headlamp does rather + # than as an arbitrary user. It does NOT determine what is measured: an + # idmap is a property of the pod's user namespace, so /proc/self/uid_map + # reads identically for every process in it whatever UID that process is. runAsNonRoot: true runAsUser: 100 runAsGroup: 101 From 3a29e4c7b9f7a13bb45a0f37f1cbe21ef1a6bc93 Mon Sep 17 00:00:00 2001 From: Nikolai Emil Damm Date: Sun, 2 Aug 2026 16:29:58 +0200 Subject: [PATCH 7/7] docs(ci): drop an over-strong RWO claim and fix the CoreDNS issue attribution CodeRabbit round 1 at 9e9d6241, both findings valid: * "ReadWriteOnce ... cannot be co-mounted at all" overstated the mode. RWO restricts read-write attachment to one NODE; several pods on that node can still mount the claim (ReadWriteOncePod is the per-pod one). Removed rather than qualified with a scheduling caveat -- the primary evidence already settles it, since the live StatefulSet does not mount that PVC at all. * Inserting the #2904 clause left "including CoreDNS" attached to it, while coredns/deployment.yaml tracks CoreDNS in #2901. Restructured so the partition reads unambiguously: 2 vendored (#2899) + 4 withdrawn (#2904) + 5 unconstrained (#2901) = 11, with CoreDNS in the five. Part of #2898 --- .mega-linter.yml | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/.mega-linter.yml b/.mega-linter.yml index 2d461deb1..699894ff4 100644 --- a/.mega-linter.yml +++ b/.mega-linter.yml @@ -89,15 +89,15 @@ DISABLE_ERRORS_LINTERS: # checkov's 37 are all `kubernetes` framework, and all 11 CKV_K8S_40 findings stand unsuppressed. # Four scoped skips were tried and withdrawn: each named a per-site technical constraint that does # not survive checking. The two vault-backup jobs claimed to share the vault-snapshots PVC with the - # OpenBao StatefulSet — the live StatefulSet mounts `config`/`unseal-keys`/`tmp`/`home` plus claim - # templates `data`/`audit` and never that PVC, which is `ReadWriteOnce` and so cannot be co-mounted - # at all. The two user-namespace probes claimed their UID was the measurement subject — but + # OpenBao StatefulSet — but the live StatefulSet mounts `config`/`unseal-keys`/`tmp`/`home` plus + # claim templates `data`/`audit`, and never that PVC at all, so there is no shared ownership to + # preserve. The two user-namespace probes claimed their UID was the measurement subject — but # `/proc/self/uid_map` describes the pod's namespace and reads identically for every process in it # regardless of UID, and `fsGroup` supplies volume access as a GID independent of `runAsUser`. - # #2904 carries the per-site remediation and the risk-acceptance question. Of the 11, two are - # inside the vendored KubeVirt and CDI release bundles and need a mechanism that survives a vendor - # bump (#2899); five have no demonstrated constraint (#2901) and the four above are candidates to - # raise or accept together (#2904) — including CoreDNS, whose pinned + # #2904 carries the per-site remediation and the risk-acceptance question for those four. Of the + # 11, two are inside the vendored KubeVirt and CDI release bundles and need a mechanism that + # survives a vendor bump (#2899). The remaining five have no demonstrated constraint and are + # candidates to simply raise (#2901) — including CoreDNS, whose pinned # image config reads `User: "nonroot:nonroot"`, so an explicit `runAsUser: 65532` would both match # the image and clear the check. CKV_K8S_40 is NOT disabled: a new workload running # below UID 10000 with no stated reason is still flagged. The `secrets`