docs(seal-check): record k3d/k3s NetworkPolicy substrate verification (backend#1184) - #504
Conversation
… (backend#1184) RFC-0003 §8.4 says 'do not assume k3d enforces NetworkPolicy'. Ran the substrate check on a throwaway k3d v5.8.3 / k3s v1.33.6+k3s1 cluster: a standalone deny-egress NetworkPolicy on a probe pod took a curl to 1.1.1.1:443 reachable -> BLOCKED -> reachable-again-after-removal (HTTP 301 -> connect failure -> HTTP 301). The block is attributable to the policy, so k3s's embedded kube-router controller does enforce egress NetworkPolicy on this k3d version — the substrate doubt is resolved. Records the result in the §8.4 runbook + the follow-ups list; the §8.3 k3d cell now reads 'substrate verified; full-probe run pending'. Recording the full-chart egress-enforcement probe run against a deployed release stays open (still backend#1184). Docs-only. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
👋 Heads-up — Code review queue is at 33 / 30 Above the WIP limit. The team convention is to review existing PRs before opening new work. Open PRs currently in Code review (oldest first):
Pull from review before opening new work. (This is a nudge from the kanban WIP check, not a block.) |
saqlainsyed007
left a comment
There was a problem hiding this comment.
Docs-only review of docs/SEAL-CHECK.md. The experiment itself is sound and honestly scoped (curl to an IP so no DNS is needed; reachable→blocked→reachable proves attribution). But the change asserts an edit to the §8.3 matrix cell that it never actually makes, leaving the document self-contradictory. Details inline.
🤖 Generated with Claude Code
…ain #504) - Update the §8.3 k3d matrix cell (was 'verification run pending; do not assume') to 'Substrate verified; full-probe run pending', matching the prose that cross-references it — the concrete edit the PR promised but never made (Saqlain #i). - Status note now LEADS with 'still UNSEALED for the egress guarantee on k3d until the full-chart probe is recorded', honoring this doc's unsealed-never-silently-sealed philosophy — a skimmer can no longer read the bold VERIFIED as 'sealed' (Saqlain #ii). - State the run evidence (versions + reachable->blocked->reachable) ONCE in the status note; the post-runbook paragraph, the follow-ups bullet, and the matrix cell now reference it instead of restating — which is how the cell drifted out of sync in the first place (Saqlain #iii). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Thanks — all three addressed in
|
saqlainsyed007
left a comment
There was a problem hiding this comment.
All three review findings addressed in b30bdf9:
- §8.3 k3d matrix cell now reads "substrate verified; full-probe run pending" — matches the prose that references it; self-contradiction resolved.
- Status block now leads with the explicit "still UNSEALED for the egress guarantee on k3d" framing, restoring the doc's "never silently sealed" philosophy.
- Evidence now lives in a single record (the §8.4 Status note); the post-runbook paragraph and follow-ups reference it instead of restating.
Clean, root-cause fixes.
🤖 Generated with Claude Code
|
/fr-pass |
…184) (#541) * test(seal-check): run the egress-enforcement probe live in k3d CI (#1184) RFC-0003 D12: the chart's enforcement probes ship as `helm.sh/hook: test` Jobs, but `helm test` ran nowhere in CI — SEAL-CHECK §8.4 recorded the k3s NetworkPolicy substrate as verified (#504) while the full-chart egress probe run stayed "pending". This closes that gap. - scripts/tests/e2e-seal-check.sh: install the local chart on a real k3d cluster with public images + the egress lockdown engaged (allowExternalHttps=false), then `helm test --filter` the egress-enforcement seal-check. Requires BOTH a zero exit AND the probe's `OK egress lockdown verified` marker in the logs — guarding the helm-test-`--filter`-matches-nothing silent-pass trap. - .github/workflows/helm-ci.yaml: a `seal-check-e2e` job mirroring upgrade-e2e (stock ubuntu runner, zero secrets — public curl vs 1.1.1.1). - client/tests/egress_enforcement_check_test.yaml: pin the probe Job's metadata.name so the e2e --filter can never silently drift off it. Local: shellcheck clean · helm template renders the probe Job · helm-unittest 27 suites / 320 tests green. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(seal-check): install k3d/helm/kubectl before create_cluster (exit 127) The sourced libs define install_kubectl/install_k3d/install_helm but do not call them; create_cluster + helm need the binaries on PATH first, and a stock runner has none preinstalled. Mirror e2e-auto-upgrade.sh's prerequisite block + the post-create node-ready wait. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(seal-check): drive helm test off exit code, not --logs (Job hook) The egress-enforcement probe is a Job-type test hook with hook-delete-policy hook-succeeded. `helm test --logs` looks up the pod by the Job's bare name, but a Job's pod has a generated suffix ("pods not found"), and Helm deletes the Job on success anyway — so --logs errored even though the probe passed. Drive off the exit code instead (the probe exits 0 only when egress is verified blocked), guard the --filter-silent-pass by asserting the hook is in `helm get hooks` first, and dump the pod log via kubectl only on failure (the Job persists when it fails). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(seal-check): address Bugbot review (autostart, paths, shellcheck) Three Bugbot findings on the new e2e seal-check: - Medium: set TRACEBLOC_NO_AUTOSTART=1 before create_cluster (like the sibling e2e-*.sh) so it never mutates the host's Docker restart policy / runs `systemctl enable docker`. - Medium: add scripts/tests/e2e-seal-check.sh to helm-ci.yaml on.push/ on.pull_request paths so script-only edits re-trigger the k3d job. - Low: enumerate the script in the installer-tests ShellCheck gate (both the error and warning passes), matching the other e2e entrypoints. The High finding ("Job logs never reach marker check") was already resolved in the prior commit — the script drives off `helm test`'s exit code and no longer greps --logs output. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(seal-check): avoid SIGPIPE/pipefail false-fail in hook guard (Bugbot) `helm get hooks | grep -q` lets grep close the pipe on first match, SIGPIPE-ing helm mid-write; under set -o pipefail that false-fails the guard even when the hook exists. Capture helm output to a var, then grep a here-string — no pipe. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * ci(seal-check): bump job timeout 20m->30m (Bugbot) The script's own bounds (create_cluster up to 15m + helm test 360s + tool install + helm install) can exceed a 20m GHA cap on a slow cluster bring-up, false-failing even while each component is inside its own timeout. Match the sibling k3d job (upgrade-e2e = 30m). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(seal-check): positive control + probe timeout + lib paths + NS (Saqlain) - Positive control (blocking): before trusting a BLOCKED probe, prove the cluster can REACH the host from a non-policied pod (in `default`, ungoverned by the namespace-scoped training-egress policy). Without it, egress failing for an unrelated reason (runner firewall / target outage / rate-limit) would make the probe print OK and pass green while the NetworkPolicy did nothing. A reachable positive + a blocked training pod = the block is attributable to the policy. Positive-control failure now fails the seal-check as inconclusive. - Bump enforcementProbeTimeoutSeconds 60s->240s (blocking): on a cold GHA runner k3s can take >60s to program the pod iptables while the chart installs; the probe is single-shot (backoffLimit 0), so 60s false-fails. 240s is well inside the 360s helm-test budget. - Add scripts/lib/** to helm-ci paths (blocking): the script sources scripts/lib/{common,setup-linux,cluster,preflight}.sh, so a lib-only edit must re-trigger seal-check-e2e + upgrade-e2e (both depend on it). - Derive NS from CLUSTER_NAME so a CLUSTER_NAME override isolates a run under one name instead of desyncing cluster vs release/namespace. shellcheck clean; workflow parses. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * fix(seal-check): --tlsv1.2 (house rule) + SA wait + required Lint list (Bugbot) - Add --tlsv1.2 to the positive-control curl — the curl-tls house rule (the required quality/house-rules gate) rejects a curl that could negotiate a downgraded TLS version. - Wait for the default ServiceAccount before the positive-control kubectl run: a fast runner can schedule the pod before the SA exists, aborting under set -e with 'serviceaccount default not found' before the attribution message. - Add e2e-seal-check.sh to the standard-checks.yml Lint shellcheck list (the REQUIRED branch-protection gate) — installer-tests had it, the required Lint did not, so a shellcheck regression in the new script could miss the gate. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
Summary
Records the k3d/k3s NetworkPolicy substrate verification that RFC-0003 §8.4 explicitly demands ("do not assume k3d enforces NetworkPolicy"). Docs-only — one file.
What was run
On a throwaway
k3d v5.8.3cluster (k3s v1.33.6+k3s1), a standalone deny-egressNetworkPolicy(podSelector on a probe pod,policyTypes: [Egress], emptyegress:) was applied and acurlfrom the pod to1.1.1.1:443was measured across three states:The reachable → blocked → reachable transition makes the block attributable to the policy, not a fluke or an image-pull artifact. So k3s's embedded (kube-router) controller does enforce egress NetworkPolicy on this k3d version — flannel handles pod networking, but the netpol controller is active and enforcing.
Honest scope (deliberately precise)
This verifies the substrate primitive — the specific doubt §8.4 raises. It is not the full-chart
egress-enforcementprobe run against a deployed release; that end-to-end run stays open under backend#1184, and I've kept the §8.3 k3d cell reading "substrate verified; full-probe run pending" rather than overclaiming "sealed".Why it matters
The chart's
egress-enforcementseal-check relies on the CNI actually dropping egress. That reliance was documented-but-unverified ("expected is not verified"). It's now verified for the k3d/k3s substrate, so the local-dev "nothing gets out" guarantee rests on measured behavior, not an assumption.Epic: backend#1151 · Ticket: backend#1184 (D12)
🤖 Generated with Claude Code
Note
Low Risk
Documentation-only change to seal-check status and runbook text; no runtime, chart, or security behavior changes.
Overview
docs/SEAL-CHECK.mdnow records a completed k3s egress NetworkPolicy substrate check on k3d v5.8.3 / k3s v1.33.6+k3s1 (2026-07-30): deny-egress policy produced reachable → blocked → reachable again oncurlto1.1.1.1:443, with that evidence centralized in the §8.4 Status note.The §8.3 k3d matrix cell and follow-ups are updated to substrate verified; full-chart
egress-enforcementprobe still pending, so k3d remains unsealed for the egress guarantee until the deployed-release runbook (steps 1–4) is recorded—without overclaiming sealed status.Reviewed by Cursor Bugbot for commit b30bdf9. Bugbot is set up for automated code reviews on this repo. Configure here.