test(e2e): full seal suite vs the dev backend (backend#1184 fast-follow) - #598
Conversation
… fast-follow New e2e-full-seal.sh installs the working-tree chart on real k3d as the dedicated dev e2e-test-agent (real credentials, CLIENT_ENV=dev), waits for every release PVC to Bind and jobs-manager to hold a real backend session, then runs helm test UNFILTERED — egress-enforcement + backend-reachability + storage-assertions in one release, hook-presence-guarded so a regated check can't vanish silently. New helm-ci job full-seal-e2e runs it on push/dispatch only and skips green with a notice until the e2e-test-agent secrets (TB_E2E_CLIENT_ID / TB_E2E_CLIENT_PASSWORD) are provisioned. The egress positive control moves verbatim into e2e-common.sh (one copy, shared with e2e-seal-check.sh). SEAL-CHECK.md gains the CI-coverage map and drops three follow-ups that shipped since (#541 live probe, cli#393 verdict, cli#449 matrix). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
bugbot run |
- credentials travel in a mode-0600 temp values file, never on argv (process-list exposure on a shared runner + helm --set comma/brace mangling); removed on every exit path — installer parity - the PVC wait does one guarded fetch per iteration, so a transient kubectl failure retries until the deadline instead of aborting under set -euo pipefail - full-seal-e2e job timeout 30m -> 45m: the script stacks a 300s PVC wait, two 300s rollouts and a 600s unfiltered helm test on create_cluster's 15m bound — GHA must not kill a slow-but-healthy run Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
All three findings addressed: creds now travel in a 0600 temp values file (never argv — and immune to helm --set's comma/brace mangling), the PVC wait retries through kubectl blips instead of aborting under bugbot run |
…ull-seal install (Bugbot) The chart's PVC default is ReadWriteMany; rancher.io/local-path never provisions it, so every claim sat Pending and both the Bound pre-wait and storage-assertions were guaranteed to fail once the job activates. The installer writes pvcAccessMode: ReadWriteOnce for this exact storage path — the harness now installs with the same value. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Second-round finding fixed: the install now pins bugbot run |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit d639aca. Configure here.

What (the backend#1184 deferred fast-follow)
backend#1184 closed accepting CI-substrate coverage as v1-sufficient, with one recorded residual: "the FULL suite (incl. backend-reachability + bound-PVC storage-assertions) on the dev-env e2e-test-agent with real credentials." This is that harness.
scripts/tests/e2e-full-seal.sh(new)Real k3d cluster → install the working-tree chart as the dedicated dev
e2e-test-agentclient (real creds,CLIENT_ENV=dev, lockdown engaged, same base profile as the sibling) → crisp pre-asserts (every release PVC Bound within 300s — separates "slow to bind" from "assertions wrong"; mysql + jobs-manager rolled out, i.e. a real backend session) → shared egress positive control → hook-presence guard for all three checks (an unfilteredhelm test"passes" a release whose hooks silently stopped rendering — same trap the sibling guards for its--filter) →helm testunfiltered:egress-enforcement+backend-reachability+storage-assertionsin one release. On failure, every seal-check pod log dumps via the §8.2 enumeration label.full-seal-e2ejob (helm-ci)workflow_dispatchonly, never PRs — fork PRs can't read secrets, and a real dev-backend login per PR is platform churn for no extra signal (the secret-free enforcement probe keeps covering PRs).::noticeuntil provisioned — the job is live the moment the secrets exist, no second PR needed.workflow_dispatchadded to helm-ci for on-demand runs (e.g. right after provisioning).Refactor + docs
e2e-seal-check.shintolib/e2e-common.sh(provenance + test(seal-check): run the egress-enforcement probe live in k3d CI (#1184) #541 review rationale documented at the function) — one copy, two callers, no drift.SEAL-CHECK.md: new CI-coverage map (what runs where, activation contract, "never a person's shared dev identity — the backend#1180 token-churn failure class"), and the follow-ups list drops three items that shipped since it was written (test(seal-check): run the egress-enforcement probe live in k3d CI (#1184) #541 live probe, cli#393 CLI verdict, cli#449 §8.3 matrix).installer-tests.yamlshellcheck gates now include the new script; helm-ci path filters gain the new script +lib/e2e-common.sh(which was already load-bearing for the seal job but absent from the filter).Provision a dedicated dev-platform client (
e2e-test-agent) and add two repo Actions secrets:TB_E2E_CLIENT_ID,TB_E2E_CLIENT_PASSWORD. Until then the job skips green by design. (Public repo: fork PRs can't read the secrets; the job never runs on PRs anyway.)Test plan
bash -n+shellcheck --severity=errorgreen on all three touched scripts (both CI shellcheck lists updated).scripts/tests/isn't manifest-covered); style guard clean.seal-check-e2e) runs in THIS PR's CI — proving the factored positive control live on k3d.Epic: tracebloc/backend#1151 (workstream G residual) · closes the backend#1184 fast-follow · RFC-CLI-0003 D12 §8.2
🤖 Generated with Claude Code
Note
Low Risk
Changes are CI/e2e harness and documentation only; no chart runtime or production install path. Real credentials stay in Actions secrets and a dedicated dev test client, with PRs excluded from the credentialed job.
Overview
Adds
e2e-full-seal.shand a helm-cifull-seal-e2ejob to run the complete chart seal suite (egress-enforcement,backend-reachability,storage-assertions) on k3d against the dev backend with dedicatede2e-test-agentcredentials—not just the secret-free egress probe PRs already get.The new script installs the working-tree chart with real creds in a mode-0600 values file, waits for PVCs to bind and for
jobs-managerrollout, verifies all three test hooks exist (guards against silent emptyhelm test), then runs unfilteredhelm testwith failure log dumps via the seal-check label.CI wiring:
workflow_dispatchon helm-ci; path filters include the new script ande2e-common.sh; installer shellcheck lists updated. The job runs only on push/dispatch (not PRs), uses 45m timeout and concurrency so one dev-agent session runs at a time, and skips green with a notice untilTB_E2E_CLIENT_ID/TB_E2E_CLIENT_PASSWORDsecrets exist.Refactor: egress positive-control logic moves from
e2e-seal-check.shintoe2e_egress_positive_controline2e-common.shfor both seal e2e scripts.Docs:
SEAL-CHECK.mdgains a CI coverage map and trims follow-ups that shipped elsewhere.Reviewed by Cursor Bugbot for commit d639aca. Bugbot is set up for automated code reviews on this repo. Configure here.