Skip to content

feat(scenarios): expand Terraform and Kubernetes corpus to Network-level depth - #51

Merged
wonkwonlee merged 22 commits into
mainfrom
wonkwonlee/scenario-navigation-and-workbench-2
Jul 31, 2026
Merged

feat(scenarios): expand Terraform and Kubernetes corpus to Network-level depth#51
wonkwonlee merged 22 commits into
mainfrom
wonkwonlee/scenario-navigation-and-workbench-2

Conversation

@wonkwonlee

Copy link
Copy Markdown
Owner

Summary

  • Adds 12 new scenarios (5 Terraform, 7 Kubernetes), bringing the corpus from 13 to 25 scenarios
  • Terraform: 2 → 7 scenarios, now spanning LOW/MEDIUM/HIGH/CRITICAL risk and its full 6-policy set
  • Kubernetes: 2 → 9 scenarios, now spanning LOW/MEDIUM/HIGH/CRITICAL risk and its full 10-policy set
  • Both domains now match Network's existing depth, including one adversarial case each

Notable fixes found along the way

  • Wired the terraform context (untrusted PR/commit text) field through the test-facing scenario registry (scenarios/domains.ts) so UNTRUSTED_INSTRUCTION can be exercised for Terraform
  • Found and fixed the same gap in the CLI-facing registry (packages/cli/src/domains.ts via packages/cli/src/scenario.ts) during final verification — this is the registry CI's corpus job actually uses (scenario check --dir <domain> --domain <domain>), and without this second fix one scenario would have passed npm test locally but failed CI
  • Documented the new context field in docs/SCENARIO_AUTHORING.md

Test plan

  • npm test — 1111 passed / 4 skipped
  • npm run lint — clean
  • npm run typecheck — clean
  • changesafe scenario check --dir scenarios/network --domain network — 9/9 ok
  • changesafe scenario check --dir scenarios/terraform --domain terraform — 7/7 ok
  • changesafe scenario check --dir scenarios/kubernetes --domain kubernetes — 9/9 ok
  • changesafe scenario gallery --check — current, no drift
  • Every new scenario's policy-status claims independently re-derived from the real policy source during task review (not copied from draft guesses)
  • Final whole-branch review: Ready to merge

Design: docs/superpowers/specs/2026-07-30-scenario-corpus-expansion-design.md
Plan: docs/superpowers/plans/2026-07-30-scenario-corpus-expansion.md

🤖 Generated with Claude Code

wonkwonlee and others added 20 commits July 30, 2026 14:51
…rnetes/Terraform replay) (#48)

* feat: add app review contract schemas

* fix: validate contract version mismatches

* test: pin network regression baseline

* test: probe blocked simulation directly

* test: guard core domain presentation boundaries

* feat(ui): add semantic design tokens

* feat(ui): add static review workbench shell

* test: guard design token regex captures

* fix(ui): ground static shell in replay fixture

* feat(ui): expose public replay workbench route

* test(ui): baseline workbench responsive semantics

* feat(ui): disclose workbench runtime variants

* fix(ui): distinguish queue review-record storage

* test(e2e): baseline public workbench

* test(e2e): harden workbench static boundary

* test(e2e): intercept workbench data requests

* test(e2e): add workbench visual baseline

* test(e2e): make workbench baseline portable

* test(e2e): add platform-specific workbench baselines

* feat: add v1 domain review transport contracts

* fix: enforce review risk and replay fallback contracts

* feat: add closed domain runtime registry

* fix: compose immutable session capabilities

* fix: source durability from transport capability

* feat: add pure review workflow controller

* fix: correlate review controller attempts

* fix: bind review input identity at initialization

* feat: add v1 review analyze API contracts

* feat: add registry-backed review analyze route

* feat: add review controller React hook

* fix: bind hook results to active attempts

* refactor: lazily load domain runtimes

* feat: complete review controller lifecycle

* fix: bind receipts to active review workflow

* fix: bind receipt identity provenance

* feat: expose review hook lifecycle actions

* fix: validate review rebind before abort

* feat: add public replay review transport

* refactor: isolate review API contracts

* feat: add network review examples

* feat: compose network review receipts

* feat: publish network policy version in examples

* fix: bind receipts to session policy version

* fix: version review policy binding contract

* feat: add network workflow compatibility facade

* fix: bind legacy analyses to review session

* fix: bind replay responses to requested fixture

* fix: bind replay provenance to requested fixture

* test: reject authored synthetic fixture substitution

* feat: make network workbench replay interactive

* fix: improve workbench replay accessibility

* fix: mark review canvas busy during replay

* test: prove network review parity

* test: bind parity provenance to fixtures

* test: prove workbench replay provenance

* test: make workbench route assertion order-insensitive

* feat: add Terraform replay examples

* feat: resolve Terraform public replay

* fix: bundle Terraform replay context safely

* feat: add Terraform public replay workbench

* test: prove Terraform replay parity

* fix: keep public replay decision-free

* fix: preserve legacy network review lifecycle

* test: distinguish durable review hook sessions

* feat: add Kubernetes replay examples

* test: prove Kubernetes replay derivation

* fix: isolate Kubernetes YAML parser boundary

* fix: isolate kubernetes offline metadata

* feat: resolve Kubernetes public replays

* fix: load Kubernetes runtime through offline boundary

* feat: add Kubernetes public replay workbench

* fix: match Kubernetes selectors against pod labels

* test: prove Kubernetes public replay parity

* feat: add durable review proof contracts

* fix: bind durable intake and receipt proofs

* fix: verify durable records before persistence

* feat: add durable review record store

* fix: harden durable review store integrity

* fix: block durable review replace conflicts

* fix: guard explicit durable review sequence replacement

* fix: migrate durable review conflict trigger

* feat: split pending durable review resolution

* feat: split durable review pending records

* fix: bind durable resolution to parent at database boundary

* feat: add authenticated durable review endpoints

* fix: scope durable review intake provenance

* fix: isolate durable review owner identities

* fix: migrate ownerless durable review schema

* fix: quarantine ownerless review migrations

* fix: preserve exact durable review migrations

* feat: resolve durable reviews server-side

* fix: serialize durable receipt issuance

* fix: harden durable decision claim lifecycle

* feat: expose durable receipt proof

* fix: verify durable recovery authorship

* fix: expose corrupted receipt proof evidence

* feat: add authenticated self-hosted workbench

* fix: harden self-hosted workbench state

* feat: expose domain policy coverage

* feat: document future domain capability boundary

* fix: validate runtime policy coverage metadata

* feat: add accessible network topology tables

* fix: enforce client telemetry privacy

* fix: close client telemetry bypasses

* feat: harden public workbench accessibility

* fix: meet WCAG contrast for muted text

* fix: harden large workbench evidence

* fix: close public workbench review gaps

* test: close domain route family gaps

* test: enforce public client bundle budgets

* fix: harden public client bundle verification

* fix: scan every emitted client chunk

* fix: guard self-hosted client dependencies

* feat: cut over to vnext network workbench

* docs: align operator guidance with vnext workbench

* chore: remove retired live rate limiter

* docs: correct vnext release references

* refactor: retire legacy local review authority

* build: refresh shipped CLI bundle

* test: keep Action self-test fixtures available

* docs: refresh public bundle baselines

* chore: complete multidomain console ultragoal

* Create 2026-07-29-multidomain-console-vnext-g001.md

* fix: adapt vNext scenario consumers to the reorganized scenarios API

Merging origin/main (#47) renamed ScenarioDefinition.bundle to .input and
scenario.title to .label, and split fixtures out as nullable. Update the
public reviews/analyze route, ReviewWorkbenchShell, and the durable review
store test to the new shape.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

* fix: restore the Network route's domain bundle boundary after the scenarios reorg

Merging origin/main pulled in scenarios/index.ts's unified registry, which
resolves every domain (network, terraform, kubernetes) through
scenarios/domains.ts at module scope. Any importer — including
app/page.tsx's Network workbench — therefore pulled in
@changesafe/domain-terraform and @changesafe/domain-kubernetes too,
violating the per-route bundle boundary enforced by
tests/unit/workbench-performance-boundaries.test.ts.

Add scenarios/network.ts as a self-contained, network-only scenario
registry (no dependency on scenarios/domains.ts), have scenarios/index.ts
compose SCENARIOS from it instead of duplicating the definitions, and
repoint the Network route's browser-reachable consumers
(ReviewWorkbenchShell, features/domains/network/examples) at it directly.
Update the remaining test files for the renamed ScenarioDefinition fields
(.bundle -> .input, .title -> .label) and the hardcoded pre-reorg fixture
paths in packages/server/tests/reviews.test.ts.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
…es parity)

Plans 12 new scenarios to bring Terraform and Kubernetes coverage up to
Network's existing risk-level and policy-surface depth.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
checkOne called domain.parseInput with only the raw incident JSON, so the
terraform domain's context (used for UNTRUSTED_INSTRUCTION detection) was
always dropped on the CLI path even though the test-harness registry
already forwarded it, letting scenario-p-injected-pr-context silently pass
CI's corpus-check job with the wrong verdict.
…narrative

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@vercel

vercel Bot commented Jul 30, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
change-safe Ready Ready Preview Jul 31, 2026 7:51am

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: e49df6c74b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread features/reviews/selfHostedReviewTransport.ts
Comment thread app/api/reviews/analyze/route.ts
@wonkwonlee

Copy link
Copy Markdown
Owner Author

Addressed in commit 1d56ed7.

  • Updated the self-hosted transport response schema to accept the server receipt decision values (approved / rejected), and added a regression test covering a successful decision response.
  • Bound Kubernetes replay results to KUBERNETES_PUBLIC_REPLAY_SNAPSHOT.snapshotId and updated the replay/parity expectations accordingly.

Verification: npm run lint; npm run typecheck; npm test (1,112 passed, 4 skipped); npm run build:cli; node packages/cli/dist/changesafe.js scenario check --domain terraform (7 scenarios passed).

@wonkwonlee

Copy link
Copy Markdown
Owner Author

Addressed Codex review feedback:

P1 - Parse the receipt decision values returned by the server

  • Fixed in to accept server receipt decision values (/) instead of request verbs (/)
  • Added regression test to verify the schema accepts the correct values

P2 - Bind Kubernetes results to the snapshot identifier

  • Updated to return the snapshot's snapshotId as the inputId instead of fixture-specific inputId
  • Updated Kubernetes review parity tests to expect the snapshot identifier

Verification: npm test (1112 passed), npm run lint (clean), npm run typecheck (clean)

@wonkwonlee
wonkwonlee merged commit 4d1eb2c into main Jul 31, 2026
6 of 9 checks passed
@wonkwonlee
wonkwonlee deleted the wonkwonlee/scenario-navigation-and-workbench-2 branch July 31, 2026 07:51
@wonkwonlee
wonkwonlee restored the wonkwonlee/scenario-navigation-and-workbench-2 branch July 31, 2026 07:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant