Pipeline Plan 20

Here's the implementation plan:

Implementation Plan: End-to-End Integration Test Suite in CI

Files to Create/Modify

File	Action	Purpose
scripts/sw-integration-test.sh	Create	Main integration test script (smoke + live tests)
templates/pipelines/integration.json	Create	Minimal pipeline template (intake → build → test, 3 iterations)
.github/workflows/integration-test.yml	Create	CI workflow: smoke on every PR, live tests with budget cap
package.json	Modify	Add test:integration and test:integration:live scripts
.github/workflows/test.yml	Modify	Add integration smoke tests as parallel job

Implementation Steps

Step 1: templates/pipelines/integration.json

Minimal 3-stage template: intake → build → test. Model: sonnet. Max 3 iterations. No audit/quality gates/PR stage.

Step 2: scripts/sw-integration-test.sh — Tier 1 Smoke Tests

Uses the existing test harness pattern (PASS/FAIL counters, ERR trap, colored output). Mock binaries but runs real pipeline orchestration:

1. Dry-run smoke — verify --dry-run exits 0, outputs "Dry run", creates .claude/ dirs
2. Stage ordering — mock pipeline runs intake → build → test in order, state file updated
3. State file format — validate required fields, stage progress, timestamps
4. Budget enforcement — $0 budget causes clean exit, not crash

Step 3: Tier 2 Live Tests (gated behind INTEGRATION_LIVE=true + ANTHROPIC_API_KEY)

5. README change — real Claude modifies a README in a temp repo; verify git diff shows changes, clean status
6. PR creation — full fast pipeline creates a real PR; verify PR exists and branch is clean
7. Budget cap — live run stays under $1.00; cost tracking confirms

Step 4: .github/workflows/integration-test.yml

• smoke job: every PR, no secrets needed, runs smoke tests
• live job: PRs with ANTHROPIC_API_KEY, budget=$1.00, 15min timeout
• regression job: push to main, re-runs live tests post-merge
• All jobs write per-stage results to $GITHUB_STEP_SUMMARY

Step 5: Update package.json with test:integration scripts

Step 6: Add smoke tests as parallel job in existing test.yml

Task Checklist (10 tasks)

• Create templates/pipelines/integration.json
• Create scripts/sw-integration-test.sh with Tier 1 smoke tests
• Add Tier 2 live integration tests (gated behind env vars)
• Create .github/workflows/integration-test.yml
• Update package.json with new scripts
• Update .github/workflows/test.yml with smoke job
• Add CI summary reporting ($GITHUB_STEP_SUMMARY)
• Run smoke tests locally — all pass
• Verify npm run test:integration works
• Update CLAUDE.md documentation

Definition of Done

• npm run test:integration exits 0 (smoke tests, no API key needed)
• npm run test:integration:live runs both tiers when API key present
• CI runs smoke on every PR, live when secrets available, regression on merge
• Budget hard-capped at $1.00 per live run
• Per-stage pass/fail in CI summary
• Existing npm test still passes