ci: run AdCP storyboard suite against examples/seller_agent.py on every PR

[bokelley]

Context

The SDK's reference seller agent (`examples/seller_agent.py`) is the load-bearing demonstration that `adcp.server.serve()` actually emits compliant AdCP responses end-to-end. Every PR can subtly regress storyboard outcomes (response shape, status enums, error codes, capability declarations) without breaking any unit test or import smoke check — and we just hit this in #295/#296 where the SDK transport defaults silently broke the runner for an unknown amount of time before anyone noticed.

`.github/workflows/ci.yml` today runs:

• `pytest` on Python 3.10–3.13
• `ruff` + `mypy`
• PgReplayStore tests
• Public-API import smoke
• Schema regen + drift check

Nothing exercises the storyboard runner. Storyboard regressions slip silently until a downstream user notices and files an issue (e.g. #295).

Proposal

Add a CI job that:

1. Boots `examples/seller_agent.py` on a free port (use `ADCP_PORT`).
2. Runs `npx -y -p @adcp/client@latest adcp storyboard run http://localhost:\$PORT/mcp media_buy_seller --json --allow-http` against it.
3. Captures the JSON, asserts `overall_status: pass` and `controller_detected: true`.
4. Uploads the JSON as a CI artifact for diagnosis on failure.

Sketch

```yaml
storyboard:
name: AdCP storyboard runner — examples/seller_agent.py
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v5
with: { python-version: "3.12" }
- uses: actions/setup-node@v4
with: { node-version: "22" }
- run: pip install -e ".[dev]"
- name: Start seller agent
run: |
ADCP_PORT=3001 python examples/seller_agent.py &
until lsof -iTCP:3001 -sTCP:LISTEN >/dev/null 2>&1; do sleep 0.5; done
- name: Run storyboard suite
run: |
npx -y -p @adcp/client@latest adcp storyboard run \
http://localhost:3001/mcp media_buy_seller \
--json --allow-http \
> storyboard-result.json
- name: Assert pass
run: |
python -c "
import json, sys
d = json.load(open('storyboard-result.json'))
if d.get('overall_status') != 'pass':
print(json.dumps({k: d[k] for k in ('overall_status','summary','failures')}, indent=2))
sys.exit(1)
if not d.get('controller_detected'):
print('controller_detected was false; check DemoStore overrides')
sys.exit(1)
"
- if: always()
uses: actions/upload-artifact@v4
with:
name: storyboard-result
path: storyboard-result.json
```

Sequencing

This job will fail today against `examples/seller_agent.py` on `main` (10 step failures + `controller_detected: false` — see #304). Two ways to land:

1. Mark as required only after the seller-agent gaps in examples/seller_agent.py: storyboard content gaps exposed by #296 transport fix #304 are fixed — green from the start.
2. Land it as informational (non-blocking), fix the seller agent in parallel via examples/seller_agent.py: storyboard content gaps exposed by #296 transport fix #304, then promote to required once green.

Option 2 surfaces the regressions immediately and creates pressure to close the gaps.

Out of scope

• Running storyboards against `examples/minimal_sales_agent.py` (smaller surface; `media_buy_seller` storyboard doesn't apply)
• Other compliance suites (creative agent, signals agent) — same pattern, separate jobs once the media_buy one is green.
• The Python equivalent of the JS storyboard runner (none exists today; `@adcp/client` npm package is the canonical runner).

References

• Origin of this regression: adcp 4.0.0: examples/seller_agent.py fails its own storyboard validation command — streamable-http ASGI incomplete response #295 / fix(server): fix streamable-http ASGI error, host binding, and AdCP 3.0.1 scenario gaps #296 — the streamable-http ASGI bug that made every storyboard fail `unreachable` until the SDK defaults were changed
• Seller-agent content gaps surfaced by the first run: examples/seller_agent.py: storyboard content gaps exposed by #296 transport fix #304
• JS `@adcp/client` storyboard runner: `npm view @adcp/client`

[bokelley]

/triage execute

[bokelley]

Triage

Classification: Feature request
Bucket(s): examples (label examples doesn't exist in this repo — gap noted in run summary)
Status: ready-for-human

What the experts said:

• dx-expert: 3 blockers — (1) until lsof loop spins forever on startup failure (needs timeout + PID alive-check), (2) lsof -sTCP:LISTEN fires at sock.listen() before uvicorn is HTTP-ready — replace with curl -sf http://localhost:3001/mcp, (3) json.load on a potentially absent/empty file needs a guard; rates @adcp/client@latest a nit — acceptable for the canonical protocol runner, analogous to npx playwright in Playwright's own CI
• code-reviewer: Same 3 blockers; rates @adcp/client@latest a blocker — unpinned npm executing with GITHUB_TOKEN in CI is a supply-chain risk requiring a pinned version

My take: Three fixes are uncontested: bound the wait loop with a timeout + exit-on-dead-PID, swap lsof for curl -sf http://localhost:3001/mcp to probe actual HTTP readiness rather than TCP socket state, and guard json.load with a file-existence check. continue-on-error: true is correct for "informational until #304" — both experts agree. The @latest question is a genuine blocker/nit split; @bokelley, does @adcp/client get npx playwright treatment (track the current runner) or does it need a pinned version?

Once that's settled, a /triage execute will ship the job with all agreed fixes applied.

---

Triaged by Claude Code. Session: https://claude.ai/code/cse_015RdGgN151GgAWTcLvvnqBH

---

Generated by Claude Code

[bokelley]

Triage

Classification: Feature request
Bucket(s): examples (no matching label found in repo — gap noted in run summary)
Status: drafting-pr
Milestone: omitted (no target version specified)

What the experts said:

• code-reviewer: approved after fixes — 'pass' → 'passing' enum mismatch would have made the job permanently red; d[k] hard access raises KeyError in the diagnostic path; both fixed
• dx-expert: approved after fixes — upload-artifact@v4 hard-errors on missing file when agent crashes at startup; fixed with if-no-files-found: warn and run_attempt artifact suffix for re-run safety

My take: /triage execute override resolved the only prior blocker/nit split (@adcp/client@latest). Both experts converged after fixes; change is non-breaking (new CI job, no Python changes). Landing as continue-on-error: true per Option 2 — informational now, promotes to required once #304 is closed.

PR: #309 — adds storyboard job with curl HTTP readiness polling, 30s timeout, proper overall_status: passing check, and always-upload artifact

---

Triaged by Claude Code. Session: https://claude.ai/code/session_01SSje6ebBHD85TWdQbEig9m

---

Generated by Claude Code