Skip to content

feat(addie): render fix plans for the four 5.18 hint kinds#3220

Merged
bokelley merged 1 commit intomainfrom
bokelley/extend-hint-formatter-kinds
Apr 25, 2026
Merged

feat(addie): render fix plans for the four 5.18 hint kinds#3220
bokelley merged 1 commit intomainfrom
bokelley/extend-hint-formatter-kinds

Conversation

@bokelley
Copy link
Copy Markdown
Contributor

Closes the Addie-side followup tracked in adcp-client#935.

Summary

`@adcp/client` 5.18.0 widened `StoryboardStepHint` from a single-kind (`context_value_rejected`) to a five-kind union. The original formatter (#3084) only rendered `context_value_rejected`; the four new kinds (`shape_drift`, `missing_required_field`, `format_mismatch`, `monotonic_violation`) silently dropped.

This PR extends the formatter with a Diagnose / Locate / Fix / Verify render arm per kind, dispatched off `hint.kind`. Unknown future kinds return `null` so the upstream `hint.message` still surfaces them at the consumer's discretion.

Verbatim renders (from the inline snapshots)

`shape_drift` (canonical bare-array case)

```
💡 Wire-shape drift detected. Your `list_creatives` response doesn't match the envelope the spec requires.

Diagnose — observed: `bare_array`. Expected: `{ creatives: [...] }`.

Locate — at the response root.

Fix — reshape the response to match the expected envelope. `@adcp/client/server` ships typed response builders (e.g. `listCreativesResponse`, `getMediaBuysResponse`, `buildCreativeResponse`) — using one of those gives you the spec-correct shape from a single helper call and keeps the typing tight when the spec evolves.

Verify — re-run `run_storyboard_step` with `step_id: "list-creatives"` and the same context.
```

`missing_required_field`

```
💡 Required-field gap detected. Your `list_creatives` response is missing fields the spec requires.

Diagnose — missing required fields: `total_count`, `has_more`.

Locate — at the response root; the schema requirement is at `#/required` (schema: `https://adcp/list-creatives-response.json\`).

Fix — populate each missing field with a value matching the schema's type for it. The typed response builders in `@adcp/client/server` enforce the requirement at the type level, so emitting through one of those prevents this class of failure.

Verify — re-run `run_storyboard_step` with `step_id: "list-creatives"` and the same context.
```

`format_mismatch` (with cheat sheet for common AJV keywords)

```
💡 Strict format violation. Your `list_creatives` response has a value the lenient validator accepts but strict (AJV) rejects — the kind of thing a strict dispatcher would block in production.

Diagnose — strict `format` keyword rejected at `/creatives/0/url`.

Locate — schema names the constraint at `#/properties/creatives/items/properties/url` (schema: `https://adcp/list-creatives-response.json\`).

Fix — emit a value matching the constraint. Common cases:

  • `format: date-time` → ISO 8601 with timezone, e.g. `2026-04-25T15:00:00Z`
  • `format: uri` → fully-formed URL with scheme + host
  • `format: uuid` → 8-4-4-4-12 hex with hyphens
  • `pattern` → see the regex in the schema
  • `enum` → pick from the schema's allowed list

Verify — re-run `run_storyboard_step` with `step_id: "list-creatives"` and the same context.
```

`monotonic_violation`

```
💡 Lifecycle violation detected. Your `media_buy` `mb_001` transitioned `active` → `pending_creative`, which isn't on the spec's lifecycle graph.

Diagnose — from `active`, the only legal next states are: `paused`, `completed`, `cancelled`.

Locate — the previous status was set at step `create_media_buy`. Lifecycle graph: `https://adcp/enums/media-buy-status.json\`.

Fix — pick one of: `paused`, `completed`, `cancelled`. If `pending_creative` should be reachable from `active`, that's a spec gap — file an issue against the lifecycle enum.

Verify — re-run `run_storyboard_step` with `step_id: "update-media-buy"` and the same context.
```

A separate terminal-state branch fires when `legal_next_states` is empty (the resource was already terminal).

Files

  • `server/src/addie/services/storyboard-fix-plan.ts` — refactored dispatcher; four new render functions; widened `FixPlanInput.hint` type to `StoryboardStepHint`; trust-model docstring per kind; polymorphic dedup
  • `server/tests/unit/storyboard-fix-plan.test.ts` — 24 new field-level tests
  • `server/tests/unit/storyboard-fix-plan-snapshot.test.ts` — 5 new inline snapshots
  • `server/tests/unit/storyboard-fix-plan-e2e.test.ts` — second e2e via real MCP transport (`build_creative` returning platform-native top-level keys → `platform_native_fields` shape_drift variant)
  • `.changeset/extend-hint-formatter-four-kinds.md` — empty changeset

Trust model

The formatter sanitizes every seller-controlled field at its boundary regardless of upstream docstrings. Per kind:

  • `shape_drift`: all runner-controlled
  • `missing_required_field` / `format_mismatch`: `instance_path` may leak seller keys when `additionalProperties: true` is allowed; `missing_fields[]` extracted from AJV via regex (fallback emits raw AJV message); `schema_url` is storyboard-author-controlled
  • `monotonic_violation`: `resource_id`, `to_status`, AND `from_status` (the previously-recorded status from a prior step's seller response) are all seller-controlled

Reviews

  • code-reviewer: LGTM, no blockers
  • security-reviewer: 2 Should Fix + 1 nit, all addressed:
    • S1 — trust-model docstring corrected (`from_status` is seller-controlled; code already sanitized it)
    • S2 — `MAX_EXPECTED_VARIANT_LEN` tightened 400 → 120 (longest legitimate value is ~70)
    • N1 — `'truncated'` sentinel check now reads the raw `hint.keyword`
    • N2 (filed as adcp#3219) — `step.error` / `validation.error` raw rendering in member-tools.ts bypasses the formatter's sanitizer; defense-in-depth gap, out of scope here

Test plan

  • `npx tsc --noEmit -p server/tsconfig.json` — clean
  • 58/58 tests pass across the three test files (29 unit + 8 snapshots + 2 e2e)
  • e2e against real MCP transport via `runAgainstLocalAgent`
  • CI green

🤖 Generated with Claude Code

`@adcp/client` 5.18.0 widened `StoryboardStepHint` from a single-kind
(`context_value_rejected`) to a five-kind union — `shape_drift`,
`missing_required_field`, `format_mismatch`, `monotonic_violation` are
the four new kinds the runner emits but the formatter previously
silently dropped.

This adds a Diagnose / Locate / Fix / Verify render arm per kind off
the structured fields the runner provides. The dispatcher in
`renderHintFixPlan` switches on `hint.kind`; unknown future kinds
return null so the upstream `hint.message` still surfaces them at the
consumer's discretion. `renderAllHintFixPlans`'s dedup is now
polymorphic per-kind.

Trust model documented per kind in the formatter's docstring. Every
seller-controlled field (rejected_value, accepted_values, error_code,
request_field, resource_id, to_status, from_status, AJV instance_path
when additionalProperties: true) flows through `sanitizeAgentString`
at the formatter boundary, regardless of what intermediate runner
docstrings claim about who controls the bytes.

Tests:
- 24 new field-level assertions (storyboard-fix-plan.test.ts)
- 5 new inline snapshots — one per kind plus the terminal-state
  monotonic_violation branch (storyboard-fix-plan-snapshot.test.ts)
- Second e2e through real MCP transport — `build_creative` returning
  platform-native top-level keys provokes the platform_native_fields
  shape-drift variant (storyboard-fix-plan-e2e.test.ts)

Closes the Addie-side followup tracked in adcp-client#935.

Reviews addressed:
- code-reviewer: LGTM
- security-reviewer: 2 Should Fix + 1 nit. Docstring corrected
  (monotonic_violation.from_status is seller-controlled — code already
  sanitized it, only docstring was wrong); MAX_EXPECTED_VARIANT_LEN
  tightened 400→120; truncated-sentinel branch now checks the raw
  `hint.keyword` rather than the post-sanitize value. Filed adcp#3219
  for the residual `step.error`/`validation.error` raw-rendering
  exposure noted in N2 (out of scope for this PR).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bokelley bokelley merged commit d505769 into main Apr 25, 2026
13 checks passed
@bokelley bokelley deleted the bokelley/extend-hint-formatter-kinds branch April 25, 2026 23:25
bokelley added a commit that referenced this pull request Apr 25, 2026
… output (#3233)

Closes #3219.

The hint fix-plan formatter (adcp#3084, #3220) carefully sanitized every
seller-controlled field at its boundary. But sibling renders on the
same StoryboardStepResult — `step.error`, `validation.error`,
`result.next.narrative` — and adjacent tools (`evaluate_agent_quality`,
`compare_media_kit`, `test_io_execution`, `get_storyboard_detail`)
emitted runner/agent strings raw, letting a hostile seller bypass the
formatter's prompt-injection protection through a sibling field on the
same Claude-bound output.

This pass routes every such site through `sanitizeAgentField`. Adds a
new `RUNNER_ERROR_MAX_LEN` constant (400) with a docstring framing it
explicitly as a prompt-injection budget — not a UX choice — so future
contributors don't raise it without thinking about blast radius.

Sites covered (per triage on #3219):
- `evaluate_agent_quality` per-step error rendering (line 3399)
- `run_storyboard` step.error + per-validation error (lines 3746, 3749)
- `run_storyboard_step` per-validation + result error (lines 3857, 3861)
- `run_storyboard_step` next.narrative preview (line 3892)
- `compare_media_kit` per-brief error (line 4113)
- `test_io_execution` failure message (line 4720)
- `get_storyboard_detail` storyboard / phase / step narratives — added
  for consistency since they render onto the same LLM-bound output
  path through the same trust model.

CODE_VERSION bumped 2026.04.4 → 2026.04.5 (MCP tool output behavior
change, per playbook).

Reviews on #3219 (already complete in the triage comment):
- security-reviewer: fix is correct; flagged the same additional sites
  the triage scoped in
- prompt-engineer: 400-char cap should be documented as injection
  budget — done via the `RUNNER_ERROR_MAX_LEN` docstring

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant