Skip to content

feat(compliance): pagination cursor↔has_more invariant lint and storyboard#3095

Merged
bokelley merged 1 commit into
mainfrom
bokelley/pagination-honesty-test
Apr 25, 2026
Merged

feat(compliance): pagination cursor↔has_more invariant lint and storyboard#3095
bokelley merged 1 commit into
mainfrom
bokelley/pagination-honesty-test

Conversation

@bokelley
Copy link
Copy Markdown
Contributor

@bokelley bokelley commented Apr 25, 2026

Summary

  • New build-time lint (scripts/lint-pagination-invariant.cjs) scans schema examples[] and storyboard sample_request/sample_response for has_more=true without cursor and has_more=false with stale cursor. Wired into build-compliance so CI gates on it.
  • New universal storyboard (pagination_integrity) seeds three creatives, walks list_creatives with max_results=2, asserts the invariant on a continuation page (has_more=true, cursor present, captured) followed by a terminal page (has_more=false, cursor absent or null).
  • Fixes the training agent's list_creatives to honor pagination.max_results (default 50, capped at 100 per the request schema) and emit an opaque base64url offset cursor when the page is non-terminal. Malformed cursors surface INVALID_REQUEST rather than silently restarting at offset 0.

Why

Brian flagged that "most adapters' pagination returns has_more: false without populating total_count — works but isn't fully honest if the platform paginates server-side." The schema permits omitting total_count, but the prose contract on pagination-response.json (cursor "Only present when has_more is true") was unenforced. Two complementary guards close the gap: a static lint on authored examples, and a runtime storyboard that walks pagination on a real agent.

The training agent fix was discovered when running the storyboard against it: every list_creatives response carried has_more: false regardless of max_results. That was the exact dishonest shape the storyboard catches; ref-implementation should not teach it.

Reviewed by

Three subagents reviewed before push. Findings addressed:

  • Code review: decodeCreativeCursor now returns INVALID_REQUEST on malformed input (was: silent reset to 0); removed silent lower-bound clamp on max_results; added WeakSet cycle guard in walkPaginationObjects.
  • Test architecture: added three coverage tests — malformed JSON guard, malformed YAML / missing-phases guard, <unnamed> fallback rendering, plus a self-referential cycle test for the walker (16 tests total, all passing).
  • Protocol: added inline comment in the lint explaining the intentional asymmetry vs. the storyboard (lint enforces strict cursor absence on authored fixtures; storyboard tolerates cursor: null at runtime per runner-output-contract.yaml's absent-as-null reporting).

Out of scope (deferred)

total_count monotonicity check (total_count >= returned_so_far) — protocol reviewer flagged as a free honesty check that doesn't require upstream visibility. Worth a follow-up PR; not bundling here to keep the surface small.

Test plan

  • npm run build — full schema/compliance/tarball build (CI step)
  • npm run test:server-unit — 2151 tests, 1 LLM flake unrelated to changes (passes on retry)
  • npm run test:unit — 820/820 pass
  • npm run test:schemas && test:json-schema && test:extension-schemas && test:composed — all clean
  • npm run check:registry && test:platform-agnostic — both clean
  • npm run test:hmac-vectors && test:hmac-signer-conformance — both clean
  • npm run test:pagination-invariant — 16/16 pass
  • All 8 storyboard lints (auth-shape, branch-sets, contradictions, context-entity, scoping, test-kits, sample-request-schema, pagination-invariant)
  • pagination_integrity storyboard against training agent: 6/6 steps clean
  • All 8 creative storyboards still pass (61/62 steps, 1 expected skip in a branch_set)

🤖 Generated with Claude Code

@bokelley @claude
feat(compliance): pagination cursor↔has_more invariant lint and story…
e2f8f01 
…board

Adds a build-time lint that scans schema examples and storyboard fixtures
for the two violation classes the prose contract on pagination-response.json
doesn't enforce: has_more=true without cursor, and has_more=false with a
stale cursor. Wired into build-compliance so CI gates on it.

Adds a universal storyboard (pagination_integrity) that seeds three
creatives, walks list_creatives with max_results=2, and asserts the
invariant on a continuation page (has_more=true, cursor present, captured)
followed by a terminal page (has_more=false, cursor absent or null).

Fixes the training agent's list_creatives to honor pagination.max_results
(default 50, capped at 100 per the request schema) and emit an opaque
base64url offset cursor when the page is non-terminal. Malformed cursors
now surface INVALID_REQUEST instead of silently restarting from offset 0.
Previously every response carried has_more: false regardless of input —
the exact dishonest shape the new storyboard catches.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@bokelley bokelley merged commit df09cfb into main Apr 25, 2026
15 checks passed
@bokelley bokelley deleted the bokelley/pagination-honesty-test branch April 25, 2026 09:54
@bokelley bokelley mentioned this pull request Apr 25, 2026
Merged
7 tasks
bokelley added a commit that referenced this pull request Apr 25, 2026
@bokelley @claude
feat(compliance): pagination total_count honesty assertions (#3100)
6099ff6 
Extends `pagination_integrity` with three runtime assertions per page:
- `query_summary.total_matching = 3` (always — schema requires the field)
- `query_summary.returned` matches the slice (2 on continuation, 1 on terminal)
- `pagination.total_count = 3` when volunteered (field_value_or_absent
  preserves the schema's optional stance)

Catches the dishonest pagination class where an agent honors max_results
and the cursor handshake but under-reports total_count to hide inventory
the same way a dishonest has_more: false would. Verified by flipping the
training agent's total_count to page-local count — page 1 assertion
fires with the expected diagnostic.

Closes the deferred follow-up flagged by the protocol reviewer on #3095.

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced Apr 25, 2026
Closed
Closed
Closed
Merged
Merged
Closed
Closed
Closed
bokelley added a commit that referenced this pull request Apr 25, 2026
@bokelley @claude
feat(compliance): get_signals pagination cursor integrity
14ba482 
Adds get_signals_pagination_integrity universal storyboard mirroring the
cursor↔has_more invariant that gates list_creatives (#3095/#3100) onto
the signals marketplace. Page 1 asserts has_more=true with a cursor
under a broad query; page 2 follows the cursor and asserts schema
conformance.

Fixes the training agent's handleGetSignals to honor pagination.max_results
/ pagination.cursor (was: capped at MAX_SIGNAL_RESULTS=10 internally and
emitted no pagination block — exactly the dishonest shape this storyboard
catches). Reads pagination.max_results in preference to legacy top-level
max_results for forward compatibility.

Generalizes the offset cursor codec from #3095 into a kind-prefixed pair
so cursors can't be replayed across endpoints. encodeCreativeCursor /
decodeCreativeCursor preserved as wrappers; list_creatives behavior
unchanged.

Negative-test verified: flipping the agent back to has_more: false fires
the page-1 assertion with the expected diagnostic.

Closes the second target in the rolling pagination conformance series.
list_creative_formats deferred to a separate issue (#3108) pending a
seed_creative_format scenario; list_accounts deferred to #3106 pending
the missing handler.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bokelley added a commit that referenced this pull request Apr 25, 2026
@bokelley @claude
feat(compliance): get_signals pagination cursor integrity (#3109)
9525717 
* feat(compliance): get_signals pagination cursor integrity

Adds get_signals_pagination_integrity universal storyboard mirroring the
cursor↔has_more invariant that gates list_creatives (#3095/#3100) onto
the signals marketplace. Page 1 asserts has_more=true with a cursor
under a broad query; page 2 follows the cursor and asserts schema
conformance.

Fixes the training agent's handleGetSignals to honor pagination.max_results
/ pagination.cursor (was: capped at MAX_SIGNAL_RESULTS=10 internally and
emitted no pagination block — exactly the dishonest shape this storyboard
catches). Reads pagination.max_results in preference to legacy top-level
max_results for forward compatibility.

Generalizes the offset cursor codec from #3095 into a kind-prefixed pair
so cursors can't be replayed across endpoints. encodeCreativeCursor /
decodeCreativeCursor preserved as wrappers; list_creatives behavior
unchanged.

Negative-test verified: flipping the agent back to has_more: false fires
the page-1 assertion with the expected diagnostic.

Closes the second target in the rolling pagination conformance series.
list_creative_formats deferred to a separate issue (#3108) pending a
seed_creative_format scenario; list_accounts deferred to #3106 pending
the missing handler.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(training-agent): preserve legacy 50-cap on top-level get_signals max_results

Code-reviewer flagged on #3109: the cap on top-level `max_results` shifted
from 50 (prior behavior) to 100 (new code's uniform cap) when I unified
the read path. No spec basis for either cap on the top-level field, but
silently widening it is the wrong direction for a behavioral preserve.

Restructure: pagination.max_results gets the schema's documented 100 cap,
top-level max_results keeps the historical 50 cap. The two forms
diverge intentionally — pagination is the standard envelope, top-level
is the predecessor. Spec ambiguity on which form wins when both are
present is tracked at #3113.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(conformance): add get_signals_pagination_integrity to universal-storyboards tables

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This was referenced Apr 25, 2026
Closed
Merged
bokelley added a commit that referenced this pull request Apr 25, 2026
@bokelley @claude
fix(training-agent): wire list_accounts handler + reuse shared cursor…
76c0fed 
… codec

Addresses three blockers from the independent expert review on #3110:

1. **Idempotency_key.** The `sync_three_accounts` setup step on the new
   storyboard omitted `idempotency_key`, failing the build-time
   storyboard-idempotency lint. Adds the conventional
   `$generate:uuid_v4#<storyboard>_<phase>_<step>` alias.

2. **Cursor codec duplication.** Triage shipped private
   `encodeAccountCursor` / `decodeAccountCursor` in account-handlers.ts
   with format `offset:N`. PR #3095 (now merged) generalized the codec
   to a kind-prefixed pair. Moves the shared codec to
   `server/src/training-agent/pagination.ts` (avoids the
   task-handlers ↔ account-handlers circular import the in-place
   re-export would create) and replaces the local helpers with
   `encodeOffsetCursor('accounts', n)` / `decodeOffsetCursor('accounts', c)`.
   list_creatives and get_signals handlers move to the shared module
   with no behavior change.

3. **Bootstrap-pattern deviation note.** The seed-DAG documented at
   storyboard-schema.yaml does not include accounts; this storyboard
   uses `sync_accounts` as setup rather than `controller_seeding: true`.
   Adds explicit narrative explaining why and pointing at a future
   `seed_account` controller scenario.

Also fixes:

- **Wire-shape leak.** `accountStateToWire()` was passing `account.brand`
  through verbatim, including a `name` operational hint that brand-ref.json
  forbids (additionalProperties: false). Strips to declared fields only
  (`domain`, optional `brand_id`); display name and advertiser fields are
  still derived from the operational hint but never round-trip on the
  wire. Compliance fixture pool's brand entries cleaned up to match.

- **Framework registration.** Triage added `handleListAccounts` to
  `HANDLER_MAP` but never registered the tool through the
  framework-server's `accounts:` block, so the SDK never advertised
  `list_accounts` and the storyboard's pagination steps were skipped
  (no-such-tool). Wires it alongside `sync_accounts`.

- **Doc-parity.** Adds the new storyboard to both
  `docs/building/conformance.mdx` and `docs/building/compliance-catalog.mdx`
  per the lint that gates the build.

Verified: negative-test flips `has_more` to false on the agent — first-page
assertion fires with `Expected true, got false`. Reverted, clean run
restored. 4/4 storyboard steps pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
bokelley added a commit that referenced this pull request Apr 25, 2026
@bokelley @claude
feat(training-agent): add list_accounts handler with pagination confo…
5b6d7c8 
…rmance (#3110)

* feat(training-agent): add list_accounts handler with pagination conformance

Closes #3106

https://claude.ai/code/session_01N6Bgto9hbbN5pepe9uhgxp

* fix(training-agent): wire list_accounts handler + reuse shared cursor codec

Addresses three blockers from the independent expert review on #3110:

1. **Idempotency_key.** The `sync_three_accounts` setup step on the new
   storyboard omitted `idempotency_key`, failing the build-time
   storyboard-idempotency lint. Adds the conventional
   `$generate:uuid_v4#<storyboard>_<phase>_<step>` alias.

2. **Cursor codec duplication.** Triage shipped private
   `encodeAccountCursor` / `decodeAccountCursor` in account-handlers.ts
   with format `offset:N`. PR #3095 (now merged) generalized the codec
   to a kind-prefixed pair. Moves the shared codec to
   `server/src/training-agent/pagination.ts` (avoids the
   task-handlers ↔ account-handlers circular import the in-place
   re-export would create) and replaces the local helpers with
   `encodeOffsetCursor('accounts', n)` / `decodeOffsetCursor('accounts', c)`.
   list_creatives and get_signals handlers move to the shared module
   with no behavior change.

3. **Bootstrap-pattern deviation note.** The seed-DAG documented at
   storyboard-schema.yaml does not include accounts; this storyboard
   uses `sync_accounts` as setup rather than `controller_seeding: true`.
   Adds explicit narrative explaining why and pointing at a future
   `seed_account` controller scenario.

Also fixes:

- **Wire-shape leak.** `accountStateToWire()` was passing `account.brand`
  through verbatim, including a `name` operational hint that brand-ref.json
  forbids (additionalProperties: false). Strips to declared fields only
  (`domain`, optional `brand_id`); display name and advertiser fields are
  still derived from the operational hint but never round-trip on the
  wire. Compliance fixture pool's brand entries cleaned up to match.

- **Framework registration.** Triage added `handleListAccounts` to
  `HANDLER_MAP` but never registered the tool through the
  framework-server's `accounts:` block, so the SDK never advertised
  `list_accounts` and the storyboard's pagination steps were skipped
  (no-such-tool). Wires it alongside `sync_accounts`.

- **Doc-parity.** Adds the new storyboard to both
  `docs/building/conformance.mdx` and `docs/building/compliance-catalog.mdx`
  per the lint that gates the build.

Verified: negative-test flips `has_more` to false on the agent — first-page
assertion fires with `Expected true, got false`. Reverted, clean run
restored. 4/4 storyboard steps pass.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* test(training-agent): bump expected tool count to 49 for list_accounts

---------

Co-authored-by: Claude <noreply@anthropic.com>
This was referenced Apr 25, 2026
Closed
Merged
bokelley added a commit that referenced this pull request Apr 25, 2026
@bokelley @claude
feat(training-agent): add cursor pagination to list_content_standards…
28956dd 
…, list_collection_lists, list_property_lists (#3147)

* feat(training-agent): add cursor pagination to governance list handlers (#3112)

Adds cursor-based pagination to list_content_standards, list_collection_lists,
and list_property_lists — the fifth batch in the rolling pagination conformance
series (#3095, #3100, #3109, #3110).

https://claude.ai/code/session_018AkoeaWmnrsnbXBtK8FLD2

* fix(compliance): scope identity + idempotency_key + valid channel on governance pagination storyboards

Triage's #3147 hit two build-time lints CI surfaced:

1. **Storyboard scoping** — 15 sample_request blocks (3 storyboards × 5 steps
   each, minus capability_discovery) omitted brand/account identity. The
   create_* and list_* tasks for governance lists are tenant-scoped per
   `lint-storyboard-scoping`. Adds the canonical
   `account: { brand: { domain: 'acmeoutdoor.example' }, operator: 'pinnacle-agency.example' }`
   to all 15.

2. **Idempotency_key on mutating setup steps** — 9 create_* sample_requests
   (3 per storyboard) on `create_collection_list`/`create_content_standards`/
   `create_property_list` omitted `idempotency_key`, which their request
   schemas mark as required. Adds `$generate:uuid_v4#<storyboard>_setup_<step>`
   per the established convention.

3. **Invalid channel value** — `create_standards_2` used `channels_any: ["video"]`
   which isn't in the channels enum. Replace with `["olv"]` (the standardized
   value for online video advertising outside CTV per
   `static/schemas/source/enums/channels.json`).

Verified: 8/8 pagination storyboards pass against the training agent
(list_creatives, get_signals, list_creative_formats, list_accounts,
get_media_buys, content_standards, collection_lists, property_lists)
— 41/41 steps clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@bokelley