[validation] Close generated Pydantic vs canonical JSON Schema semantic gaps

[bokelley]

Summary

Generated Pydantic models do not enforce all semantics in the canonical JSON Schemas. In particular, allOf + if/then/else conditionals and x-adcp-validation invariants can silently disappear from runtime model validation.

Reproduction from reporting

The reporting revision schema requires finality_basis, finality_policy_id, and finalized_at when finality == "official", and forbids that evidence for non-official revisions. The generated ReportingRevision Pydantic model accepted an official revision without those fields.

The same implementation also had to enforce canonical media-buy set ordering manually because that behavioral invariant is not represented by the generated model.

Impact

An adopter can construct or accept a Python object that appears validated but is rejected by the canonical protocol schema. For billing/reporting this can turn an unsupported finality assertion into apparently valid application state.

Decision needed

Choose and document one supported boundary:

1. generated Pydantic models have validation parity with canonical JSON Schema, including conditionals and selected x-adcp-validation rules; or
2. Pydantic models are structural application types only, and every wire boundary runs a provided canonical-schema validator in addition.

Expected work

• Add differential fixtures that validate the same payload with Pydantic and the bundled canonical JSON Schema.
• Cover conditional required/forbidden fields, discriminated unions, dependent constraints, numeric/string/array bounds, and selected behavioral extensions.
• Fail SDK CI when a generated model accepts a canonical negative fixture unless the divergence is explicitly allowlisted and documented.
• Provide an ergonomic validation helper if full codegen parity is not practical.

Discovered while implementing #1095 against adcontextprotocol/adcp#6953.

[bokelley]

Triage

Classification: Bug — validation gap
Bucket(s): validation (label not in repo — flagging gap in run summary)
Status: ready-for-human

What the experts said:

• ad-tech-protocol-expert: Architectural — gap is a real billing-compliance risk; ReportingRevision with missing finality_basis/finality_policy_id breaks the adcp#6953 audit chain; Option 2 (wire-boundary jsonschema validator, already partially implemented in src/adcp/validation/schema_validator.py) is the correct primary path; Option 1 (full Pydantic parity via codegen) is unsound with datamodel-code-generator and breaking.
• code-reviewer: Analysis was still completing at post time; protocol expert covered implementation-layer findings from direct codebase inspection.

My take: The core decision — full Pydantic conditional parity vs structural types + wire-boundary validator — is architectural with breaking-change implications that needs a maintainer call before implementation begins. A non-breaking third option exists: narrow @model_validator(mode='after') stubs in _ergonomic.py for billing-critical conditionals only (precedented by src/adcp/validation/legacy.py), but all paths hinge on whether bundled schemas already include the if/then/else blocks for ReportingRevision.

Open questions for @bokelley:

1. Do bundled schemas (via scripts/sync_schemas.py) already carry the if/then/else conditionals for ReportingRevision? If yes, scope narrows to _ergonomic.py stubs and CI fixtures only — wire-boundary may already be protected.
2. Does draft PR feat(reporting): prototype reliable ledger reconciliation #1095 introduce these schema fields, or consume them from an upstream schema that already carries them? (Determines whether a schema re-sync must precede SDK work.)
3. Does the idempotency replay cache short-circuit validate_response? If so, replay paths are unprotected even with Option 2 in place.

Additional concern: the RequestContext.metadata context-echo contract (CLAUDE.md) means a structurally-incomplete ReportingRevision can replay null-filled fields into governance agents, and discriminated union shadowing may silently misroute an official revision as provisional when co-required fields are absent.

---

Triaged by Claude Code. Session: https://claude.ai/code/session_01FUxGWD4ut2q3aq6UGxt9mm

---

Generated by Claude Code