Skip to content

Enforce schema contract: levers count + required summary#276

Merged
neoneye merged 1 commit intomainfrom
fix/enforce-schema-contract
Mar 15, 2026
Merged

Enforce schema contract: levers count + required summary#276
neoneye merged 1 commit intomainfrom
fix/enforce-schema-contract

Conversation

@neoneye
Copy link
Member

@neoneye neoneye commented Mar 15, 2026

Summary

  • Add min_length=5, max_length=5 to DocumentDetails.levers to enforce exactly 5 levers per response
  • Make summary a required str field (was Optional[str] with default=None)

Evidence

  • Run 52 (haiku) produced 16 levers (hong_kong_game) due to unconstrained list
  • Null summaries in runs 47 (14/15), 48 (10/15), 51 (4/15) silently accepted
  • Both code-review agents independently identified these as the top priority fix

Expected Impact

  • 6-lever overflow becomes a parse-time error instead of silently corrupting output
  • Null summaries become parse-time errors instead of being saved as successful
  • All models affected; no prompt or logic changes required

Test plan

  • Run experiments with all 7 models
  • Verify lever count violations cause parse errors (not silent corruption)
  • Verify summary is populated in successful runs
  • Run analysis pipeline and check assessment verdict

🤖 Generated with Claude Code

…required

- Add min_length=5, max_length=5 to DocumentDetails.levers Field
- Change summary from Optional[str] with default=None to required str
- Turns silent 6-lever overflow (run 52) and null summaries (runs 47/48/51)
  into parse-time errors that can be caught, logged, or retried

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@neoneye neoneye merged commit 91ec3fa into main Mar 15, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant