[bench] constitution: split rule from commentary, send the models the rule only (v7) - #50
Merged
Conversation
… rule only (v7) Every stage prompt carried the whole of bench.json, so each byte was paid for three times per governed edit and C-008's 3,067-character rule was the largest single item. Roadmap v2.1 item 2.1. - pipeline/constitution.py: optional per-constraint `commentary` field (string when present) and prompt_view(), which renders a constraint as id, name, scope, rule, severity, and the raised-severity marker. Rationale and commentary never reach a model. The view is derived; the ledger's constitution_hash still covers the authored file. - challenger, defender, oracle: _build_user_content sends prompt_view(constitution). All three stages see identical text. - cli constitution: prints the rule and, when present, the commentary on a line labeled "not sent to models". - bench.json v6 -> v7: C-004 and C-008 move auditor procedure and explanatory prose from `rule` into `commentary`. Every (a)-(d) requirement in C-008 stays in the rule. Nothing a constraint forbids changes. - tests/test_constitution_prompt_view.py: field selection, verbatim rule passthrough, commentary schema, each builder omits rationale and commentary, and a budget gate on Bench's own constitution (6,000 chars, 1,500 tokens at 4 chars/token). The gate failed at v6 (6,522 chars) and passes at v7 (5,522). - README: constitution section, per-project schema, cost paragraph. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WVBooRMkd73P8J3PPLt1Uu
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 940a081129
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… address review Codex: the README says `cli constitution` shows rationale and commentary apart from the rule, but the command printed commentary only. Both now print on lines labeled "not sent to models", and a test asserts it. Sonar S3776: cmd_constitution's per-constraint block moves to _print_constraint, with _preview replacing three copies of the truncation. Sonar S5778: the commentary rejection test builds its path outside the assertRaises block so only one call can throw. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01WVBooRMkd73P8J3PPLt1Uu
Contributor
Author
|
@codex review |
|
|
Codex Review: Didn't find any major issues. Nice work! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Roadmap v2.1 item 2.1. Every stage prompt carried the whole of bench.json, so each byte was paid for three times per governed edit.
What changes
pipeline.constitution.prompt_view()renders a constraint as id, name, scope, rule, severity, and the raised-severity marker.rationaleand the new optionalcommentaryfield never reach a model. The view is derived: the ledger'sconstitution_hashstill covers the authored file.prompt_view(constitution). A test asserts the Challenger, Defender, and Oracle see identical constitutional text.python -m cli constitutionprints the rule and, when present, the commentary on a line labeled "not sent to models" (release check R2).ruleintocommentary. Every (a) through (d) requirement in C-008 stays in the rule word for word. Nothing a constraint forbids changes. The roadmap card says 5 to 6; the constitution was already at 6 since the 2026-08-22 sanitation amendment.tests/test_constitution_prompt_view.pyasserts Bench's own constitution renders under 6,000 characters (1,500 tokens at four chars per token, no tokenizer dependency). It failed at v6 and passes at v7:Median tokens per edit on the ledger before this PR: 36,146 (p90 47,696). The dashboard will show the after figure once entries accrue on v7.
Receipts
Ten governed edits, all PASS. Ruff and strict mypy clean. Targeted suites green (constitution, layers, challenger, defender, oracle, commands, runner, input validation, readme claims, ledger hygiene).
python -m cli verifyVALID.