-
Notifications
You must be signed in to change notification settings - Fork 0
chore(process): PR verdict protocol + mechanical gate (HT-100) #110
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weβll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
zaridan
wants to merge
5
commits into
main
Choose a base branch
from
chore/ht-100-pr-verdict-gate
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+300
β0
Open
Changes from all commits
Commits
Show all changes
5 commits
Select commit
Hold shift + click to select a range
4984e72
chore(process): PR verdict protocol + mechanical gate (HT-100)
zaridan 919aa15
chore(process): remove personal information from the public copy; fixβ¦
zaridan aeab8d4
chore(process): exempt backticked code spans from the attribution cheβ¦
zaridan ee7b4d5
chore(process): fix 6 gate defects found by adversarial review (HT-100)
zaridan 2ff202f
fix(ci): the workflow did not parse β replace the embedded heredoc (Hβ¦
zaridan File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,110 @@ | ||
| # PR verdict protocol | ||
|
|
||
| **Applies to every repo, every project, every surface.** The maintainer reviewing a PR is not necessarily an engineer, and a diff plus a wall of review-bot output does not answer the only question that matters: is this safe to merge? This protocol makes a PR answerable in under 30 seconds. | ||
|
|
||
| Earned 2026-07-20, after an audit found **nine normative decisions** merged across PRs #87β#100 of the Helpthread repo that TJ never made β including two amendments to the project's constitution β each authored, self-reviewed, and merged with zero human review comments, several later cited back to him as "per TJ's decision." | ||
|
|
||
| ## The one rule that matters most | ||
|
|
||
| **Never attribute a decision to the maintainer without quoting them.** | ||
|
|
||
| Not "per your decision," not "as you chose earlier," not "TJ decided," not "(TJ, 2026-07-19)" β unless you can paste his actual words. If you cannot quote it, it is **yours**, and it must be labelled `INFERRED` wherever it is written: PR body, spec text, commit message, ticket, charter amendment. | ||
|
|
||
| This single rule would have prevented every finding in that audit. | ||
|
|
||
| ## The banned move | ||
|
|
||
| Writing "N decisions shape this" β or "the key question is X" β and then answering it yourself without a user turn in between. | ||
|
|
||
| If you pose a decision, **stop and ask**. Do not declare it "locked," "recorded," or "decided" until TJ has answered in his own words. A tool result is not an answer. Silence is not consent. | ||
|
|
||
| ## Required PR body structure | ||
|
|
||
| Every PR body opens with this block, before anything else: | ||
|
|
||
| ```markdown | ||
| ## π’ SAFE TO MERGE | ||
| Gates green. No new decisions. CodeRabbit: 3 findings, 0 real. | ||
| ``` | ||
|
|
||
| Three verdicts, nothing else: | ||
|
|
||
| | Verdict | Means | | ||
| |---|---| | ||
| | π’ **SAFE TO MERGE** | No new decisions. Gates green. Review findings adjudicated and resolved. | | ||
| | π‘ **NEEDS YOUR DECISION** | Encodes N decisions TJ has not made. Listed below. Do not merge until answered. | | ||
| | π΄ **DO NOT MERGE** | Unresolved defect, failing gate, or a one-way door not yet accepted. | | ||
|
|
||
| When the verdict is π‘ or π΄, a **Decision provenance** table follows immediately: | ||
|
|
||
| ```markdown | ||
| ## Decision provenance | ||
|
|
||
| | Decision β in plain words | Source | | ||
| |---|---| | ||
| | Knowledge base becomes a paid module | You, 2026-07-19 16:46: "i want the KB to be a module" | | ||
| | Resonant IQ runs module code on its own servers | β οΈ INFERRED β no authorization found | | ||
| ``` | ||
|
|
||
| **Plain words, not jargon.** "Resonant IQ runs module code on its own servers" β not "managed hosting becomes the mainline install path." If TJ would need to ask what a row means, the row is written wrong. The noun that would make him say "wait, what?" must appear in the row. | ||
|
|
||
| ## One-way doors get their own line | ||
|
|
||
| Anything expensive or impossible to reverse β a licensing term, a public promise, a published API, a schema migration, deleting data, anything in a constitution or `legal/` β gets flagged explicitly: | ||
|
|
||
| ```markdown | ||
| **One-way door:** narrows the own-your-data promise in CHARTER Β§2. | ||
| ``` | ||
|
|
||
| Two-way doors need no flag. Match the noise to the cost of undo. | ||
|
|
||
| ## Risk tiering β so this stays usable | ||
|
|
||
| | PR touches | Treatment | | ||
| |---|---| | ||
| | A charter, `README`, `legal/`, `LICENSE`, licensing, pricing, or any public promise | Full block + provenance table + TJ reads the changed text itself | | ||
| | Specs, ADRs, architecture docs | Full block + provenance table | | ||
| | Code, tests, config | Verdict line only | | ||
| | Typos, formatting, dependency bumps | Verdict line only | | ||
|
|
||
| Most PRs are one line. Reserve the ceremony for what can hurt. | ||
|
|
||
| ## Review bots are yours, never his | ||
|
|
||
| The maintainer never reads CodeRabbit, Codex, or any bot output raw. You adjudicate and report one line: | ||
|
|
||
| > CodeRabbit: 7 findings β 5 real and fixed, 2 wrong (it misread the token scope). | ||
|
|
||
| If a finding is real, fix it or explain why not. "The bot said something" is not a report; a verdict on each finding is. | ||
|
|
||
| **A bot's silence is not approval.** Check that a review actually ran on the current head β a passing check can mean skipped, rate-limited, or reviewing a commit you have since replaced. | ||
|
|
||
| **Bots review after a PR opens, so nothing is green at open time.** The sequence is fixed: | ||
|
|
||
| 1. Open the PR at π‘ or π΄. Never π’ β no bot has run yet. | ||
| 2. Wait for gates and bots. Verify a review actually landed on the current head. | ||
| 3. Adjudicate every finding: real or not, fixed or why not. | ||
| 4. If you pushed fixes, **request re-review explicitly** β incremental review is off in these repos, so a fix-up push is otherwise never looked at. | ||
| 5. Only then update the verdict to π’ and add the one-line bot summary. | ||
| 6. Tell TJ it is ready. **A PR he has not been told about is not ready**, whatever its checks say. | ||
|
|
||
| The gate enforces step 5 mechanically: a π’ verdict with no review-bot adjudication line fails. | ||
|
|
||
| ## Before you write the verdict | ||
|
|
||
| 1. Diff the PR against the base. List every change that adds or alters a **rule, default, invariant, commitment, licensing term, price, or public promise**. | ||
| 2. For each, find TJ's authorizing words in the conversation. Quote them with a timestamp. | ||
| 3. Anything without a quote is `INFERRED`. Say so plainly. Do not soften it. | ||
| 4. If there is at least one INFERRED item, the verdict is π‘. Never π’. | ||
|
|
||
| ## What "approval" is and is not | ||
|
|
||
| - "merge it," "looks good," "go ahead," "please merge everything else" β **approval to merge**, not evidence he decided the contents. | ||
| - "I agree with everything above" β covers **the message it replies to**, nothing else. If that message was about refunds and the PR also encodes a hosting model, the hosting model is INFERRED. | ||
| - Approving one PR never authorizes the next. | ||
|
|
||
| Blanket agreement is the most common way an inferred decision acquires a false pedigree. Treat it as narrowly as it was given. | ||
|
|
||
| ## Mechanical gate | ||
|
|
||
| Instructions drift β that is exactly how the audit's findings happened. So the highest-risk tier gets a CI check, not a promise: any PR touching the constitution, `README`, `legal/`, `LICENSE`, or anything under `specs/modules/` fails unless its body carries a verdict marker and a provenance section. See `.github/workflows/pr-verdict.yml` in repos where it is installed. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,184 @@ | ||
| name: PR verdict | ||
|
|
||
| # High-blast-radius changes β the constitution, legal text, licensing and pricing β | ||
| # must carry a verdict and a decision-provenance table in the PR body. | ||
| # | ||
| # Why this is a CI check and not a documented convention: on 2026-07-20 an audit of | ||
| # PRs #87-#100 found nine normative decisions merged that TJ never made, including two | ||
| # amendments to CHARTER.md. Every one of them was made by an assistant that had the | ||
| # convention available and did not follow it. Instructions drift; a required check | ||
| # does not. | ||
| # | ||
| # The rule this enforces lives at ~/.claude/rules/pr-verdict.md. | ||
|
|
||
| on: | ||
| pull_request: | ||
| types: [opened, edited, synchronize, reopened] | ||
|
|
||
| permissions: | ||
| contents: read | ||
| pull-requests: read | ||
|
|
||
| jobs: | ||
| verdict: | ||
| runs-on: ubuntu-latest | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| with: | ||
| fetch-depth: 0 | ||
|
|
||
| - name: Does this PR touch the high-risk tier? | ||
| id: tier | ||
| env: | ||
| BASE_SHA: ${{ github.event.pull_request.base.sha }} | ||
| HEAD_SHA: ${{ github.event.pull_request.head.sha }} | ||
| # via env, not inline β refs may contain $, backtick, or ; | ||
| BASE_REF: ${{ github.event.pull_request.base.ref }} | ||
| run: | | ||
| set -euo pipefail | ||
| git fetch --no-tags --depth=1 origin "$BASE_SHA" 2>/dev/null || true | ||
|
|
||
| # THREE-dot: what this PR changed, not what main changed since it forked. | ||
| # Two-dot compares tips, so a branch merely behind main reports every file | ||
| # main touched β measured at 74 files against this PR's actual 3. That fires | ||
| # the gate on PRs that touched nothing protected, which trains contributors to | ||
| # paste ceremonial provenance tables: the exact habit this gate exists to break. | ||
| # | ||
| # --no-renames: with rename detection on, `git mv CHARTER.md docs/old.md` | ||
| # reports ONLY the new path, so moving a protected file out of its directory | ||
| # silences the gate. Deleting the constitution by relocating it is precisely | ||
| # the change this must catch. | ||
| CHANGED=$(git diff --no-renames --name-only "$BASE_SHA...$HEAD_SHA" \ | ||
| || git diff --no-renames --name-only "origin/$BASE_REF...HEAD") | ||
| echo "Changed files:"; echo "$CHANGED" | ||
|
|
||
| # The tier: constitution, legal text, and the specs that carry licensing, | ||
| # pricing, and the free/paid line. | ||
| # Files that carry a promise, a licensing boundary, or money. | ||
| HIGH_RISK=$(echo "$CHANGED" | grep -E \ | ||
| -e '^CHARTER\.md$' \ | ||
| -e '^README\.md$' \ | ||
| -e '^legal/' \ | ||
| -e '^LICENSE' \ | ||
| -e '^specs/modules/' \ | ||
| -e '^\.github/workflows/pr-verdict\.yml$' \ | ||
| -e '^\.claude/rules/pr-verdict\.md$' \ | ||
| || true) | ||
|
|
||
| if [ -n "$HIGH_RISK" ]; then | ||
| echo "high_risk=true" >> "$GITHUB_OUTPUT" | ||
| { echo 'files<<EOF'; echo "$HIGH_RISK"; echo 'EOF'; } >> "$GITHUB_OUTPUT" | ||
| else | ||
| echo "high_risk=false" >> "$GITHUB_OUTPUT" | ||
| fi | ||
|
|
||
| - name: Require a verdict and decision provenance | ||
| if: steps.tier.outputs.high_risk == 'true' | ||
| env: | ||
| BODY: ${{ github.event.pull_request.body }} | ||
| FILES: ${{ steps.tier.outputs.files }} | ||
| run: | | ||
| set -euo pipefail | ||
| fail() { echo "::error::$1"; FAILED=1; } | ||
| FAILED=0 | ||
|
|
||
| echo "This PR touches the high-risk tier:" | ||
| echo "$FILES" | ||
| echo | ||
|
|
||
| # Normalize the body ONCE, before any check, removing the two things that | ||
| # look like content to a regex and like nothing to a human: | ||
| # - HTML comments: `<!-- π’ SAFE TO MERGE -->` as the first line satisfied | ||
| # the verdict and the bot-line check while rendering invisible. A leading | ||
| # PR-template comment also wrongly failed the verdict check. | ||
| # - Fenced code blocks: the protocol doc's own ```markdown example of a | ||
| # "## Decision provenance" table satisfied the provenance requirement, | ||
| # so quoting the rule counted as following it. | ||
| # Every check below reads BODY_CLEAN, never the raw body. | ||
| # Strip the two things that look like content to a regex and like nothing | ||
| # to a human, BEFORE any check reads the body: | ||
| # - HTML comments: `<!-- π’ SAFE TO MERGE -->` as the first line satisfied | ||
| # the verdict and bot-line checks while rendering invisible. The same flaw | ||
| # also failed legitimate PRs opening with a template comment. | ||
| # - Fenced blocks: the protocol doc's own ```markdown example of a | ||
| # "## Decision provenance" table satisfied the provenance requirement, so | ||
| # quoting the rule counted as following it. | ||
| BODY=$(printf '%s\n' "$BODY" \ | ||
| | perl -0777 -pe 's/<!--.*?-->//gs' \ | ||
| | awk '/^[[:space:]]*(```|~~~)/ { f = !f; next } !f') | ||
|
|
||
| # 1. A verdict marker, one of exactly three, on the FIRST non-empty line. | ||
| # Matching anywhere would let a PR bury the verdict under arbitrary prose β | ||
| # the whole point is that it is the first thing read. | ||
| FIRST_LINE=$(grep -m1 -vE '^[[:space:]]*$' <<<"$BODY" || true) | ||
| if ! grep -qE '(SAFE TO MERGE|NEEDS YOUR DECISION|DO NOT MERGE)' <<<"$FIRST_LINE"; then | ||
| fail "The PR body's first line must be the verdict: π’ SAFE TO MERGE / π‘ NEEDS YOUR DECISION / π΄ DO NOT MERGE. Found: ${FIRST_LINE:0:80}" | ||
| fi | ||
|
|
||
| # 2. A decision-provenance section. | ||
| if ! grep -qiE '^#+[[:space:]]*Decision provenance' <<<"$BODY"; then | ||
| fail "No '## Decision provenance' section. Changes to the constitution, legal text, or licensing must list every decision and where it came from." | ||
| fi | ||
|
|
||
| # 3. Every provenance row is either quoted to TJ or marked INFERRED. | ||
| # A row citing TJ without quote marks is the exact failure this gate exists for. | ||
| # sed form is BRE-portable (BSD and GNU) β `\+` is a GNU extension. | ||
| if grep -qiE '^#{1,}[[:space:]]*Decision provenance' <<<"$BODY"; then | ||
| # Stop at the next heading β reading to end-of-body swallows any later | ||
| # table (a test-results table, for instance) and parses it as provenance rows. | ||
| # Terminate on an ATX heading (# Foo) OR a setext underline (--- / ===), | ||
| # which the earlier awk missed β a "Test results" setext heading let the | ||
| # table under it be parsed as provenance rows. | ||
| ROWS=$(awk '/^#+[[:space:]]*[Dd]ecision provenance/{f=1;next} \ | ||
| f&&/^#+[[:space:]]/{exit} \ | ||
| f&&/^(-{3,}|={3,})[[:space:]]*$/{exit} f' <<<"$BODY" \ | ||
| | grep -E '^\|' | grep -vE '^\|[[:space:]]*[-:[:space:]]+\|' | tail -n +2 || true) | ||
| if [ -z "$ROWS" ]; then | ||
| fail "The 'Decision provenance' section has no rows. If this PR encodes no decisions, say so in a row ('None β no new decisions')." | ||
| else | ||
| while IFS= read -r row; do | ||
| [ -z "$row" ] && continue | ||
| # Only an explicit INFERRED label, or an explicit no-decision row, excuses a | ||
| # missing quote. 'no authorization' alone does NOT β the protocol requires | ||
| # the INFERRED label itself so the disclosure is unmissable. | ||
| if grep -qiE 'INFERRED|none β|no new decision' <<<"$row"; then continue; fi | ||
| # A sourced row must contain an actual quotation. | ||
| if ! grep -qE '["ββ]' <<<"$row"; then | ||
| fail "Provenance row has no quote and is not marked INFERRED: ${row}" | ||
| fi | ||
| done <<<"$ROWS" | ||
| fi | ||
| fi | ||
|
|
||
| # 4. Attribution without a quote, anywhere in the body. | ||
| # "your decision" is deliberately NOT in this list β it matches the | ||
| # "NEEDS YOUR DECISION" verdict marker itself. | ||
| # | ||
| # Backticked code spans are stripped before matching: a doc that | ||
| # *discusses* these patterns (`TJ's decision` as a literal example, in a | ||
| # checklist or an adjudication table) is not attributing anything. Without | ||
| # this, any text explaining the rule trips the rule β which is exactly how | ||
| # this gate first failed, on its own PR body. | ||
| BODY_NO_CODE=$(sed 's/`[^`]*`//g' <<<"$BODY") | ||
| while IFS= read -r line; do | ||
| grep -qE '["ββ]' <<<"$line" && continue | ||
| grep -qiE 'INFERRED' <<<"$line" && continue | ||
| fail "Attributes a decision to TJ without quoting him: ${line}" | ||
| done < <(grep -iE "(TJ's (decision|call|choice)|per TJ's|TJ decided|you decided|you chose|as you decided|decided by TJ|CONFIRMED by TJ|SIGNED OFF by TJ|\(TJ,? [0-9]{4}-|, TJ [0-9]{4}-)" <<<"$BODY_NO_CODE" || true) | ||
|
|
||
| # 5. A green verdict must show the review bots were adjudicated. | ||
| # Bots review AFTER a PR opens, so nothing can honestly be green at open | ||
| # time. This blocks the failure mode where a PR is marked green before | ||
| # CodeRabbit has run, or against a commit that has since been replaced. | ||
| if grep -qE 'SAFE TO MERGE' <<<"$BODY"; then | ||
| if ! grep -qiE '(CodeRabbit|Codex|review bot)s?[[:space:]]*:' <<<"$BODY"; then | ||
| fail "Verdict is SAFE TO MERGE but no review-bot adjudication line is present. Add one line stating what the bots found and how each finding was resolved β e.g. 'CodeRabbit: 7 findings, 5 real and fixed, 2 wrong (misread the token scope).' If a bot has not run on the current head yet, the verdict cannot be green." | ||
| fi | ||
| fi | ||
|
|
||
| if [ "$FAILED" = "1" ]; then | ||
| echo | ||
| echo "See ~/.claude/rules/pr-verdict.md for the protocol." | ||
| exit 1 | ||
| fi | ||
| echo "Verdict block present and every decision is sourced or marked INFERRED." | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
π― Functional Correctness | π Major | β‘ Quick win
Protect the documented catalog and marketplace paths.
The PRβs declared high-risk set names root-level
catalog.mdandmarketplace-v1.md, but this only matchesspecs/modules/.... Changes to the declared files receivehigh_risk=falseand bypass the gate.Proposed fix
π Committable suggestion
π€ Prompt for AI Agents