[CAT-UNVERIFIED tag](1) One shared escape-hatch tag; bare UNVERIFIED: excuses nothing - #415
Merged
mergify[bot] merged 1 commit intoSep 11, 2026
Conversation
Bare `UNVERIFIED:` silenced every check in the message it appeared in. It
cost four characters and bought a whole turn, so it got used to stop
checking rather than to report a check that could not run.
Six hooks read that marker, each with a private regex, and they had
drifted: hedge-runs-prove-it required a named blocker, diu-stop did not.
engine/hooks/_markers now holds one definition of
{{CAT-UNVERIFIED: <claim> -- cannot verify: <reason>}}, the names-a-blocker
test, and the retired bare marker. install.sh links it beside the hooks.
Every suppressor reads it: diu-stop, external-claim-gate (which borrows
diu-stop's matcher at detect.py:13), hedge-runs-prove-it,
history-claim-check, incidence-needs-repetition, named-verb-guard and
prove-it-ship-gate. A well-formed tag excuses its paragraph. A tag that
names no blocker excuses nothing. Bare UNVERIFIED: is prose, and the block
text names the tag that replaced it.
Same three messages through diu-stop, on 509b9cd and on this commit:
legacy marker in front of a real claim
before exit=2 contains an `UNVERIFIED:` claim...
after exit=2 unverified-shaped claim ("because")...
well-formed tag beside the claim
before exit=2 contains an `UNVERIFIED:` claim...
after exit=0 (silent)
tag naming no blocker
before exit=2 contains an `UNVERIFIED:` claim...
after exit=2 unverified-shaped claim ("because")...
Two bugs found by tests while building this, both now pinned:
- A hook directory is itself a symlink, so joining ".." onto it resolves
the link first and lands beside the checkout. Two textual dirnames reach
the sibling module; test_installed_layout builds the symlinked layout.
- Block texts are .format() templates, so a literal {{CAT-UNVERIFIED}}
rendered as {CAT-UNVERIFIED}. Every existing test passed through it,
because they asserted the substring CAT-UNVERIFIED. The tag is now a
format argument, and _markers tests assert on the braces.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VKsvxJk65w6q7KnPSRYvNg
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_841d75b0-61da-4b3c-80fe-622bcfca960c) |
3 tasks
Contributor
|
Queued — the merge queue status continues in this comment ↓. |
Owner
Author
Owner
Author
|
@Mergifyio queue |
Contributor
Merge Queue Status
This pull request spent 5 minutes 9 seconds in the queue, including 3 minutes 44 seconds running CI. Required conditions to merge
|
5 tasks
EdbertChan
added a commit
that referenced
this pull request
Sep 11, 2026
Three detectors each had a hand-rolled backtest.py. scripts/backtest_detector.py now replays any PATH:CALLABLE detector over local Claude Code and OMP transcripts, with --compare against another detector or a git ref. The hooks gain replay functions beside their unchanged decision code, and the three per-detector scripts are deleted. Carved out of #371 onto current main, with two fixes its tests needed there: - materialize_revision resolves symlinks before taking the path relative to the git toplevel. On macOS /var is a link to /private/var, so the relative path climbed out of the archived copy and --compare HEAD loaded the working tree instead of HEAD. - the near-miss test used a bare UNVERIFIED: line, which diu-stop no longer honours (#415); it now uses inline code excused by fenced output. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Gnua5cuAAF1ey1b7zRqJCC Change-Id: I3066797e7031407d5a7074b62a57d7e0e2db25b7
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Every evidence hook now accepts one escape hatch:
{{CAT-UNVERIFIED: <claim> -- cannot verify: <reason>}}. BareUNVERIFIED:excuses nothing.The old marker cost four characters and silenced a whole message. It got used to stop checking, not to report a check that could not run.
Six hooks also kept private copies of the marker regex, and they had drifted apart on whether a blocker must be named.
One shared module now defines the tag. All seven hooks that read a marker use it, and a tag with no named blocker excuses nothing.
Supersedes #383: same paragraph scoping and must-name-a-blocker rule, moved onto the new tag.
Before and After
The same three messages through
engine/hooks/diu-stop/claude_stop_check.py, on509b9cdand on this branch:Review Claim
Approve one shared definition of the escape-hatch tag, read by all seven hooks that honour a marker, with bare
UNVERIFIED:no longer excusing anything.Review Lane
behavior
Review Unit
engine-runtime
Safety Invariant
Every message blocked on
509b9cdis still blocked, except one carrying a tag that names a blocker -- syntax that did not exist before. Messages that relied on bareUNVERIFIED:to pass are now blocked; that is the intended change.Slice Rationale
The seven hooks move together because external-claim-gate borrows diu-stop's matcher (
detect.py:13). Splitting them left external-claim-gate failing 2 of 40 tests. The retry change is the next PR.Non-goals
scripts/check_history_claims.py, the CI twin of history-claim-check, keeps its own copy. It is a script, not a hook, so it cannot reach_markersby the sibling path.Test Plan
Test Plan
python3 engine/skills/make-pr/scripts/preflight.py --base <this PR's base>bash scripts/run_all_tests.sh-- exit 0, 45 suites OK, 0 failed, 1668 testspython3 -m unittest discover -s engine/hooks/_markers/tests-- includestest_installed_layout, which builds the symlinked$HOME/.claude/hooks/layout and imports through it{"last_assistant_message": ...}into the hook from509b9cdand from this branchRevert Plan
Revert Plan
git revert <sha>./install.shso the_markerslink is removed🤖 Generated with Claude Code
https://claude.ai/code/session_01VKsvxJk65w6q7KnPSRYvNg
Note
Medium Risk
Changes enforcement behavior across multiple Stop/PreToolUse hooks: messages that relied on bare
UNVERIFIED:will now block, though coverage is broad via tests and shared marker logic.Overview
Introduces a shared
engine/hooks/_markersmodule so every evidence hook uses one definition of the escape hatch:{{CAT-UNVERIFIED: <claim> -- cannot verify: <reason>}}. Only tags that name a blocker aftercannot verify:excuse the paragraph they sit in; malformed tags and unverified claims still block.Bare
UNVERIFIED:is retired as a free pass—diu-stopand the other hooks now treat it as legacy prose, surface guidance toward the new tag, and no longer silence whole messages. Seven hooks (diu-stop,external-claim-gate,hedge-runs-prove-it,history-claim-check,incidence-needs-repetition,named-verb-guard,prove-it-ship-gate) import the shared module via the symlinked$HOME/.claude/hooks/layout;install.shadds the_markerslink. Unit and layout tests cover tag parsing, installer wiring, and block messages that preserve double braces in.format()templates.Scenario fixtures in
tests/scenarios/prove-it-claims.jsonare updated for well-formed tags, tags without blockers, and legacy markers still firing ship/hedge gates.Reviewed by Cursor Bugbot for commit a25cdb5. Bugbot is set up for automated code reviews on this repo. Configure here.