feat: add review-toolkit plugin (6 reviewer agents + quality-gate and code-review-fanout skills)#71
Conversation
… code-review-fanout skills)
The gh pr list line embedded $(git branch --show-current) command substitution, which the permission engine cannot statically analyze — the whole skill fails to load under restricted permissions (headless -p runs, scoped allowlists). List PRs with headRefName instead and let the model match against the branch line above.
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_54d67de6-752d-4d78-b16a-842d486550b8) |
… code-review-fanout skills)
The gh pr list line embedded $(git branch --show-current) command substitution, which the permission engine cannot statically analyze — the whole skill fails to load under restricted permissions (headless -p runs, scoped allowlists). List PRs with headRefName instead and let the model match against the branch line above.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4c5f7b61dd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_f918e4cb-6b6b-457a-8cb9-bd310bdf1236) |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_ad235509-60e8-4b42-9d6a-41867011d8ca) |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 34212e05a5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
On a committed-clean branch git diff --shortstat HEAD is empty, so the default mode under-classified large PRs and skipped tier-gated reviewers. Use the same origin/HEAD -> origin/main -> HEAD fallback chain the review diff base documents.
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_a2f5b458-d258-47b7-9a1b-44876be5c23f) |
|
Claude finished @kyle-sexton's task in 4m 24s —— View job PR Review: feat/publish-review-toolkit
Overall AssessmentThis is a well-architected, repo-agnostic plugin with thoughtful design decisions throughout. The CLAUDE.md rules are followed carefully. Three correctness bugs in the instruction documents warrant attention before merge; the Codex merge-base-shortstat P2 was already fixed in commit Confirmed Bugs1. Clean-tree gate silently drops committed-clean branches that have no PR
Case 1 fires when Minimal fix: Add 1. **Truly clean** — `git status --porcelain` empty AND no open PR AND NOT ahead of its base →
report "no changes to review", spawn nothing, write no findings file.Note: the pre-computed shortstat ( 2. Fix mode picks the wrong file when quality-gate runs after fanout The fix action takes "the newest The fanout file always has ## Step 1: Locate the findings file (current branch ONLY)
... take the newest `*.md` whose frontmatter contains `tier:` (fanout format)
rather than the newest `*.md` overall. Quality-gate artifacts have `mode:` in
frontmatter instead and must not be consumed by the fix action.3. Partial extraction silently drops findings
rawFallback: (records.length === 0 && returned.length > 0) ? returned : null,
The fix is to always return the raw outputs alongside the records — not conditionally on all-zero — and let the main thread compare return {
records,
rawLeafOutputs: returned, // always; main thread uses when records < returned count
nulls,
ran: roster.map(l => l.label),
}Resolved (Codex P2 — already fixed)The Codex finding about Suggestions4. TIER2_AGENTS and TIER2_SLICES dispatched sequentially, not concurrently
5. Single-quote injection risk in OWNERLESS_SLICES substitution When the Workflow script is constructed at dispatch, slice paths are injected as CLAUDE.md Compliance
The SummaryThree correctness bugs affect the fanout skill's instruction docs (committed-clean branch gate, fix-mode file disambiguation, partial-extraction drop). All three are localized text fixes in markdown instruction files with no manifest or agent changes required. The architecture, agent definitions, manifest, and marketplace entry are well-formed and CLAUDE.md-compliant. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c838b26516
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- default-mode: clean-tree short-circuit now checks branch-ahead state so a committed-clean pre-PR branch is classified reviewable - fix-pass-mode: findings-file lookup filters on the type: review-findings frontmatter marker instead of newest-any-markdown, since quality-gate shares the directory with different shapes - run-everything-mode: Workflow script always returns raw leaf outputs so partial extraction preserves unparsed surfaces per the normalization contract, not only the all-zero case
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_194dccfa-52ca-47b7-ab22-fea8b95489ae) |
|
Claude finished @kyle-sexton's task in 2m 45s —— View job PR Review: feat/publish-review-toolkit
Previous Findings — All Three Confirmed Fixed ✓Commit Bug #1 (clean-tree gate) — Bug #2 (fix-mode file disambiguation) — Bug #3 (partial extraction drop) — New Open Findings (from Codex scan on c838b26)These were flagged after the previous fix round and are not yet addressed: P2-A — Dirty-tree diff omitted from tier sizing The pre-computed shortstat uses the three-dot form The fix is to include the dirty portion: augment the pre-computed line with a P2-B — PR base not resolved; non-default target branches get wrong diff surface The review diff base chains through The pre-computed # current
gh pr list --json number,title,headRefName --limit 10
# proposed
gh pr list --json number,title,headRefName,baseRefName --limit 10The shared diff base in SummaryThe three correctness bugs from the prior review are cleanly fixed in |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a42250740d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- Split the pre-computed diff size into committed-vs-merge-base and uncommitted shortstats; three-dot diff compares commits only, so a large uncommitted change was tiered small. Tier size is now the sum. - Pre-compute baseRefName and prefer the open PR's real base over origin/HEAD/origin/main in both skills' review diff base, so PRs targeting develop or release branches review the intended surface.
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_5ec31f31-2152-46a0-ad48-63acb2b95508) |
|
Claude finished @kyle-sexton's task in 3m 30s —— View job PR Review: feat/publish-review-toolkit
d674f2c: P2-A and P2-B Confirmed Fixed ✅P2-A — Dirty-tree diff now included in tier sizing
P2-B — PR base resolved before diffing
Remaining Open Findings (from Codex P2 scan of a422507)Three P2 items surface-flagged after the prior fix round are not yet addressed in d674f2c: C — Branch slug collision lets fix-mode apply wrong branch's findings Step 1 filters by Minimal fix: add one discriminator to Step 1 alongside the existing type check: take the newest `*.md` whose frontmatter declares BOTH `type: review-findings`
AND `branch: <current-branch>` (exact match, not slug). Slug ambiguity means
the directory alone is not a safe fence; the branch field is.D — All six agents have Under Claude Code's memory system, Fix options (pick one):
E — Orchestrator The skill promises at SKILL.md:22–23: "The The orchestrator being optional ( Fix: add one gate in the orchestrator description so the side effect is explicit: - **`code-review`** — `/code-review:code-review`: parallel reviewers + confidence scorer
for an existing PR. **Note: this orchestrator posts a PR comment when an open PR
exists — this is a visible remote side-effect outside the findings file; skip it
when running in a findings-only context or add `--no-comment` if the plugin
supports it.**Minor Suggestion (Carryover)
Cumulative Status
Five of eight findings are confirmed fixed. P2-C, P2-D, and P2-E are localized to |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_cf736022-f94e-41a9-9bb3-315efdce057b) |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4b4812dc5a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ks, PR-list cap - restatement artifact declares type: restatement-review so the fix action's type filter cannot pick it up as fanout findings - line-less findings dedup on path + category + content gist, not path alone, so distinct file-scoped findings survive Stage 3 - all four diff-driven agents resolve the open PR's baseRefName before the origin/HEAD fallback, matching the skills' review diff base - ci-log-auditor lists all step conclusions and hunts continue-on-error masks via workflow YAML + step logs; the REST conclusion can read success for a failed continue-on-error step - both skills fall back to gh pr list --head when the capped pre-computed PR list omits the current branch
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_55dbad5f-d866-42e0-a2b3-7b14bf653fbd) |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3033b51d05
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…oolkit # Conflicts: # .claude-plugin/marketplace.json # README.md
- Leaf diff target uses the merge base unconditionally — git diff HEAD on a dirty ahead-of-base branch drops the committed changes - ci-log-auditor paginates the jobs endpoint (per_page defaults to 30, so page-2+ matrix jobs were invisible to the audit)
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_b7e31aa4-2b56-4b8d-9e14-6f26673d43a1) |
|
Claude finished @kyle-sexton's task in 3m 37s —— View job PR Review: feat/publish-review-toolkit
All Previous Open Findings — Confirmed Fixed ✅Commits Finding F — Restatement artifact passes fix-mode filter — Finding G — Path-only dedup drops distinct null-line findings — Finding H — Direct reviewer agents use wrong diff base — ✅ Fixed (
Codex — CI masked failures — Codex — gh pr list cap — Codex — Dirty branch leaf diff — Codex — CI pagination — Remaining Non-Blocking Suggestion
Cumulative Status
All 15 tracked correctness findings are confirmed fixed. The plugin is well-formed, CLAUDE.md-compliant, repo-agnostic, and ready to merge. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8fa4465954
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…tcher The self and per-slice prompt templates hardcoded the origin/HEAD merge-base, bypassing the PR-base resolution in Shared inputs; the dispatcher now substitutes the resolved base into the prompt.
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_5d3c068a-c859-4d4a-99d2-59094db468d5) |
|
Claude finished @kyle-sexton's task in 1m 41s —— View job PR Review: feat/publish-review-toolkit
0db7bf8 — Final Codex P2 Confirmed Fixed ✅Codex P2 — quality-gate worker prompts hardcode
Both worker prompt templates previously embedded The design correctly delegates resolution to the main-thread orchestrator and documents the substitution requirement clearly in the template itself. No further changes needed. Cumulative Status — All Findings Resolved
All 16 tracked correctness findings are confirmed fixed. The plugin is well-formed, CLAUDE.md-compliant, repo-agnostic, and ready to merge. CLAUDE.md Compliance
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0db7bf80e6
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
- Agents fetch origin/<baseRefName> before computing the merge base — shallow and single-branch clones lack the ref, and the silent origin/main fallback reviewed the wrong diff - Findings-file contract requires escaping literal | and flattening newlines in Finding/Action cells so the fix action's table parse cannot split rows on reviewer text
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_d4aee8c9-8ef9-489a-90f9-464712ed0496) |
…oolkit # Conflicts: # .claude-plugin/marketplace.json # README.md
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_786fa80a-15b6-40e2-9961-b1a71fd708ef) |
|
Claude finished @kyle-sexton's task in 1m 57s —— View job PR Review: feat/publish-review-toolkit
5b357d7 — Final Two Codex P2 Findings Confirmed Fixed ✅P2-I — Fetch resolved PR base before merge-base in shallow clones
PR_BASE="$(gh pr list --head "$(git branch --show-current)" --json baseRefName -q '.[0].baseRefName' 2>/dev/null)"
[ -n "$PR_BASE" ] && git fetch origin "$PR_BASE" 2>/dev/null # shallow/single-branch clones may lack the base ref
git diff "$(git merge-base "origin/${PR_BASE:-HEAD}" HEAD 2>/dev/null || git merge-base origin/main HEAD 2>/dev/null || echo HEAD)"The same fetch-before-merge-base pattern is consistently applied across all four diff-driven agents: P2-II — Escape
This closes the silent-drop path identified by Codex: a TypeScript union type or shell pipeline in a finding body would have split the Markdown row into phantom columns, causing the fix-pass parser to misread or skip the affected row. Cumulative Status — All Findings Resolved
All 18 tracked correctness findings are confirmed fixed. The only open item is the non-blocking performance suggestion to merge the sequential CLAUDE.md Compliance
The plugin is well-formed, CLAUDE.md-compliant, repo-agnostic, and ready to merge. ✅ |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1a57262d43
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
|
||
| ## Primary path — `pr-review-toolkit` orchestrator plugin (when installed) | ||
|
|
||
| When the `pr-review-toolkit` plugin (from the `claude-plugins-official` marketplace) is available, invoke `/pr-review-toolkit:review-pr` with aspects detected from the changed files: |
There was a problem hiding this comment.
Gate PR-only review toolkit on an actual PR
In the pre-PR local-review case this mode advertises, having pr-review-toolkit installed sends the review to /pr-review-toolkit:review-pr and never reaches the local code-reviewer fallback, even when the current branch has no PR. That command is PR-oriented, so a dirty or committed branch before PR creation can fail/no-op instead of receiving code review; check for an open PR before using it, otherwise dispatch the local reviewer.
Useful? React with 👍 / 👎.
|
|
||
| ## Primary path — `code-review` orchestrator plugin (when installed) | ||
|
|
||
| When the `code-review` plugin (from the `claude-plugins-official` marketplace) is available, invoke `/code-review:code-review`. It detects the current branch's PR, runs parallel review agents with confidence scoring, and posts findings as a PR comment. |
There was a problem hiding this comment.
Pass explicit PR targets to the PR reviewer
When the user asks to review someone else's PR, or invokes something like pr 123, this path drops that target and calls /code-review:code-review in current-branch detection mode; the fallback gh pr diff is also unqualified. From any branch other than the PR branch this reviews the wrong PR or fails, so preserve a PR number/URL argument and pass it through to the orchestrator and gh command.
Useful? React with 👍 / 👎.
…ss lint (#203) ## Summary Closes decisions #38, #71, #72 (Decisions Log: https://claude.ai/code/artifact/232ecdce-8316-4880-8c0a-dc3c7dcf3a63). - **#38** `formatting-create-md-attribution-footer-drift`: `create.md`'s PR-body heredoc now emits the `🤖 Generated with Claude Code` attribution footer it documents elsewhere but previously didn't actually emit. - **#71** `naming-work-items-checklist-template-staleness`: `templates/checklist.md`'s `add` action reconciled to `add.md`'s actual current behavior — `--force` skip, `--type` native-Issue-Type resolution (org repos) vs. `type:` label (personal/non-org repos), `--agent-ready` body template, `--recurring` title prefix — none of which the checklist previously mentioned. - **#72** `tooling-gov-commit-skill-composition-lint`: new `guardrails` plugin hook `flag-commit-pr-skill-bypass.sh` — advisory `PreToolUse`/`Bash` guard that flags direct `git commit` (missing the canonical `-F -` + `--trailer` shape) or any `gh pr create`, nudging toward this marketplace's own `/commit` / `/pull-request create` skills. Gated on the consuming project actually having `source-control` enabled in its own `.claude/settings.json` (fails quiet on uncertain state, per the plugin's existing advisory-guard posture). Mirrors `block-hook-bypass.sh`'s literal-stripping detection and the shared telemetry envelope. ## Verification - New hook's own test suite: 19/19 passed. - `shellcheck` on both the hook and its test — clean. - `guardrails` version bumped 0.3.2 → 0.4.0 (new toggleable guard), README's guard table/kill-switch table/consumer-seams section updated to match. --------- Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>



Publishes the review-toolkit plugin from the medley migration EPIC (melodic-software/medley#1282).
What's included
/review-toolkit:quality-gate(single-lens checkpoint, 8 modes) and/review-toolkit:code-review-fanout(multi-surface fan-out with severity-ranked normalized findings + fix pass)Gate evidence
claude plugin validate --strict— PASS (plugin + marketplace manifest, post-rebase)claude plugin details): ~931 tok always-on; per-component on-invoke ~860–2.3k (largest: security-reviewer ~2.3k, code-review-fanout ~2.2k)claude --plugin-dir ./plugins/review-toolkit -p: invokedreview-toolkit:quality-gate criteria; skill resolved under the/review-toolkit:namespace and loaded successfullyFix found during smoke test
The skills' pre-computed context embedded
$(git branch --show-current)command substitution in thegh pr listline — the permission engine cannot statically analyze it, so the entire skill failed to load under restricted permissions (headless-p, scoped allowlists). Replaced with a substitution-freegh pr list --json number,title,headRefNamethat the model correlates against the branch line above.🤖 Generated with Claude Code
Note
Low Risk
Adds documentation and agent/skill definitions only; consumer-repo impact is optional review artifacts and explicit fix-pass edits, with no changes to existing plugins' runtime behavior.
Overview
Adds the
review-toolkitmarketplace plugin: six read-only reviewer agents (code, security, architecture, doc drift, ecosystem build/test/lint, CI log audit) withmemory: local, plus sharedcontext/severity.mdwhen the consumer repo defines no review vocabulary./review-toolkit:quality-gateis a single-lens checkpoint (eight modes including fresh-context self, delegated code/architecture/security, PR, criteria, per-slice, and markdown restatement)./review-toolkit:code-review-fanoutscales reviewer breadth by diff size (or run-everything), runs an optional Workflow fan-out with main-thread fallback, normalizes heterogeneous outputs through a five-stage pipeline, persists a stabletype: review-findingsreport under.claude/review/<branch-slug>/(or project override), and exposesfixas the only mutating path (branch-scoped findings, correctness vs cleanup routing).Catalog updates:
.claude-plugin/marketplace.jsonentry and README catalog row. Skill pre-computed blocks use a staticgh pr list --json …line (no nested command substitution) so skills load under restricted permissions.Reviewed by Cursor Bugbot for commit 1a57262. Bugbot is set up for automated code reviews on this repo. Configure here.