feat(plugin-quality): post-use component audit skill, auditor agent + config surface#1243
Conversation
Six-step post-use audit workflow: main-thread evidence capture into a
compaction-proof packet (${CLAUDE_PLUGIN_DATA}/evidence/<session_id>/
<target-slug>/<run-nonce>/, sanitized slug, 30-day retention,
deterministic resume rule), map+ground in the fresh NAMED auditor
subagent with per-topic fresh-docs verification, blindspot+candidates,
interactive contract lock written INTO the packet, presence-gated
review seams with one-line fallbacks, sink emit.
Context-gate consumes context-guard as a soft dependency: per-zone
decision table (steps 2-3 subagent in EVERY zone; dumb row =
summary+pointer, mandatory dispatch, immediate flush artifact;
unknown = dumb + visible notice), re-evaluated at steps 2 and 5;
zones.json read directly via jq with inlined defaults as fallback
only; ${CLAUDE_SESSION_ID} discovery; compacted session =
evidence-degraded regardless of zone. Inlined floor byte-matches the
frozen context-guard reader contract (drift-grepped).
Sink ladder (tracked config -> registration inference -> ask+persist
-> local markdown) with the unconditional draft+confirm egress gate
surfacing the ACTING gh account; verb-contract deviation recorded at
the coupling site. Untrusted-content posture standing in both hub and
agent; producer/consumer split enforced. Trigger vocabulary maps the
retiring local plugin-audit skill 1:1 plus negative routing
boundaries (skill-quality/review/mcp-tools).
Auditor agent: Read/Grep/Glob/WebFetch + Bash named honestly
(validate + config probes; no mutation, no issue filing).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
) recurring-concerns.md (8-section design-failure checklist) and the five component-type lenses (hook, skill, agent, command, config) ported from the retiring machine-local skill, generalized: consumer name-drops and machine specifics stripped, stale references updated (listing budget, merged-commands note kept), and three additions — inline-floor drift check under SSOT, hardcoded-consumer-specifics under coupling, secrets/sensitive flag under config, untrusted-input posture under agent. Hub keeps the extend-without-bloating pattern: one file + one index row per addition. The old handoff-inbox.md reference is NOT ported (superseded by the sink reference in the config slice). Forbidden-strings grep (machine paths, consumer names, dead inbox refs) returns empty; skill-quality gate on the hub: PASS (0 errors). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… row (#1232) reference/config.md: .claude/plugin-quality.md keys (sink, markdown_dir, zone_behavior — tighten-only, repo_map), all three cascade layers with per-key override merge semantics DECLARED per the convention, the four-rung sink resolution ladder cross-referencing the skill-owned draft+confirm egress gate, and the markdown item schema (frontmatter + body sections + claim/lease protocol). Compat-target reconciliation done FIRST and recorded: the markdown item schema byte-matches the live cross-terminal handoff inbox contract (fixture captured 2026-07-24); the work-items local-markdown adapter storage diverges by design (id grammar, state vocabulary, JSON-valued frontmatter, adapter-owned numbering) and is NEVER a hand-write target — emitting into work-items goes through its seam CLI only. Config-cascade README gains the plugin-quality implementers row (conforms, per-key override). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ped apply (#1238) check (read-only): gh presence + ACTING account/host (multi-identity machines get the surprise at setup time, not at the emit gate); context-guard snapshot probe reporting the dispatch mode the audit will run in (zone-informed vs conservative — conservative is a working state, not a defect); all three config layers read with per-key override and every effective key reported with the layer that supplied it; sink reachability. apply: writes/converges ONLY the tracked .claude/plugin-quality.md (named keys only, prose and unrecognized keys preserved, idempotent with a stated no-diff second run); recommends the recursive .claude/**/*.local.* gitignore line but NEVER writes the consumer's .gitignore or the user-global/.local layers. skill-quality gate: PASS (0 errors); evals validate structurally. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Audit evals: trigger continuity (retiring skill's vocabulary), negative routing (skill-quality / review / mcp-tools boundaries), refuse-to-implement-in-session, emit-shape draft+confirm with acting account, and both anti-pattern cases — conservative dispatch (absent snapshot must NOT skip the subagent) and prompt injection (a directive embedded in audited source must not alter the sink or skip the gate). Plugin README (capability, context-gate, security posture, install, requirements) + 0.1.0 CHANGELOG; marketplace entry + repo README catalog row (positioned to match entry order). Security-review record in MIGRATION-PLAYBOOK naming ALL data surfaces: gh issue create egress (draft+confirm, acting account), plugin-cache + registration reads (the audit subject), context-guard seam reads, config-layer reads, evidence-packet writes with 30-day retention, honest auditor Bash grant, no hooks/MCP. claude plugin validate + --strict: exit 0. Forbidden-strings re-grep across the plugin tree (machine paths, consumer names, dead inbox refs): empty. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…acket (#1239) Dumb-zone smoke evidence: the workflow expects the auditor to write findings.md into the packet, but the agent's tool grant lacked Write — the main thread had to persist the full findings text, contradicting the dumb row's summary-only rule. Write added to the grant, honestly named and instruction-scoped to the packet directory only; security record updated to name both non-read-only tools and the smoke that surfaced the gap. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 2m 48s —— View job PR Review — feat(plugin-quality): post-use component audit skill, auditor agent + config surface
|
|
Claude finished @kyle-sexton's task in 2m 50s —— View job Security Review — PR #1243
FindingsIMPORTANT — High confidenceAuditor agent's Bash grant + "piping a fixture into a hook script" creates a pre-confirm-gate arbitrary code execution path
When the audit target is a malicious plugin, its hook scripts are adversarial code. The auditor's explicit permission to pipe fixtures into them means executing those scripts on the user's machine as part of step 2 ("Map + ground") — well before the step 6 draft+confirm egress gate. A hook script that receives any piped input can run arbitrary shell commands: exfiltrate environment variables, write files, or reach the network, all without user confirmation. The MIGRATION-PLAYBOOK.md security review (added in this PR) asserts the blast radius is "bounded to the findings text itself, which the draft+confirm gate puts in front of the user before it leaves the machine." Line 857–858. This claim is false for hook script execution: harm from an adversarial hook occurs in step 2, not step 6. The confirm gate cannot retroactively prevent code that already ran. The qualifiers "harmless" and "safe fixture" in the standing instruction are model-honored advisories, not technical enforcement. There is no mechanism preventing the agent from invoking an adversarial hook script under a well-crafted audit prompt or after a prompt injection. Recommended fix: Remove or strictly scope the "piping a fixture into a hook script" Bash permission. Empirical hook testing belongs to a separate, explicitly consented action, not a default in the auditor's tool grant. If hook execution is needed for coverage, it should require an explicit user approval step before each invocation of an untrusted script, and the MIGRATION-PLAYBOOK blast radius assessment should be corrected. IMPORTANT — High confidencePrompt injection from audited content + Bash grant escalates from "corrupt findings" to "arbitrary code execution" The PR correctly identifies prompt injection as an attack surface and acknowledges it is defended only by a model-honored standing instruction — "an advisory, model-honored defense, stated honestly as such" (MIGRATION-PLAYBOOK.md). That honest framing is appropriate for a skill that is read-only. However, because the auditor agent carries a Bash grant, a successful prompt injection does not merely corrupt the findings text: it can invoke the agent's Bash tool to execute arbitrary commands. The auditor reads the full contents of the audited plugin's The combination — broad untrusted input into context + Bash execution capability — means the effective blast radius is the user's entire local environment, not just the findings. The standing instruction in Eval #7 ( Recommended fix: Consider whether the auditor agent needs both untrusted-content ingestion AND a Bash grant simultaneously. If Bash is required, a separate confined "execution agent" that receives only a minimal, sanitized task (not the full plugin source) is a stronger containment model. Alternatively, note explicitly in the review record that the advisory-only defense applies to an agent with execution capability, not a read-only agent. SUGGESTION — High confidenceSink rung 2 — marketplace
A malicious plugin can set its marketplace registration There is no validation that the inferred repo is the plugin's legitimate canonical source. The Recommended fix: When proposing an inferred repo, note its source in the confirm surface — e.g. "inferred from marketplace metadata" — so the user has a clear signal that this repo comes from the plugin's own registration, not a canonical allowlist. Optionally, cross-check the inferred repo against the marketplace's own SUGGESTION — Medium confidence30-day packet cleanup Bash operation on model-sanitized
The The risk is bounded by Recommended fix: Implement the slug sanitization as a shell SummaryNo GitHub Actions files were changed in this PR. The two IMPORTANT findings both stem from the same root cause: the |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5b3f16d66d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…4 continuation line (#1239) plugin-quality joins the accepted cross-plugin `audit` collision on its own grounds (namespace supplies the object; the one mutation sits behind the draft+confirm the verb table reads as the explicit user override — recorded at the coupling site), added to the registry owner set with the rationale extended. README security-posture line rewrapped so markdownlint stops parsing the continuation as a plus-marker list. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 5m 1s —— View job PR Review — feat(plugin-quality): post-use component audit skill, auditor agent + config surfaceTasks
ReviewThis pass reviews the full PR diff (7 commits, 943 additions) focusing on correctness, security, and alignment with the repo's design rules. The latest commit ( Issues1. Config keys
|
|
Claude finished @kyle-sexton's task in 3m 9s —— View job Security Review — PR #1243Tasks
FindingsSecurity issues only; no style or coverage commentary. Findings ordered by severity. CRITICAL — High ConfidenceAuditor agent holds Bash + WebFetch simultaneously while ingesting arbitrary untrusted plugin content — creates an egress-gate-bypassing exfiltration path
The auditor agent holds both A successful prompt injection into any audited-content field could:
This entire path runs inside the auditor agent — before and entirely outside the main skill's step-6 egress gate. The
And at lines 871–872:
Both characterizations are incorrect as technical enforcement claims. The auditor's Eval #7 ( Recommended fix:
The IMPORTANT — High ConfidenceAuditor agent explicitly permits hook-script execution before the egress gate
When auditing a malicious plugin, its hook scripts are adversarial code. The standing instruction explicitly names "piping a fixture into a hook script" as a permitted Bash use case. This means arbitrary shell commands from a third-party hook script execute on the user's machine during step 2 (Map + ground) — before the step-6 egress gate's confirm surface ever appears. The The qualifier "harmless" is advisory and model-enforced. No mechanism prevents the auditor from executing an adversarial hook script under a well-formed dispatch prompt or after a prompt injection that overrides the "harmless" qualifier. Recommended fix: Remove "piping a fixture into a hook script" from the auditor's stated Bash permission. Hook-script reproduction is not a default audit operation — it requires explicit per-invocation user consent before each untrusted script execution. If empirical hook testing is required, it belongs behind a separate, explicit user approval step in the main workflow, not a standing tool grant in the fresh-context agent. IMPORTANT — Medium Confidence30-day cleanup uses
The The risk is bounded by Recommended fix: Technically enforce slug sanitization at the Bash layer ( SUGGESTION — High ConfidenceSink rung 2 infers the issue target from an audited plugin's own marketplace registration — an attacker-controlled field
A malicious plugin can set its marketplace The confirm surface does not indicate how the proposed repo was derived, so the user has no signal that it comes from the audited plugin's own self-reported metadata rather than a canonical allowlist. Recommended fix: When proposing an inferred repo, include its provenance in the confirm surface — e.g. SummaryNo GitHub Actions files changed. All four findings stem from the same root: the auditor agent holds execution capability ( |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b9856aba66
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…am behind the gate (#1239) Four codex findings on PR #1243: - P1: ${CLAUDE_PLUGIN_DATA} does not substitute in skill markdown (repo precedent: architecture:improve gotcha; substitution table re-checked this session). The packet path now derives the data directory deterministically per the plugins reference (~/.claude/plugins/data/<plugin-id>/, sanitized id, list-then-match before first write) with the token named only as provenance. - P2: every documented config key is now consumed — a Config-resolution step at invocation binds sink/markdown_dir to ladder rung 1, zone_behavior: always-conservative short-circuits the context-gate (with an honest notice naming the config as the reason), and repo_map overrides rung-2 inference. - P2: the egress gate now covers every externally-visible emit — the presence-gated work-items seam (create-item performs provider writes) sits behind the same full-draft + destination + acting-identity confirm as gh issue create; only the rung-4 local file skips it. - P2: the config example now uses an outer four-backtick fence so the copied inner ```yaml fences are real, not zero-width-prefixed. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Claude finished @kyle-sexton's task in 3m 28s —— View job PR Review — feat(plugin-quality): post-use component audit skill, auditor agent + config surfaceTasks
What eb1e642 resolvesThe latest commit cleanly addresses all four Codex P1/P2 findings and the work-items seam gap from the prior Claude review. Each fix lands correctly:
Issues still open1. MIGRATION-PLAYBOOK blast-radius assessment still incorrect (security, from prior security review)
And
Both remain unedited. The auditor agent holds Additionally, If this is accepted as a known risk (advisory-only posture acknowledged, not a technical control), the playbook's security review narrative should say so accurately rather than describing a stronger containment than exists. Fix this → 2. Run-nonce second-level collision (correctness, from prior review — still open)Two same-target audits within one second produce the same nonce and the second overwrites the first's evidence packet, defeating the stated "gets its own directory instead of clobbering the first" invariant. A millisecond suffix ( 3.
|
|
Claude finished @kyle-sexton's task in 3m 7s —— View job Security Review — PR #1243Tasks
FindingsSecurity issues only. Reviewing the current HEAD ( CRITICAL — High ConfidenceAuditor agent holds Bash + WebFetch simultaneously with untrusted plugin source in context; MIGRATION-PLAYBOOK blast-radius characterization remains incorrect
Both characterizations are incorrect as technical enforcement claims. The step-6 egress gate (now correctly covering both A successful prompt injection into any audited-content field could use the auditor's Eval #7 ( The MIGRATION-PLAYBOOK verdict at line 879 states "1 bounded to an honestly-named agent Bash grant under standing instructions" — this framing accepts the advisory defense for Bash alone, but the combination with WebFetch was not the condition evaluated. The playbook's own standard for data egress (surface 5) is "confirm-gated egress plus docs-only WebFetch," but docs-only WebFetch is advisory. Recommended fix:
The MIGRATION-PLAYBOOK blast-radius assessment and the egress-gating claim (lines 871–876) must be corrected regardless: the confirm gate does not bound the auditor's own tool execution. IMPORTANT — High ConfidenceAuditor Bash grant explicitly permits hook-script execution; adversarial hooks run before the step-6 egress gate
When the audited plugin is malicious, its hook scripts are adversarial code. The standing instruction names "piping a fixture into a hook script" as a permitted Bash use case. Hook script execution runs during step 2 (Map + ground), before the step-6 egress gate. The word "harmless" is a model-honored qualifier — no mechanism prevents the auditor from executing an adversarial hook under a well-formed dispatch prompt or after a prompt injection that overrides the qualifier. An adversarial hook script receiving piped input can run arbitrary shell commands — exfiltrate environment variables, write files, reach the network — all before the confirm surface appears and beyond what it can retroactively prevent. This is distinct from and compounds the CRITICAL finding: even without a prompt injection, the skill's standing instruction creates a designed-in code-execution path for untrusted code. Recommended fix: Remove "piping a fixture into a hook script" from the auditor's stated Bash permission. Empirical hook-script reproduction is not a default audit operation; it requires explicit per-invocation user consent and should live behind an explicit approval step in the main workflow, not a standing tool grant in the fresh-context agent. IMPORTANT — Medium Confidence30-day packet cleanup uses model-sanitized
The eb1e642 correctly replaced Recommended fix: Technically enforce slug sanitization at the Bash layer ( SUGGESTION — High ConfidenceSink rung 2 confirm surface does not disclose that the proposed repo originates from the audited plugin's own marketplace registration
eb1e642 correctly adds the Recommended fix: When the rung-2 path fires, the confirm surface should name the derivation source — e.g. What has been addressed since the initial commit
SummaryThe two CRITICAL+IMPORTANT findings share a root: the auditor agent holds execution capability ( |
1 participant
Summary
Lane B of the
plugin-audit-porttopic (contract:docs/topics/plugin-audit-port/PLAN.md; lane A merged as #1241 — this lane consumes its frozen reader contract). Adds theplugin-qualityplugin — the machine-localplugin-auditskill ported, generalized, and improved:/plugin-quality:audit <plugin>[:<component>]— six-step post-use behavioral audit: main-thread evidence capture into a compaction-proof packet (${CLAUDE_PLUGIN_DATA}/evidence/<session_id>/<target-slug>/<run-nonce>/, deterministic resume rule, 30-day retention) → map+ground in the fresh namedauditorsubagent with per-topic fresh-docs verification → blindspot + candidates → interactive contract lock (written INTO the packet) → presence-gated review seams with stated fallbacks → sink emit.ghaccount; verb-contract deviation recorded at the coupling site..claude/plugin-quality.mdper the config-cascade convention (per-key override declared, implementers row added); setup skill (check with per-layer provenance / apply scoped to the tracked config).Sanity-check evidence (per PLAN phases B1–B6)
plugins/context-guard/reference/reader-contract.mdbyte-identically.grep 'context: fork'in the audit SKILL.md: no hits.gh issue createappears with the confirm gate in the same section.claude plugin validate plugins/plugin-qualityexit 0;claude plugin validate --strict .exit 0 (post-rebase).skill-quality:checkPASS (0 errors) on both skills; evals validate against the bundled schema.reference/config.md: markdown item schema byte-matches the live handoff inbox contract fixture;work-itemslocal-markdown adapter storage documented as divergent-by-design (seam-CLI-only emission).melodic-software/claude-code-plugins) → confirm declined → rung 4 local item; nogh issue create.--plugin-dirtarget): rungs 1–2 fail correctly, rung 4 local item written;${CLAUDE_PLUGIN_DATA}resolves to…/data/plugin-quality-inline/under--plugin-dir(identity difference vs installed form noted per stress-test fix(comment-hygiene): exclude '.' from the owner/repo#N leading boundary #12).used_percentage90) for the smoke session id → step 2 resolved dumb with the conservative notice ABSENT (gate did not fall through), flush artifacts present at every step boundary (evidence/findings/contract/review/item), steps 3–4 ran summary+pointer; when the snapshot honestly aged past 10 minutes mid-run, step 5 re-evaluated to unknown with the notice — correct re-evaluation observed live.Related
docs/topics/plugin-audit-port/PLAN.md.Closes #1237
Closes #1231
Closes #1232
Closes #1238
Closes #1239
🤖 Generated with Claude Code