Cut the audit-pr runtime, stop audit-docs breaking tables, and make the audits portable - #594
Merged
Merged
Conversation
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Contributor
There was a problem hiding this comment.
Pull request overview
This PR updates the repository’s audit prompts and skill bundles to reduce audit runtime, tighten evidence requirements (including credential redaction), and make the audits more portable across different agent hosts. It also includes small dependency bumps reflected in the lockfile.
Changes:
- Refines
audit-pr/audit-qualityevidence and refutation guidance, adds an explicit execution budget, and makes examples non-runnable shapes. - Adds safer Markdown table-editing rules to
audit-docs(prompt + skill + voice reference) to avoid breaking table structure. - Bumps
webpackandsass, updatingpackage-lock.jsonaccordingly.
Reviewed changes
Copilot reviewed 13 out of 14 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| package.json | Bumps webpack and sass versions. |
| package-lock.json | Updates lockfile for the dependency bumps (including transitive changes). |
| .github/prompts/readme.md | Documents how prompt vs skill halves resolve # references, and clarifies portability expectations. |
| .github/prompts/audit-quality.prompt.md | Updates context-resolution table wording to describe capabilities rather than product-specific tool names. |
| .github/prompts/audit-pr.prompt.md | Tightens evidence, redaction, execution budget, and refutation-pass guidance in the prompt half. |
| .github/prompts/audit-docs.prompt.md | Clarifies #codebase/#activePullRequest/#changes meaning inline, adds explicit table-editing rules. |
| .claude/skills/audit-quality/SKILL.md | Updates portability language, evidence/redaction rules, and adds an execution budget. |
| .claude/skills/audit-pr/SKILL.md | Updates description, evidence requirements, execution budget guidance, and refutation expectations. |
| .claude/skills/audit-pr/references/supply-chain.md | Rewrites vulnerable examples into non-runnable “shapes” to avoid scanner false positives. |
| .claude/skills/audit-pr/references/security-and-privacy.md | Rewrites vulnerable examples into non-runnable “shapes” to avoid scanner false positives. |
| .claude/skills/audit-pr/assets/review-summary.template.md | Updates summary template rules to require code fixes and mechanism-traceable explanations. |
| .claude/skills/audit-pr/agents/finding-refuter.md | Updates refuter instructions for the new 6-question refutation pass and no-per-finding tool runs. |
| .claude/skills/audit-docs/SKILL.md | Updates the “prove it” rule and adds explicit table-editing guidance. |
| .claude/skills/audit-docs/references/voice-and-ai-tells.md | Adds table-cell editing cautions and scope limits for the self-check section. |
Suppressed comments (1)
.claude/skills/audit-pr/agents/finding-refuter.md:111
- The refuter's REFUTED template still limits the failed-question range to "<1 to 5>", but the agent now has 6 refutation questions (Q1–Q6 above). This makes the template internally inconsistent and can mislead callers.
Q6 fix: verified | unverified | none proposed, then what was read
VERDICT: REFUTED
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
audit-prwas taking 11+ minutes, describing fixes instead of showing them, and sometimes explaining findings in ways the code doesn't back up. Also fixingaudit-docsbreaking Markdown tables, the audits not working on Codex, and the Snyk/Socket findings that #591 didn't clear.audit-prruntime:SKILL.mdsaid use the refuter subagent only on large reviews, its own description said once per finding, and each run could kick off lint/tsc/tests. Refutation is inline by default now, tool runs are out of the agentaudit-docstables: neither half had a rule for editing a table safely. Added one to both, and repeated it insidevoice-and-ai-tells.mdwhere the cell edit actually happensaudit-qualitywas the only skill hardcodingGlob/Grep/Read, describes the capability nowaudit-docsprompt: bare#codebase/#activePullRequest/#changestokens now have their plain meaning beside them (no context table, doesn't fit the line budget and prompts are Copilot-first)mainis what confirms. Snyk's line numbers are made up, it cited lines that have only ever been a table separator row