Skip to content

Cut the audit-pr runtime, stop audit-docs breaking tables, and make the audits portable - #594

Merged
AlexJSully merged 2 commits into
mainfrom
alexjsully-260806-adjusting-skills-again
Aug 6, 2026
Merged

Cut the audit-pr runtime, stop audit-docs breaking tables, and make the audits portable#594
AlexJSully merged 2 commits into
mainfrom
alexjsully-260806-adjusting-skills-again

Conversation

@AlexJSully

Copy link
Copy Markdown
Owner

audit-pr was taking 11+ minutes, describing fixes instead of showing them, and sometimes explaining findings in ways the code doesn't back up. Also fixing audit-docs breaking Markdown tables, the audits not working on Codex, and the Snyk/Socket findings that #591 didn't clear.

  • audit-pr runtime: SKILL.md said use the refuter subagent only on large reviews, its own description said once per finding, and each run could kick off lint/tsc/tests. Refutation is inline by default now, tool runs are out of the agent
  • Added an execution budget to all three audits, none of them said when to stop
  • Dropped the "or pseudocode" escape hatch, so a code finding carries actual code
  • New refutation question: point at a line for every step of the explanation, or the finding is dropped
  • audit-docs tables: neither half had a rule for editing a table safely. Added one to both, and repeated it inside voice-and-ai-tells.md where the cell edit actually happens
  • audit-quality was the only skill hardcoding Glob/Grep/Read, describes the capability now
  • audit-docs prompt: bare #codebase/#activePullRequest/#changes tokens now have their plain meaning beside them (no context table, doesn't fit the line budget and prompts are Copilot-first)
  • Snyk W007: redaction folded into the quote rule itself instead of sitting beside it
  • Socket: vulnerable examples rewritten as non-runnable shapes
  • Can't verify either scanner locally, the re-audit on main is what confirms. Snyk's line numbers are made up, it cited lines that have only ever been a table separator row

@vercel

vercel Bot commented Aug 6, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
portfolio Ready Ready Preview Aug 6, 2026 8:36pm

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR updates the repository’s audit prompts and skill bundles to reduce audit runtime, tighten evidence requirements (including credential redaction), and make the audits more portable across different agent hosts. It also includes small dependency bumps reflected in the lockfile.

Changes:

  • Refines audit-pr/audit-quality evidence and refutation guidance, adds an explicit execution budget, and makes examples non-runnable shapes.
  • Adds safer Markdown table-editing rules to audit-docs (prompt + skill + voice reference) to avoid breaking table structure.
  • Bumps webpack and sass, updating package-lock.json accordingly.

Reviewed changes

Copilot reviewed 13 out of 14 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
package.json Bumps webpack and sass versions.
package-lock.json Updates lockfile for the dependency bumps (including transitive changes).
.github/prompts/readme.md Documents how prompt vs skill halves resolve # references, and clarifies portability expectations.
.github/prompts/audit-quality.prompt.md Updates context-resolution table wording to describe capabilities rather than product-specific tool names.
.github/prompts/audit-pr.prompt.md Tightens evidence, redaction, execution budget, and refutation-pass guidance in the prompt half.
.github/prompts/audit-docs.prompt.md Clarifies #codebase/#activePullRequest/#changes meaning inline, adds explicit table-editing rules.
.claude/skills/audit-quality/SKILL.md Updates portability language, evidence/redaction rules, and adds an execution budget.
.claude/skills/audit-pr/SKILL.md Updates description, evidence requirements, execution budget guidance, and refutation expectations.
.claude/skills/audit-pr/references/supply-chain.md Rewrites vulnerable examples into non-runnable “shapes” to avoid scanner false positives.
.claude/skills/audit-pr/references/security-and-privacy.md Rewrites vulnerable examples into non-runnable “shapes” to avoid scanner false positives.
.claude/skills/audit-pr/assets/review-summary.template.md Updates summary template rules to require code fixes and mechanism-traceable explanations.
.claude/skills/audit-pr/agents/finding-refuter.md Updates refuter instructions for the new 6-question refutation pass and no-per-finding tool runs.
.claude/skills/audit-docs/SKILL.md Updates the “prove it” rule and adds explicit table-editing guidance.
.claude/skills/audit-docs/references/voice-and-ai-tells.md Adds table-cell editing cautions and scope limits for the self-check section.
Suppressed comments (1)

.claude/skills/audit-pr/agents/finding-refuter.md:111

  • The refuter's REFUTED template still limits the failed-question range to "<1 to 5>", but the agent now has 6 refutation questions (Q1–Q6 above). This makes the template internally inconsistent and can mislead callers.
Q6 fix: verified | unverified | none proposed, then what was read
VERDICT: REFUTED

Comment thread .claude/skills/audit-quality/SKILL.md Outdated
@AlexJSully
AlexJSully merged commit 948e92b into main Aug 6, 2026
2 checks passed
@AlexJSully
AlexJSully deleted the alexjsully-260806-adjusting-skills-again branch August 6, 2026 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants