feat: add adversarial review agents for code and documents#403
Merged
Conversation
Two new conditional reviewers that take a falsification stance -- actively constructing failure scenarios rather than checking against known patterns. Code adversarial-reviewer: assumption violation, composition failures, cascade construction, and abuse cases. Auto-scales quick/standard/deep based on diff size and risk signals. Document adversarial-document-reviewer: premise challenging, assumption surfacing, decision stress-testing, simplification pressure, and alternative blindness. Auto-scales based on document size and domain risk. Both integrate into existing review ensembles (ce-review and document-review) as cross-cutting conditional reviewers using the standard findings contract.
e27db33 to
6c6aabe
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds two adversarial review agents that take a fundamentally different stance from existing reviewers: instead of evaluating quality against known criteria, they actively try to falsify the artifact by constructing scenarios that break it.
Both use the standard JSON findings contract, integrate as cross-cutting conditionals in their respective ensembles, and have explicit suppress conditions to avoid overlap with existing reviewers. Skill-level routing excludes test-only, generated, and lockfile diffs from the size threshold.
Measured impact
Tested the adversarial code reviewer against 3 synthetic high-risk diffs (payment webhook handler, Redis rate limiter + cache, CSV user import with migration) and 1 negative control (small config rename).
Assertion pass rate: 100% adversarial vs 61% baseline on findings that require multi-step failure reasoning -- cascades, cross-component composition failures, and abuse scenarios. Examples of findings unique to the adversarial reviewer:
KEYSinvalidation through Redis blocking to fleet-wide rate-limit bypassThe existing persona reviewers (correctness, security, reliability) consistently caught individual bugs in the same code but did not trace multi-step failure chains across component boundaries. The adversarial reviewer also used fewer tokens per run (34K avg vs 49K) since it focuses on its specific domain rather than full-spectrum review.
Negative control (small config rename): adversarial reviewer correctly not selected on either skill version. No noise regression -- both versions produced clean reviews.
Design decisions
Test plan
bun testpasses (497/497), including the path-sanitization collision test that caught the original shared-name issuebun run release:validateconfirms 46 agents, 42 skills, 1 MCP server in sync🤖 Generated with Claude Opus 4.6 (1M context) via Claude Code