v0.2.1 - real reviewer-agent lab
Put a real AI reviewer behind Ward and see the difference
ward lab attack proved whether an injection reaches an agent's
context. ward lab review (new) goes further: it puts a real reviewer
agent behind Ward and shows what the agent does - the before/after
that is the whole portfolio thesis.
# Offline reviewer (no API key) - deterministic, models a naive agent:
ward lab review
# Real Claude reviewer:
pip install "ward-scanner[judge]" && export ANTHROPIC_API_KEY=sk-...
ward lab review --reviewer anthropicThe asymmetry, on the six bundled malicious PRs (offline)
| Count | |
|---|---|
| Blocked by Ward before the reviewer ran | 6 / 6 |
| Naive reviewer approved the malicious PR without Ward | 5 / 6 |
| Naive reviewer approved the malicious PR with Ward | 0 / 6 |
Framed honestly
The report never claims "the model always gets hijacked" - modern models
often resist. The point it makes is the honest one: with Ward the
injection is refused before the reviewer's context is populated
(deterministic, provable, model-agnostic); without Ward, whether the
model resists is a gamble. Ward turns "hope the model resists" into
"the model never sees it".
The AnthropicReviewer uses a realistic, deliberately un-hardened
prompt - a typical developer-built reviewer, i.e. the thing being
protected, in deliberate contrast to the hardened judge tier.
Also in this release
RELEASING.md- the maintainer runbook for PyPI (trusted-publisher
OIDC setup) and GitHub Marketplace, sopipx install ward-scanner
becomes real.
By the numbers
- 250 tests (up from 233)
- New
src/ward/lab_reviewer.py+ward lab reviewcommand - Reuses the judge tier's Claude plumbing (injectable client, lazy import,
graceful degradation with no key) - Detection benchmark unchanged (75.2% smoke / 53.5% full, 0.0% FPR) -
this is portfolio/demo tooling, not a detection change - mypy strict, ruff clean, CI green across the matrix
Upgrade
pipx install "ward-scanner[judge]" # reviewer lab with real Claude- uses: sonofg0tham/ward@v0.2.1