v0.13.0-beta — Judge-primary grading + Composite Judges
Pre-releaseRed-team grading: judge-primary by default + Composite Judges (ADR-021→024)
The red-team grader — which decides whether each attack succeeded — moves from a keyword/substring oracle to LLM-judge-primary grading with honest-by-construction "Composite Judges." Each verdict is split into a positive-only compromise detector ⊕ a negative-only refusal detector, each structurally clamped so it can only raise its own direction or abstain. Driven by a reproducible finding that iteratively patching a keyword oracle does not converge to an honest grader.
⚠️ Breaking change (default behavior)
--judge-modenow defaults toprimary(wasfallback). With a judge configured, Composite Judges lead the verdict. A run with no--judgestays byte-identical to the keyword oracle, and--judge-mode fallbackrestores the prior behavior.- Default judge rubric is now
evidence-anchored(wasstrict).
Added
- Composite Judges for all six semantic oracles (InferenceAPIAbuse, InsecureOutput, SupplyChain, DataPoisoning, Misinformation, ExcessiveAgency).
- ADR-024 split-then-gate gated tree for InferenceAbuse — closes the "I am Grok" model-vs-product-bot floor on
gpt-4o-miniwith no stronger model; documented bound (it regresses InsecureOutput, so it's applied only where it helps). - Reasoning-model judges (o-series / GPT-5-class) — auto-retry without
temperature. --judge-timeoutnow bounds the composite path;AgentEval.SampleGraderscompanion.
Verification
κ=1.000 (n=92), 0 directional fabrications over the 314-case adversarial corpus; K=10 stochastic = 810 trials / 0 fabrications; full suite green (net8/9/10).
Migration: add --judge-mode fallback to keep the prior default grading; no change needed for no-judge runs.
Full details: CHANGELOG.md · ADR-021/022/023/024.