Feature Adversarial Validation

Feature: Adversarial Validation (Stage 4)

What it is. After all audit foci report their findings, Stage 4 merges them and runs a second, fresh-context model session per finding whose entire job is to try to refute it. Only findings that survive get promoted. Default model: Opus — this is the lever on the accepted-report rate.

The diagram shows the full verification lifecycle — where validation sits, and every way a finding can be dropped, downgraded, or kept as it travels to the report:

Downgrade, don't delete

Validation is not binary confirm/refute. refuted is reserved for findings provably contradicted by code (or matching a false-positive carve-out from recon — see Archetype-Driven Prompts). Anything merely uncertain is kept as needs_runtime_verification with a concrete follow-up question, rather than being silently discarded. This is a deliberate design choice: an audit's value is measured by recall as much as precision, and a false "refuted" is worse than an honest "uncertain".

What feeds the validator

• Ground truth — the same invariants, baseline-correct references, and carve-outs recon extracted, so the validator doesn't waste a session re-deriving what recon already proved and wrongly refuting a real finding.
• Code excerpts — a configurable window of source lines around each cited file:line.

Cross-focus semantic dedup and citation grounding

Before the expensive adversarial fan-out, two cheap passes run:

• Semantic dedup clusters findings that describe the same root-cause bug reported independently by different audit foci at different call sites (a real, observed failure mode — one bug was reported three times by three foci). Conservative by design: a missed duplicate just costs a little extra validation later; a wrong merge would silently drop a real, distinct finding.
• Citation grounding is a deterministic, zero-LLM check that a finding's cited file/symbol actually exists in this repo — catching a hallucinated citation (e.g. a finding that cites code from a completely different, similarly-named project) before spending a session validating it. A finding whose primary file doesn't exist anywhere in the repo is auto-dropped; a finding with an absent cited symbol is kept but confidence-downgraded with a warning surfaced to the validator.

Batched sessions

Findings are batched (default 8 per session) into shared validation sessions that judge each independently — this collapsed what used to be a one-session-per-finding fan-out, cutting cost and session/rate-limit pressure with no precision loss.

What happens to a session that dies mid-validation

A validate session that fails leaves its findings needs_runtime_verification rather than losing them — survivors either way, never silently dropped. The stage tags whether an "unknown" verdict means the model genuinely couldn't tell (a real quality signal) or the session never ran at all (a pure tooling gap), so a reader isn't left guessing which happened.

Related

• Docs & History Corroboration — the next stage after validate, cross-checking survivors against docs and VCS history.
• Runtime Verification — turns needs_runtime_verification findings into HTTP-level confirmed/refuted with a real PoC.
• Interrogation Chat — reuses this exact validator (validate._validate_one) to re-check a candidate finding you propose in chat.