Feature Interrogation Chat

Feature: Interrogation Chat

The chat layer turns a one-shot report into a conversation that attacks the dominant bug-bounty failure mode — false negatives. It's seeded with the run's full context (scope, repo profile, synthesis notes — what was deprioritized and why, validated findings) plus read-only repo access, so it can re-investigate the code and answer with evidence, not vibes.

Three things that make it useful for real work (not just a demo)

A real, unedited worked transcript (linked in the repo's docs/chat-example.md) demonstrates all three, on a tiny synthetic vulnerable repo, three turns for $0.40 total:

1. Grounded explanation. "Explain ARCH-001 concretely: the tainted data flow, a one-line exploit, cite file:line" got an exact source→sink trace with line numbers, a working exploit, and a correct technical nuance the general audit prose hadn't spelled out.
2. False-positive self-correction. Asked whether a file listed in a finding's affected list was actually vulnerable, the analyst correctly identified it as a safe cross-reference (a parameterized query), not a real finding — and explained why.
3. Honest false negatives. Asked "did you report any XSS? genuinely none, or did you miss it?" the answer was neither false reassurance nor fabrication: it explained the rendering layer wasn't present in the reviewed source, so XSS couldn't be confirmed or ruled out, and that it was correctly deprioritized as a residual unknown rather than fabricated.

Re-validating a missed finding

When you challenge a specific, localized hypothesis ("why didn't you flag the IDOR at orders.py:120?"), the analyst can write a CANDIDATE_FINDING.json, which is then re-checked independently by the same adversarial validator the pipeline itself uses (validate._validate_one — isolated, read-only, refute-first). The verdict (✅ confirmed / ❌ refuted / ⚠️ needs-runtime) is appended to the reply and shown as a pill in the UI. This is the important part: it's a real re-validation, not the model being talked into agreeing with you — a coaxed or vague hypothesis gets refuted exactly as it would in the real pipeline.

Limits, kept honest

• Probe, not pipeline. A re-validated candidate is an interactive probe — it is not added to validated_findings.json or the registry.
• No token streaming — one model call per turn, history re-sent each turn (capped).
• Quality tracks the model. A strong model answers well; a small local model less so — exactly what Benchmarks & Costs measures across backends.

Test-suite generation

You can also ask the analyst to "generate a test suite for CWE-89" — generated files are written to runs/<id>/generated/ only, never into the target repo (verified: generated files never land in repo/).

In the web UI

A Chat results tab with canned actions ("explain a finding", "why didn't you find…", "generate tests for a CWE", "what did you deprioritize?"). See Web UI.

Related

• Adversarial Validation — the exact validator chat reuses for candidate re-checks.
• Guardrails & Safety — read-only repo access applies here too.