Architecture

Architecture

The pipeline is orchestration-only glue around a handful of reusable prompt assets. The security logic lives in the prompts (argo/prompts/); the code ingests a program, sequences the stages, and produces a reviewable report. It writes no audit logic of its own.

Module map

argo/
  cli.py            Typer CLI: ingest / recon / run / validate / corroborate / report / pipeline
  orchestrator.py   wiring: build a RunContext, generate run IDs, drive the stages
  config.py         PipelineConfig: per-stage models, tool allowlists, budgets, caps
  context.py        RunContext (paths + scope loading + budget guard) + artifact collection
  models.py         pydantic models mirroring the JSON Schemas (Scope, Finding, ...)
  schemas.py        Draft-07 validation against scope_schema.json / findings_schema.json
  guardrails.py     tool allowlist enforcement, prohibited-technique assertions, scope filter
  rendering.py      placeholder fill, the .j2 template, the artifact-contract epilogue
  runner.py         AgentRunner interface + HeadlessClaudeRunner · CodexRunner · MockClaudeRunner
  ranking.py        severity/confidence ordering, ref parsing, dedup_key
  ledger.py         SQLite: llm_calls (cost) + findings_ledger (cross-run dedup)
  progress.py       ProgressReporter -> runs/<id>/status.json (live stage timeline + cost)
  chat.py           interactive analyst over a completed run (read-only repo; test-gen; re-validate)
  knowledge.py      vuln-class index loader (data/vuln_index.yaml) injected into recon
  checklists.py     mandatory coverage checklist injected into every audit prompt
  census.py         cross-file variant-census worksheet (concrete site+file extent per prompt)
  costs.py          cost analytics from the ledger (by model / stage / run / archetype)
  quality.py        accept-rate (ledger) paired with benchmark recall
  archetype.py      canonical software archetypes + normalizer
  fixes.py          remediation: propose a patch per confirmed finding (opt-in)
  verify.py         patch verification on an ISOLATED COPY (applies? compiles? no new errors?)
  benchmark.py      eval: score findings P/R/F1 vs labeled suites (by archetype / CWE) + A/B
  stages/
    ingest.py  research.py  recon.py  audit.py  sca.py  validate.py  corroborate.py  runtime.py  report.py  live.py
  prompts/          the assets, version-pinned (sha256 recorded per run)

server/             HTTP API on top of the pipeline (FastAPI) — see the Web UI page
webapp/             no-build web UI (vanilla ES modules + CSS) — see the Web UI page

Stage data flow

Each stage reads the previous stage's files from runs/<RUN_ID>/ and writes its own.

Stage	Reads	Writes
1 Ingest	brief (or none → local review), repo, optional --links/--accepted-risks	scope.json, meta.json, read-only repo/
0 Research	scope.json	research_brief.md, threat_intel.json — see Threat-Informed Audit
2 Recon	scope.json, repo/, research_brief.md	repo_profile.json, prompts/audit_*.md, synthesis_notes.md, ground_truth.json — see Archetype-Driven Prompts
3 Audit	prompts/, repo/	findings/<focus>.json, variant_logs/<focus>.md
SCA	repo/ manifests, scope.json	findings/dependencies.json (opt-out)
4 Validate	findings/, repo/, ground_truth.json	validated_findings.json — see Adversarial Validation
CORROBORATE	validated_findings.json	rewritten with a corroboration block + fixed_upstream appendix — see Docs & History Corroboration
RUNTIME	validated_findings.json, repo/	runtime_results.json (opt-in) — see Runtime Verification
5 Report	validated_findings.json	REPORT.md, submission_drafts/, ledger rows

pipeline runs 1→5 (SCA between audit and validate, corroborate after validate — both on by default), and stops before any submission.

Precision + depth uplift

The single biggest quality lever is how much ground truth recon bakes into the audit prompts. Recon performs a deep ground-truth extraction and emits, per focus: invariants (location → expected → how-to-check triples), baseline-correct references (the one place a systemic pattern is done right — every sibling is diffed against it), variant families (the concrete, enumerated member list of each repeated shape), and false-positive carve-outs (also handed to the validator so it stops re-deriving and wrongly refuting real findings).

The audit template mandates a VARIANT_HUNT_LOG (one row per family member) as a coverage forcing-function, plus a completeness-critic re-pass (--critic-passes, default 1) that re-audits each focus for what was missed, looping until dry. Validate switches from binary confirm/refute to downgrade-don't-delete: refuted is reserved for findings provably contradicted by code; anything merely uncertain is kept as needs_runtime_verification.

Cross-focus semantic dedup then citation grounding run between the structural merge and the (expensive) adversarial fan-out: semantic dedup clusters findings that describe the same root cause from different angles (conservatively — a missed duplicate just costs extra validation, a wrong merge would silently drop a real finding); citation grounding is a deterministic, zero-LLM check that a finding's cited file/symbol actually exists in this repo, catching hallucinated citations before spending a validation session on them.

The verification lifecycle of a single finding — every gate, and every way it can be dropped, downgraded, or kept — is shown here (full detail on the Adversarial Validation page):

The AgentRunner abstraction

Every LLM call goes through one chokepoint, so guardrails and cost logging cannot be bypassed:

class AgentRunner(ABC):
    def run(self, *, prompt, run_dir, work_dir, model, stage, run_id,
            repo_dir=None, allowed_tools=ARTIFACT_TOOLS, label=None) -> LLMResult: ...

run() sanitizes the tool allowlist and asserts no network tool, computes the per-session budget, delegates to the backend, strictly parses the result envelope, logs the call to the ledger (always, even on error), and enforces per-session caps. See Multi-Backend for the concrete backends (HeadlessClaudeRunner, CodexRunner, MockClaudeRunner, FallbackRunner).

Files are the source of truth

Every stage artifact exists in two places, by design: runs/<id>/work/<stage>/… (scratch — the raw file the LLM session wrote, used for partial recovery and audit trail) and runs/<id>/… (canonical — the orchestrator's normalized, schema-validated result; downstream stages read only this one). collect_output_files() reads the manifest's index and unions a scratch-dir glob, so a missing/partial manifest or a session that died mid-write still recovers whatever was written.

Dedup algorithm

dedup_key = sha1(normalize(primary_file + primary_line + cwe))

Findings sharing a key collapse to one; the keeper is chosen by (highest severity, then highest confidence, then first seen), and the others' affected/variants are unioned in.

SQLite ledger

Two tables: llm_calls (cost control + the hard --budget guard) and findings_ledger (cross-run/cross-program resubmission detection, plus triager_* columns holding real-world accept/reject feedback — see Benchmarks & Costs).