Design Decisions

Design Decisions & Limitations

The deliberate choices that define what Argo is — and just as importantly, what it is not.

Where Argo sits: LLM-native SAST

Argo is a static vulnerability detector for source code, in the same family as CodeQL and Semgrep — but where those match hand-written rules/queries against a code graph, Argo has an LLM read the source semantically. The trade is explicit: no rules or queries to author, and it surfaces logic/authorization bugs that fixed patterns miss, but it is probabilistic (recall and precision vary by model and run) rather than deterministic and exhaustive. It's a complement to rule-based SAST, not a drop-in replacement — and a different tool from dynamic analyzers: Argo never executes the target, so it is not a DAST, fuzzer, or symbolic executor.

Bug bounty is one mode, not the identity. The same engine runs as a general code auditor or as bug-bounty triage — see Two Modes. The detection core is identical.

Orchestration-only glue

Argo is a sequencer, not an analyzer. It writes no audit logic of its own — the detection knowledge is in version-pinned prompt assets (sha256-recorded per run). This keeps the system small, auditable, and lets the "intelligence" be improved by editing prompts rather than code.

The key decision: no code-property-graph / AST engine

Argo does NOT use tree-sitter AST metadata, a code-property graph (CPG), PDG/CFG, or a taint engine. Code understanding is done by the LLM reading the source semantically. This is gated on evidence, not closed forever. Why not:

1. Source code is in-distribution for the model; a serialized CPG/AST is not. LLMs are pretrained overwhelmingly on source code, not on graph IRs — feeding one in adds an out-of-distribution artifact that can confuse rather than help.
2. Unproven ROI on top of a strong model. A capable LLM already performs cross-file semantic reasoning that subsumes much of what a graph provides; no evidence yet that it misses flows a graph would catch.
3. Guardrail tension (the decisive one). Build-based CPG tools must compile the target — directly violating "no code execution, read-only, source-static only".
4. Complexity & maintenance. A useful static-analysis layer is per-language and open-ended.
5. Methodological clarity for the study. Bolting a graph engine on confounds the contribution — a confirmed finding could come from the LLM or the graph, and the two can't be separated post-hoc. Keeping the pipeline LLM-pure isolates the variable under study.

What's used instead (cheap, safe, additive, no build): the per-focus recon split, the curated vulnerability-class index, Stage-0 web research, and adversarial validation — see Archetype-Driven Prompts and Adversarial Validation.

When this would get revisited. If the benchmark shows recall losses attributable to missed inter-procedural/multi-file data flows a graph would recover, a parse-only path (tree-sitter AST + call-graph, no build) comes first; build-based CPG would come only after that, only as a data-flow validation aid, never as raw context. The trigger is measured evidence, not intuition.

Detection-only, read-only, never live (recap)

Full detail: Guardrails & Safety.

Opt-in remediation, kept off the detection path

See Remediation & Fixes — a separate flow, so the audit's read-only guarantee stays absolute.

Threats to validity (for the research angle)

• LLM-centric recall — bounded by what the model reasons about. Measured directly: benchmark recall paired with the registry's real-world accept rate — see Benchmarks & Costs.
• Model-dependence, turned into a result — because the backend is swappable with the same prompts/pipeline/guardrails, "model" is the only variable that changes between backends, making cross-model comparison a clean experiment rather than a confound. See Multi-Backend.
• Non-determinism & cost — prompt-asset sha256 + the analyzed commit are pinned per run; every call's cost is logged.
• No dynamic confirmation by default — source-static by design; needs_runtime_verification is a first-class verdict precisely so this gap is explicit in the data, not hidden. (Opt-in bridges exist: Runtime Verification and Live Verification.)
• Scope honesty — a code-side scope filter drops out-of-scope findings independently of the LLM verdict.

The one-line summary: Argo is a deliberately lean, LLM-driven, source-static pipeline. Its power and its limits both come from that choice — and the choice is measured, not assumed.