Skip to content

SCOUT v2.5.0 — Structured LLM + EPSS + Observability + CRA Compliance

Choose a tag to compare

@R00T-Kim R00T-Kim released this 13 Apr 04:11

Strategic Roadmap Phase 1 implementation. Based on 30+ academic papers (LATTE, Operation Mango, HouseFuzz, VulnSage) and competitive analysis (Theori Xint, FirmAgent, EU CRA).

Highlights

LLM structured output (parse failure 100% → 0%)

  • New llm_prompts.py: 7 role-based system prompts (ADVOCATE/CRITIC/TAINT/CLASSIFIER/REPAIR/SYNTHESIS) + temperature constants
  • LLMDriver Protocol: system_prompt + temperature parameters wired into all 4 drivers (Codex, Claude API, Claude Code CLI, Ollama)
  • 5-stage JSON parser: preamble strip → fence → raw → brace-counting → common error fix; optional required_keys schema validation
  • All LLM-using stages updated: adversarial_triage, taint_propagation, semantic_classifier

Sink expansion (taint_propagation.py)

  • _SINK_SYMBOLS: 11 → 28 (memcpy, memmove, strcat, strncpy, gets, vsprintf, printf family, scanf family, dlopen, realpath)
  • _FORMAT_STRING_SINKS + _is_format_string_variable() helper for variable-controlled format string detection

EPSS scoring (cve_scan.py)

  • FIRST.org API integration with batched queries (30 IDs/request)
  • Per-run + cross-run cache
  • Confidence adjustment by EPSS percentile (≥0.10: +0.10, ≥0.01: +0.05, <0.001: -0.05)

Observability

  • Separate parse_failures vs llm_call_failures counters in adversarial_triage and fp_verification
  • LLM failure classification helpers: quota_exhausted, driver_unavailable, driver_nonzero_exit

CI/CD & Compliance

  • New .github/actions/scout-scan/: composite GitHub Action with SARIF upload to GitHub Security tab
  • docs/cra_compliance_mapping.md: EU Cyber Resilience Act Annex I 12 essential requirements mapped to SCOUT outputs
  • docs/strategic_roadmap_2026.md: 3-Phase plan (v2.5 → v3.0 → v4.0)

Bug Fixes

  • CVE scan signature-only path: removed early return so signature-only matches go through the same enrichment / finding-candidate pipeline as NVD matches
  • CVE scan comp variable bug: backport confidence adjustment now uses per-match component metadata instead of leaked outer loop variable
  • Semantic classifier batch size: reduced from 50 → 15 functions per LLM call to prevent JSON schema loss in long contexts

R7000 Verification (Netgear, 31MB, codex driver, 2026-04-13)

Metric Pre-v2.5 (1211 run) v2.5.0 (1320 run)
adversarial_triage parse_failures 100/100 0/100
adversarial_triage parsed_ok 0/100 100/100
fp_verification unverified 97/100 0/100
fp_verification true_positives 1 57
fp_verification false_positives 2 43
cve_scan EPSS enriched 0/23 23/23

Adversarial debate: 100 debated → 99 downgraded (FP) + 1 maintained (TP)

Run: `aiedge-runs/2026-04-12_1320_sha256-b28bf08e9d2c`

Stats

  • 21 files changed
  • 3,695 insertions / 792 deletions
  • 7 new files

See CHANGELOG.md for full details.