Skip to content

v2.7.1 — Phase 2C+.4 vendor corpus expansion (1/5 → 2/5 PASS)

Choose a tag to compare

@R00T-Kim R00T-Kim released this 22 Apr 14:27

Phase 2C+.4 vendor corpus expansion — quantitative refinement of v2.7.0's scenario C

v2.7.1 closes Phase 2C+.4 by extending the pair-eval corpus from 7 to 12 vendor/model pairs: D-Link DIR-859 (CVE-2019-17621), D-Link DIR-878 (vendor advisory), ASUS RT-AC68U (CVE-2020-15498), Linksys WRT1900AC v2 (progression), and Linksys EA6700 (progression). Phase 2D' Entry Gate scorecard transitions from 1/5 PASS → 2/5 PASS.

Phase 2D' Entry Gate scorecard (FINAL, WRT1900AC v2 ok measurement of record)

Gate Threshold v2.7.0 (7-pair) v2.7.1 FINAL (12-pair) Verdict
1 Recall ≥ 0.40 0.1429 0.1667 (+17% rel) ❌ FAIL
2 Tier variation ≥ 2 nonzero TP tiers 1 (symbol_only) 1 (back-slide) ❌ FAIL
3 Finding diversity < 0.50 1.000 0.917 ❌ FAIL
4 Dedicated rerun ≥ 1/N 14/14 (Codex) 14/14 + 12/12 (--no-llm) ✅ PASS
5 Corpus ≥ 10 7 12 ✅ PASS

The +1 net gain over v2.7.0 (1/5 → 2/5) comes from Gate 5 (Corpus), which clears by manifest registration alone. Gate 1 absolute recall improves but stays well below threshold; the new TP/FP pair (DIR-859 vuln + patched both hit `aiedge.findings.web.exec_sink_overlap`) corroborates the v2.7.0 diagnosis that `findings.py`'s single-synthesis-finding selection is the structural Gate 1/3 limit.

Honest figure-of-record protocol

An intermediate 1st-pass measurement under partial WRT1900AC v2 extractions (1200-second budget) transiently showed Gate 2 PASS due to `aiedge.findings.analysis_incomplete` populating the `unknown` tier. The 2400-second budget rerun ok-state measurement reverts Gate 2 to FAIL — partial-extraction artifacts can falsely populate Gate-2 tiers, so the ok-state measurement is the figure of record. See `docs/v2.7.1_release_plan.md` for the full measurement history and Gate Diagnosis Matrix.

Pivot Option D unchanged

v2.7.1 is a quantitative refinement of v2.7.0's scenario C, not a re-pivot. The compliance-led identity remains primary. The `compliance_report` stage and four standard mappings (CRA Annex I / FDA Section 524B / ISO/SAE 21434 / UN R155) shipped in v2.7.0 are unchanged.

Notable fixes

  • `scripts/score_pair_corpus.py` graceful-skip for missing pair runs — no more `StopIteration` crashes when scoring corpus growth or partial-coverage measurements; missing rows are recorded as `vulnerable_status="missing"` / `patched_status="missing"` and excluded from recall/FPR denominators.

Known issues (out of v2.7.x scope)

  • DIR-878 partial extraction — SHRS-encrypted inner `.bin` not yet decrypted. `vendor_decrypt.py` extension is a follow-on task.
  • Gate 1/2/3 structural limit — `findings.py` single-synthesis-finding selection is the root cause; resolution belongs to the external detection-engine track, not v2.7.x scope.

Full changelog

See `CHANGELOG.md` for the complete `[2.7.1]` section.