Skip to content

v2.8.0

Choose a tag to compare

@github-actions github-actions released this 25 May 16:38
572af41

Two themes: hardening the real-world document-drop journey, and shipping an honest open evaluation harness that proves the live agents detect document-buried ("narrative") risks the deterministic demo is blind to. The offline demo remains the default; all changes are backward-compatible.

Features

  • Real-world drop-flow hardening: a messy real-world pile (T12s, rent rolls, offering memos, plus junk files) now flows through classify → extract → review → workflow → export with no crashes, silent skips, or confidently-wrong numbers. Vacant $0 rent rows no longer deflate in-place rent averages; rent roll vs T12 is classified by content (not just filename); oversized/corrupt inputs degrade to a graceful parse_failed; parser errors redact local paths; and a npm run test:pile smoke test asserts a typed per-file outcome.
  • Threshold-driven IC verdict: the deterministic engine consults config/thresholds.json for dealbreakers and uses a deal-specific exit cap — fixing a clean deal wrongly marked FAIL.
  • Open evaluation harness (npm run eval): scores the orchestrator on a benchmark of 8 synthetic deals (core-plus / value-add / distressed, determinable + narrative planted risks) against committed ground truth and writes an honest trust report; npm run eval:offline runs the no-API layers.
  • Live narrative-risk proof + full 8/8 coverage: the live (Codex) layer covers all 8 deals and genuinely flags tenant concentration, insurance understatement, and missing Phase I that the deterministic fixture is blind to — narrative red-flag recall 100%, determinable financial 100%, dealbreaker recall 100%, IC verdict 88% exact (7 of 8). Two honest soft spots: model-dependent returns (IRR / equity multiple) at 50% — genuinely assumption-driven, with the agents diverging from the reference model in both directions — and one borderline deal (cp-insurance-understated) whose verdict oscillates CONDITIONAL↔FAIL across runs. Nothing tuned to flatter; ground truth fixed and committed. See eval/results/TRUST-REPORT.md.

Bug Fixes

  • eval: hardened the live-workpaper extractor — EGI sourced from the OpEx analyst, same-line value↔label matching (no cross-line binding), going-in metrics preferred over pro-forma/stabilized/exit/interest-only, and DSCR (amortizing): X parsing — backed by a required agent ## Metrics block and a threshold-driven verdict rule.
  • eval: the extractor preserves the negative sign on IRR / equity multiple, so a distressed deal's correctly-reported -99.0% IRR / -2.38x EM is no longer read as +99% / +2.38 (fixed ds-occupancy-collapse model-dependent scoring 0% → 100%; regression test added).

Full changelog: v2.7.0...v2.8.0