v2.8.0
Two themes: hardening the real-world document-drop journey, and shipping an honest open evaluation harness that proves the live agents detect document-buried ("narrative") risks the deterministic demo is blind to. The offline demo remains the default; all changes are backward-compatible.
Features
- Real-world drop-flow hardening: a messy real-world pile (T12s, rent rolls, offering memos, plus junk files) now flows through classify → extract → review → workflow → export with no crashes, silent skips, or confidently-wrong numbers. Vacant
$0rent rows no longer deflate in-place rent averages; rent roll vs T12 is classified by content (not just filename); oversized/corrupt inputs degrade to a gracefulparse_failed; parser errors redact local paths; and anpm run test:pilesmoke test asserts a typed per-file outcome. - Threshold-driven IC verdict: the deterministic engine consults
config/thresholds.jsonfor dealbreakers and uses a deal-specific exit cap — fixing a clean deal wrongly marked FAIL. - Open evaluation harness (
npm run eval): scores the orchestrator on a benchmark of 8 synthetic deals (core-plus / value-add / distressed, determinable + narrative planted risks) against committed ground truth and writes an honest trust report;npm run eval:offlineruns the no-API layers. - Live narrative-risk proof + full 8/8 coverage: the live (Codex) layer covers all 8 deals and genuinely flags tenant concentration, insurance understatement, and missing Phase I that the deterministic fixture is blind to — narrative red-flag recall 100%, determinable financial 100%, dealbreaker recall 100%, IC verdict 88% exact (7 of 8). Two honest soft spots: model-dependent returns (IRR / equity multiple) at 50% — genuinely assumption-driven, with the agents diverging from the reference model in both directions — and one borderline deal (
cp-insurance-understated) whose verdict oscillates CONDITIONAL↔FAIL across runs. Nothing tuned to flatter; ground truth fixed and committed. See eval/results/TRUST-REPORT.md.
Bug Fixes
- eval: hardened the live-workpaper extractor — EGI sourced from the OpEx analyst, same-line value↔label matching (no cross-line binding), going-in metrics preferred over pro-forma/stabilized/exit/interest-only, and
DSCR (amortizing): Xparsing — backed by a required agent## Metricsblock and a threshold-driven verdict rule. - eval: the extractor preserves the negative sign on IRR / equity multiple, so a distressed deal's correctly-reported
-99.0% IRR / -2.38x EMis no longer read as+99% / +2.38(fixedds-occupancy-collapsemodel-dependent scoring 0% → 100%; regression test added).
Full changelog: v2.7.0...v2.8.0