R-FDI v3.2.0 — Critical Risk Control & Line-Item Reliability

R-FDI v3.2.0 — Critical Risk Control & Line-Item Reliability

R-FDI v3.2.0 extends the frozen v3.0 model/runtime baseline and v3.1 analytics with row-aware line-item evaluation, production-safe risk features, grouped model selection, selective-risk analysis, and finite-sample certification analysis.

Scope

• Extractor weights did not change.
• V3.2 is a risk-control and line-item reliability extension; it does not retrain the extractor.
• CORD test is a RETROSPECTIVE_LOCKED_BENCHMARK, not a fresh untouched holdout.
• V1 NO_PROMOTION, V2 PROMOTE, V3 V3_EXTENSION_SUCCESS, and v3.1 V3_1_ANALYSIS_SUCCESS remain unchanged.
• Promotion in this extension could only mean shadow risk routing—not autonomous production action.

Methodology correction

Before public v3.2 publication, an audit found that E0 did not implement the intended historical comparison, benign row permutations were conflated with semantic row-association failures, and row matchability was named as though it represented field accuracy. The evaluator was corrected to report these concepts separately, and monetary-risk labels and all dependent risk evidence were regenerated. The pre-correction audit and methodology history remain preserved in the repository.

Corrected metrics

• E0 path-occurrence field F1 is 0.8016; E1 positional-row field F1 is 0.7887.
• E2 permutation-invariant matched-row F1 is 0.9534 and measures row matchability, not field accuracy. E2 matched-field micro-F1 is 0.8585 and row exact rate is 0.6023.
• Benign permutation-only prevalence is 0.0% in the retrospective sample; semantic row-association errors affect 10.0%. Benign permutations are excluded from monetary risk.
• The corrected broadly defined critical monetary-error prevalence is 41.0%; line-item critical-error prevalence is 33.0%.
• Raw confidence remains the best evaluated risk ranker: retrospective AURC 0.2356, PR-AUC 0.8200, and AUROC 0.8189.
• The best learned candidate, shallow gradient boosting R4, records AURC 0.3314, PR-AUC 0.5754, and AUROC 0.6554; it does not improve on raw confidence.
• At 50% review, raw confidence captures 75.6% of critical errors and leaves 20.0% critical risk among accepted documents. R4 captures 70.7% and leaves 24.0%.
• At 80% review, raw confidence captures 92.7% of critical errors and leaves 15.0% critical risk among accepted documents.
• High occlusion remains the largest controlled extraction degradation. The learned risk response is weak and inconsistent when only production-safe image descriptors vary.

Risk decision

V3_2_RISK_MODEL_NO_PROMOTION

R0 raw confidence remains stronger than R4: it has lower AURC and higher PR-AUC/AUROC, and it captures more critical errors with lower accepted critical risk at 50% review.

Certification

The already-observed certification partition was replayed: 5% has insufficient support and 10%/20% are uncertified. Because the evidence was reused after correction, this replay is non-confirmatory and promotion-ineligible; it is not fresh certification evidence.

Runtime

API and action semantics were not changed. Runtime routing remains unchanged, and the corrected-target R4 replay is retained as reproducible offline negative evidence.

Limitations

• This is a research portfolio, not a live bank deployment, regulatory certification, or production-risk guarantee.
• Only one physical GPU was used; no multi-GPU execution is claimed.
• No fresh DocILE LIR validation was available.
• CORD test and the certification partition had already been observed before correction.
• The robustness response reuses a 10-document development cohort and holds clean post-extraction features fixed.
• Fresh external line-item evidence and a genuinely fresh independent certification sample are required before reconsidering risk-model promotion.

Documentation

• Final report
• Methodology correction
• Row-aware line-item evaluation
• Risk certification

R-FDI v3.1.0 — Slicing, Error Analysis & Model Risk Analytics

R-FDI v3.1.0 — Slicing, Error Analysis & Model Risk Analytics

R-FDI v3.1.0 adds an evaluation-analytics layer to the frozen v3.0.0 model and runtime baseline. Model weights were not changed, locked-final inference was not rerun for tuning, and the historical V1 NO_PROMOTION, V2 PROMOTE, and V3 V3_EXTENSION_SUCCESS decisions remain intact.

What is new

• A canonical, provenance-traced document and field evaluation table derived from committed predictions and authorized local metadata.
• Predeclared complexity, financial-structure, image-quality, field-family, and task-compatible model slices with confidence intervals, practical thresholds, and Benjamini–Hochberg correction.
• An explicit error taxonomy, field criticality analysis, paired model deltas, and mix-shift/Simpson audit.
• Development-fit calibration evaluation using genuine Donut sequence confidence, plus selective-risk and review-budget analysis.
• Transparent illustrative business-policy costs and sensitivity frontiers; these are not costs from a bank or production deployment.
• A controlled development-only corruption study for Donut and PaddleOCR-VL 1.6 Docker/vLLM on one CUDA GPU.
• Interpretable exploratory failure-slice discovery and simulated drift monitoring, clearly separated from confirmatory and production evidence.
• Reproducible SVG figures, detailed reports, dedicated v3.1 governance files, evidence manifest, verifier, tests, and CI regeneration checks.

Scope and limitations

• v3.0.0 remains the model/runtime baseline; v3.1.0 is an analysis release, not a new production deployment.
• Calibration and routing results are limited to compatible frozen evidence. Better probability calibration does not imply better risk ranking.
• Robustness uses ten development documents and content-presence metrics for cross-model analysis; PaddleOCR-VL content presence is not official structured CORD F1.
• Drift is a transparent stress simulation, not observed bank production drift.
• Business costs are normalized examples, not institution-specific economics.
• Only one physical GPU was used. No regulatory-compliance claim is made.
• SROIE and DocILE remain optional HUMAN_ACTION_REQUIRED access items and do not block the release.

See the final v3.1 analysis report and executive findings.

R-FDI v3.0.0 — Reliable Financial Document Intelligence

R-FDI v3.0.0 — Reliable Financial Document Intelligence

R-FDI v3.0.0 completes a three-stage, evidence-governed research program for
financial document intelligence. The release decision is V3_EXTENSION_SUCCESS.
This is a production-style research simulation, not a live bank deployment or a
regulatory-certification claim.

Research lineage

• V1 — governance foundation: validated fail-closed workflow, evidence, risk,
  routing, and audit mechanics. Its historical NO_PROMOTION decision is retained.
• V2 — real-document and CUDA study: evaluated fixed model families on
  authorized public documents. The offline research candidate earned PROMOTE
  after all ten frozen gates passed.
• V3 — production hardening: executed runtime, persistence, reviewer, load,
  recovery, security, Docker GPU, and accelerated-serving studies. All eight
  frozen extension requirements passed.

Measured highlights

• Donut CORD locked-test leaf F1: 0.8372; total exact: 0.9895.
• PP-OCRv5 + rules CORD total exact: 0.4632.
• Paired total-exact gain: +0.5263, bootstrap 95% CI [+0.4316, +0.6211].
• LayoutLMv3 FUNSD locked-test macro-F1: 0.7009.
• PaddleOCR-VL on the same 20 CORD development documents: native mean latency
  20.403603 s versus Docker/vLLM 10.726820 s, or 1.902111x mean
  speedup. Served content-presence recall was 0.917128.

Runtime and reliability evidence

• Real Compose services verified for API, worker, reviewer, Redis, PostgreSQL,
  MinIO, and MLflow.
• MLflow metadata persisted in PostgreSQL and its matching artifact in MinIO.
• API/reviewer behavior passed 7/7 checks.
• Local executable fault study passed 7/7; infrastructure stop/restart recovery
  passed 3/3 for Redis, PostgreSQL, and MinIO.
• Docker exposed one RTX 4090 Laptop GPU; the bounded PaddleOCR-VL server retained
  at least 35.51% observed VRAM headroom.
• The 24-case adversarial benchmark preserved all boundaries and capability
  isolation. Detector misses and benign activations remain explicit.

Explicit limitations

PaddleOCR-VL figures are development content-presence measurements, not official
structured CORD F1 or locked-final results. The application uses a SQLite durable
queue and filesystem object store; Redis is not yet the active job transport.
DocILE and REFinD require human-authorized access. Textract was skipped without a
positive AWS budget and valid credentials. Compose credentials are local defaults.
No paid cloud, production-bank, regulatory-compliance, or multi-GPU claim is made.

See the V3 final report, V2 technical report, and reproduction guide.