From the August 2026 harness audit (accuracy of published cost/latency columns; fix lands with the v2 pipeline).
Symptoms (verified)
- Null latency exports as 0.0 (GPT-5.5 batch rows show 0.0s, rendering as impossibly fast) and
cost_is_estimated is False where the value is in fact estimated.
- Spend on failed calls and repair rounds is omitted from published per-model cost: Inkling metered $9.834 actual vs $9.1474 published.
- $64.08 aggregate discrepancy between the frozen report's cost table and live modelStats, from mixing accounting bases (litellm map consulted before PRICE_OVERRIDES in one path).
Fix
- Latency columns nullable end-to-end; render "—" not 0.0.
- One documented cost basis: billed rates via PRICE_OVERRIDES everywhere, litellm map only as fallback with the estimated flag set.
- Cost ledger includes failed + repair spend, published as total-spend; per-household cost states its numerator definition.
Tests lock each: null latency round-trip, override-before-map ordering, ledger includes a failed call.
From the August 2026 harness audit (accuracy of published cost/latency columns; fix lands with the v2 pipeline).
Symptoms (verified)
cost_is_estimatedis False where the value is in fact estimated.Fix
Tests lock each: null latency round-trip, override-before-map ordering, ledger includes a failed call.