Pricing is silently wrong: no in-process refresh, unknown models bill $0, fuzzy tiers invisible

[pitimon]

Problem

Three related defects, all in the "the dollar number is confidently wrong and nothing says so" class. They share one fix, so they are tracked together.

1. Pricing never refreshes inside a running process

src/lib/pricing/index.js:52 — ensurePricingLoaded() returns immediately once state.loaded is true. There is no loadedAt, no reload path, and the only call sites are src/commands/serve.js:90 (once at startup) and src/lib/local-api.js:1393 (a no-op after the first load). The 24h TTL in litellm-fetcher.js therefore governs only which snapshot is chosen at startup — never a refresh.

This is the systemic cause of the claude-opus-5 incident on 2026-07-25: the dashboard LaunchAgent had been up 21 hours holding a snapshot written before the model existed upstream. It showed 3.87M tokens at $0.000000, understating the day by $7.37, for 21 hours, with no signal. Deleting the cache file was incidental — a restart alone would have fixed it, and nothing short of a restart ever will.

It recurs on every new model and every upstream price change. The next occurrence is already scheduled: Sonnet 5 intro pricing reverts on 2026-08-31 (see #87), and a dashboard that has been up since August will keep billing the intro rate indefinitely.

2. Unknown model → silent $0; the provenance is computed and then discarded

src/lib/pricing/index.js:101-104 — lookupPricing() returns {hit, source, value}, but getModelPricing() throws source away and returns bare numbers. On a miss it negative-caches and returns ZERO_PRICING with no log line, no counter, no API field, no UI badge. test/pricing.test.js:265-271 tests the miss mechanism but asserts no signal, so the silence is codified.

A $0 model is indistinguishable from a genuinely free model on every surface. The maintenance cost is already visible: curated-overrides.json carries hand-written pins for claude-fable-5, claude-mythos-5, and claude-opus-4-8 precisely because each new frontier model starts life as a silent miss.

3. Fuzzy tiers assign plausible-but-wrong prices, also silently

• Curated fuzzy over-capture — curated-overrides.json maps {"match": "kimi", "ref": "kimi-k2.5"} and {"match": "composer", "ref": "composer-1"}. A future kimi-k3 falls past exact/alias and lands on the k2.5 price. The file already needed a kimi-k2.6 exact entry to escape that net, which is evidence the net catches real models.
• LiteLLM reverse substring — src/lib/pricing/matcher.js:192-203 matches longest-key-first with lower.includes(keyLower). A hypothetical gpt-5.5-preview contains the literal key gpt-5 and would price at gpt-5 rates. Minor-version successors are exactly the models most likely to launch at a different price.

Silence is defensible for step 5 (suffix-strip: -high-fast variants genuinely share base pricing). It is not defensible for steps 4, 5b, and 6, where a mis-hit is invisible forever.

Fix (one change closes all three)

The hard part is already done — lookupPricing computes the resolution tier at the exact line where getModelPricing discards it.

1. Thread result.source through getModelPricing (parallel getModelPricingMeta(), or attach a non-enumerable tier).
2. Record loadedAt on load; when a lookup is about to miss or Date.now() - loadedAt > ttl, fire one background reload() (reuse loadLitellmData, clear the negative cache on completion). Makes "a new model appeared" self-healing within one TTL instead of "until the owner happens to notice $0".
3. Surface the signal where a human sees it: one console.warn per unknown model per process, unpriced_models and fuzzy_priced_models arrays in the stats API response, and a dashboard badge on any model row whose tier is miss or *:fuzzy.

Optionally, demote step 6 to require a family match (first-token equality) before accepting a substring hit.

Acceptance

• A model missing at startup but present upstream is priced correctly within one TTL, without a restart
• Reload is single-flight and does not block the request path
• Negative cache cleared on reload
• Unknown models appear in the API response and are visibly badged in the dashboard
• Fuzzy-resolved models are distinguishable from exact matches
• Tests: reload-on-miss, no thundering herd, tier propagation, badge rendering

Sizing estimate ~150 lines plus tests. Deliberately a separate PR from the small hardening work, because it touches the pricing path that just had a live incident and deserves an isolated diff.

Found during a Fable-model QA review of 0.39.40.

[pitimon]

Closed by #92, shipped in 0.39.41 — with a follow-up in #93/#94.

What was wrong: ensurePricingLoaded() returned early once loaded, with no loadedAt and no reload path. A running process never reloaded pricing. The 24h TTL only ever chose which snapshot to load at startup, and the dashboard LaunchAgent stays up for days. claude-opus-5 billed $0 for 21 hours that way — silently understating spend on precisely the model you just switched to, the expensive one.

Worth correcting the record: the original diagnosis here was "stale cache file, delete it". That was wrong. Deleting the cache was incidental — the restart is what fixed it, which is exactly why the bug could persist across a whole day of use.

The fix: a single-flight background reload triggered by a miss or an aged-out snapshot, with a 5-minute cooldown. Plus the observability that makes a wrong price visible: pricing_tier per model, and unpriced_models / fuzzy_priced_models / stale / refreshing / last_refresh_error on the model-breakdown pricing object. The dashboard's missing-pricing chip now reads the server's tier instead of inferring from cost <= 0, which could not tell an unpriced model from a genuinely free one — and a fuzzy-matched wrong price is never $0, so it was invisible entirely.

Three traps found while fixing it:

1. My first fix re-introduced the very bug it existed to prevent. loadInto installed whatever loadLitellmData fell back to. With the cache deleted and upstream down, a model priced at $5/$25 dropped to $0. Background reloads now pass requireUpstream: true and a failed refresh is discarded rather than installed.
2. ttlMs: 0 does not force a refetch. isFresh() compares Date.now() - stat.mtimeMs < ttlMs, and mtime carries sub-millisecond precision Date.now() lacks — a cache written moments ago compares as "written in the future" and counts as fresh. Needed an explicit forceRefresh flag.
3. One of my own tests passed for the wrong reason — it left the disk cache intact, so the "failed refresh" re-read the same data and proved nothing.

Follow-up: unpriced_models shipped reporting the literal placeholder "unknown" — see #93, fixed in #94. That is also why documenting this surface waited: writing it up first would have made the bug the contract.