fix: compute costUsd from model rates when the provider reports none (#67)#73
fix: compute costUsd from model rates when the provider reports none (#67)#73stealthwhizz wants to merge 1 commit into
Conversation
costs() reported costUsd: 0 even when tokens were billed. The provider only supplies msg.usage.cost.total when pi-ai's calculateCost runs — which it does not on the openai-completions path, and cannot for custom/unregistered models that carry no price table (issue open-gitagent#67). sdk.ts read that 0 verbatim. - cost-tracker: add computeCostUsd(usage, rates) — the same per-million-token formula pi-ai uses — returning null when tokens were used but no pricing exists (cost unknown, not zero) - sdk: prefer the provider's cost when present, otherwise recompute from loaded.model.cost; feed a costResolved flag into the tracker - cost-tracker: add SessionCosts.costDataAvailable so callers can tell a real $0 apart from "pricing unavailable" - export computeCostUsd + the new types Note: the underlying gaps live in pi-ai (calculateCost emits 0 for unknown models; openai-completions never calls it) and pi-agent-core (EMPTY_USAGE on the error path); those need upstream fixes. This addresses the gitagent-side symptom and makes the unknown-cost case explicit. Covered by unit tests for computeCostUsd and the costDataAvailable flag. Closes open-gitagent#67
shreyas-lyzr
left a comment
There was a problem hiding this comment.
Clean fix for the misleading $0 cost reporting. The three-way outcome — provider cost wins when non-zero, fallback computation from model rates, and null/costDataAvailable=false for genuinely unresolvable cases — is the right design. The logic in computeCostUsd correctly mirrors pi-ai's formula, and the test coverage hits all the meaningful branches including the partial-rate and zero-token edges.
One suggestion below: null-guard loaded.model.cost in sdk.ts. It's safe today because both createCustomModel and getModel always produce a cost object, but a future model source that omits it would throw at runtime rather than fail gracefully.
| if (providerCost > 0) { | ||
| costUsd = providerCost; | ||
| } else { | ||
| const computed = computeCostUsd(tokens, loaded.model.cost); |
There was a problem hiding this comment.
loaded.model.cost is always set today (createCustomModel hardcodes it, getModel from pi-ai includes it), so this won't throw in practice. Worth adding a null-fallback anyway so a future model source that omits the field degrades gracefully rather than crashing:
| const computed = computeCostUsd(tokens, loaded.model.cost); | |
| const computed = computeCostUsd(tokens, loaded.model.cost ?? { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 }); |
Problem
costs()reportscostUsd: 0even when tokens were consumed and billed.sdk.tsmapped cost straight from the provider:But
msg.usage.cost.totalis only populated when pi-ai'scalculateCostruns, which:openai-completionspath (Bug 3 in bug: costs() always returns 0 — token usage tracked but costUsd never calculated correctly #67), andSo the 0 was read verbatim, and callers couldn't tell "this was free" from "we don't know the cost."
Root-cause split
calculateCostemits 0 for unknown model IDsopenai-completionsnever callscalculateCostEMPTY_USAGEzeros usage on the error pathsdk.tsreports that 0 with no fallback / no signalFix (gitagent side)
computeCostUsd(usage, rates)incost-tracker.ts— the same per-million-token formula pi-ai uses ((rate / 1e6) * tokens). Returnsnullwhen tokens were used but the model has no pricing, so "unknown" is distinguishable from0.sdk.ts— prefer the provider'scost.totalwhen it's present and non-zero; otherwise recompute fromloaded.model.cost(known models carry real rates, e.g. gpt-4o = $2.5/$10 per 1M). Pass acostResolvedflag to the tracker.SessionCosts.costDataAvailable— flips tofalsewhen any request consumed tokens that couldn't be priced, socosts()consumers can surface "cost unavailable" instead of a misleading $0.computeCostUsd+ the new types.Result
openai-completionspath (or otherwise arrive with no provider cost) now get a correctly computedcostUsdfrom their own rate table.costDataAvailable: falseinstead of silently claiming $0.Tests
test/cost-compute.test.ts— pricing math (input/output/cache),nullfor unpriceable-with-tokens,0for no-tokens-no-rates, partial-rate models; pluscostDataAvailabledefault/flip/reset. Full suite green.Upstream follow-ups (out of scope here)
Worth filing against pi-ai / pi-agent-core: have
calculateCostreturnnull(not 0) for missing pricing, call it on theopenai-completionspath, and document/accumulateEMPTY_USAGEon the error path.Closes #67