Skip to content

fix: compute costUsd from model rates when the provider reports none (#67)#73

Open
stealthwhizz wants to merge 1 commit into
open-gitagent:mainfrom
stealthwhizz:fix/cost-usd-calculation
Open

fix: compute costUsd from model rates when the provider reports none (#67)#73
stealthwhizz wants to merge 1 commit into
open-gitagent:mainfrom
stealthwhizz:fix/cost-usd-calculation

Conversation

@stealthwhizz

Copy link
Copy Markdown
Collaborator

Problem

costs() reports costUsd: 0 even when tokens were consumed and billed. sdk.ts mapped cost straight from the provider:

costUsd: msg.usage.cost?.total ?? 0,

But msg.usage.cost.total is only populated when pi-ai's calculateCost runs, which:

So the 0 was read verbatim, and callers couldn't tell "this was free" from "we don't know the cost."

Root-cause split

Bug Where Fixable in gitagent?
calculateCost emits 0 for unknown model IDs pi-ai ❌ upstream
openai-completions never calls calculateCost pi-ai ❌ upstream
EMPTY_USAGE zeros usage on the error path pi-agent-core ❌ upstream
sdk.ts reports that 0 with no fallback / no signal gitagent ✅ this PR

Fix (gitagent side)

  • computeCostUsd(usage, rates) in cost-tracker.ts — the same per-million-token formula pi-ai uses ((rate / 1e6) * tokens). Returns null when tokens were used but the model has no pricing, so "unknown" is distinguishable from 0.
  • sdk.ts — prefer the provider's cost.total when it's present and non-zero; otherwise recompute from loaded.model.cost (known models carry real rates, e.g. gpt-4o = $2.5/$10 per 1M). Pass a costResolved flag to the tracker.
  • SessionCosts.costDataAvailable — flips to false when any request consumed tokens that couldn't be priced, so costs() consumers can surface "cost unavailable" instead of a misleading $0.
  • Exported computeCostUsd + the new types.

Result

  • Known models that hit the openai-completions path (or otherwise arrive with no provider cost) now get a correctly computed costUsd from their own rate table.
  • Custom/zero-rate models report costDataAvailable: false instead of silently claiming $0.
  • Token counts are untouched; this only changes how cost is derived and reported.

Tests

test/cost-compute.test.ts — pricing math (input/output/cache), null for unpriceable-with-tokens, 0 for no-tokens-no-rates, partial-rate models; plus costDataAvailable default/flip/reset. Full suite green.

Upstream follow-ups (out of scope here)

Worth filing against pi-ai / pi-agent-core: have calculateCost return null (not 0) for missing pricing, call it on the openai-completions path, and document/​accumulate EMPTY_USAGE on the error path.

Closes #67

costs() reported costUsd: 0 even when tokens were billed. The provider only
supplies msg.usage.cost.total when pi-ai's calculateCost runs — which it does
not on the openai-completions path, and cannot for custom/unregistered models
that carry no price table (issue open-gitagent#67). sdk.ts read that 0 verbatim.

- cost-tracker: add computeCostUsd(usage, rates) — the same per-million-token
  formula pi-ai uses — returning null when tokens were used but no pricing
  exists (cost unknown, not zero)
- sdk: prefer the provider's cost when present, otherwise recompute from
  loaded.model.cost; feed a costResolved flag into the tracker
- cost-tracker: add SessionCosts.costDataAvailable so callers can tell a real
  $0 apart from "pricing unavailable"
- export computeCostUsd + the new types

Note: the underlying gaps live in pi-ai (calculateCost emits 0 for unknown
models; openai-completions never calls it) and pi-agent-core (EMPTY_USAGE on
the error path); those need upstream fixes. This addresses the gitagent-side
symptom and makes the unknown-cost case explicit.

Covered by unit tests for computeCostUsd and the costDataAvailable flag.

Closes open-gitagent#67
Copilot AI review requested due to automatic review settings July 7, 2026 11:50

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

@shreyas-lyzr shreyas-lyzr left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Clean fix for the misleading $0 cost reporting. The three-way outcome — provider cost wins when non-zero, fallback computation from model rates, and null/costDataAvailable=false for genuinely unresolvable cases — is the right design. The logic in computeCostUsd correctly mirrors pi-ai's formula, and the test coverage hits all the meaningful branches including the partial-rate and zero-token edges.

One suggestion below: null-guard loaded.model.cost in sdk.ts. It's safe today because both createCustomModel and getModel always produce a cost object, but a future model source that omits it would throw at runtime rather than fail gracefully.

Comment thread src/sdk.ts
if (providerCost > 0) {
costUsd = providerCost;
} else {
const computed = computeCostUsd(tokens, loaded.model.cost);

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

loaded.model.cost is always set today (createCustomModel hardcodes it, getModel from pi-ai includes it), so this won't throw in practice. Worth adding a null-fallback anyway so a future model source that omits the field degrades gracefully rather than crashing:

Suggested change
const computed = computeCostUsd(tokens, loaded.model.cost);
const computed = computeCostUsd(tokens, loaded.model.cost ?? { input: 0, output: 0, cacheRead: 0, cacheWrite: 0 });

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

bug: costs() always returns 0 — token usage tracked but costUsd never calculated correctly

3 participants