feat(pricing): model-aware cost calculation for dashboard#310
Merged
Conversation
Replace the flat $0.00025/token cost estimate with a three-tier pricing pipeline that prefers SDK-reported cost, falls back to a per-model rate table, and only uses the flat estimate as a last resort. Surface the cost source through the JSON API and dashboard tooltips so users know how each value was computed. Backend: - New `internal/pricing` package with `Rates`, `LookupRates`, `Compute`, `CombineSources`, and a model rate table covering Claude 3.5/4.x, GPT-5 family, GPT-4o/4.1, and Gemini 2.5. Long-prefix model-ID matching with a word-boundary check to avoid e.g. `gpt-4` matching `gpt-40-future`. - Replace `estimateCost` in webapi store + storage_adapter with `pricing.Compute(models.AggregateUsageStats(...))`. - Add `costSource` field to `RunSummary` and `SummaryResponse`. - New tests in pricing package and handlers_test covering SDK preference, table path with cache tokens, multi-model aggregation, unknown-model fallback, and mixed-source surfacing through HTTP. Frontend: - New `InfoTooltip` component using lucide-react `Info`. - `costSourceTooltip()` helper centralizes wording. - Tooltips on Runs table Cost header (with per-row source), KPI "Avg Cost" card, and Run Detail Cost stat. - Optional `costSource` on `RunSummary` / `SummaryResponse` types. Docs: - New "Cost accuracy" section in dashboard guide explaining the three sources and the rate-table effective date. Closes #308 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Contributor
There was a problem hiding this comment.
Pull request overview
This PR replaces the dashboard’s flat per-token cost estimate with a model-aware pricing pipeline (SDK-reported cost → per-model rate table → flat fallback), and surfaces the chosen costSource end-to-end so the UI can disclose how the cost was derived.
Changes:
- Added
internal/pricingwith rate-table lookup + cost computation + source aggregation, and integrated it intointernal/webapisummaries. - Plumbed
costSourcethrough API types/tests and into the web UI (Runs table, KPI cards, run detail) via tooltip affordances. - Documented cost accuracy behavior in the dashboard guide and updated embedded web build output.
Show a summary per file
| File | Description |
|---|---|
| web/src/lib/format.ts | Adds costSourceTooltip() used by UI tooltips. |
| web/src/components/RunsTable.tsx | Adds cost header info icon + per-row cost source hover tooltip. |
| web/src/components/RunDetail.tsx | Adds cost source info icon in the run detail stat card; extends StatCard. |
| web/src/components/KPICards.tsx | Adds cost source info icon to “Avg Cost”; extends Card. |
| web/src/components/InfoTooltip.tsx | Introduces a reusable inline info icon component. |
| web/src/api/client.ts | Adds CostSource union + optional costSource fields in API types. |
| web/dist/index.html | Updates embedded build asset references. |
| site/src/content/docs/guides/dashboard.mdx | Documents “Cost accuracy” sources and caveats. |
| internal/webapi/types.go | Adds CostSource fields to API responses with docs. |
| internal/webapi/store.go | Replaces estimateCost() with pricing.Compute() using aggregated usage. |
| internal/webapi/storage_adapter.go | Aggregates/returns CostSource for summary responses. |
| internal/webapi/handlers_test.go | Adds/updates tests to assert costSource behavior/round-trip. |
| internal/pricing/pricing.go | New pricing engine: rate table + lookup + compute + CombineSources. |
| internal/pricing/pricing_test.go | Unit tests covering SDK/table/estimate paths + prefix-boundary + combining sources. |
Copilot's findings
- Files reviewed: 13/18 changed files
- Comments generated: 4
- gofmt internal/webapi/types.go (resolves CI lint + format check) - Clarify CostSource doc comments to call out legacy/empty case - Treat missing costSource as 'data unavailable' rather than estimate - InfoTooltip: use <button> for keyboard a11y; swap text-muted-foreground for text-zinc-500 to match existing repo palette Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
spboyer
pushed a commit
that referenced
this pull request
Jun 6, 2026
Resolve conflicts from #310 (cost-source pricing pipeline): - Combine Credits (premiumRequests) + CostSource fields on RunSummary/SummaryResponse - Aggregate both totalPremium and pricing.Compute in Summary() - StatCard/Card components support both title (Credits tooltip) and labelExtra (Cost InfoTooltip) - Strengthen Playwright assertions to check actual fixture values
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #308
Summary
Replaces the flat
$0.00025/tokendashboard cost estimate with a three-tier pricing pipeline:sdk— use the per-request cost the Copilot SDK already reports (ModelUsage.RequestCost), when available.table— price input / output / cache_read / cache_write tokens against a per-model rate table (Claude 3.5/4.x, GPT‑5 family, GPT‑4o/4.1, Gemini 2.5).estimate— fall back to the original flat$0.00025/tokenrate when the model is unknown.The chosen source is propagated as
costSourcethroughRunSummary/SummaryResponseso the dashboard can show an info tooltip explaining how the number was computed.What changed
Backend
internal/pricingpackage —Ratesstruct,LookupRates(longest-prefix match with a word-boundary check sogpt-4doesn't matchgpt-40-future),Compute(*UsageStats) → (dollars, source),CombineSources([]string), and a model rate table with effective date2025-01-01.internal/webapi/store.go,storage_adapter.go— replacedestimateCostwithpricing.Compute(models.AggregateUsageStats(...)).Summary()now collects per-run sources and combines them withCombineSources.internal/webapi/types.go— addedCostSource stringtoRunSummaryandSummaryResponse.internal/pricingunit tests (SDK path, table path with cache tokens, multi-model aggregation, unknown-model fallback, prefix-boundary safety,CombineSources). Newhandlers_testcases assertcostSourceround-trips through the HTTP layer and aggregates correctly when mixed.Frontend
web/src/components/InfoTooltip.tsx— small lucide-reactInfoicon with a nativetitletooltip.web/src/lib/format.ts—costSourceTooltip(source?)helper centralizes the four wordings (sdk,table,estimate,mixed).web/src/api/client.ts— added optionalcostSource?: CostSourcetoRunSummaryandSummaryResponse.RunsTable— info icon in the Cost column header; per-rowtitlereveals the source per run.KPICards— info icon next to "Avg Cost" reflecting the aggregate source.RunDetail— info icon next to the Cost stat card.Docs
site/src/content/docs/guides/dashboard.mdx— new "Cost accuracy" section explaining the three sources, when each applies, the rate-table effective date, and a note that the table is best-effort.Validation
go test ./...— all packages pass (internal/pricing,internal/webapiincluding new costSource tests).go vet ./...— clean.make lintreportsgolangci-lint not installed, skipping...on this host; CI will run it.cd web && npx tsc -b && npm run build— clean.cd web && npx playwright test --project=chromium— 52/52 passed (screenshots regenerated to include the new info icons).Design notes / follow-ups
LiveView,CompareView, andTrendsPageshow per-run costs that already benefit from the new pricing pipeline (their values come from the sameRunSummary). I scoped the tooltip wiring to the three primary surfaces called out in the issue to keep the diff focused; happy to extend if reviewers prefer.resultSummaryToRunSummaryinstorage_adapter.gostill hardcodesCost: 0for theResultSummarylegacy path — that struct doesn't carry token data, so there's nothing to price. Left unchanged;CostSourceis omitted (omitempty) for those rows.Co-authored-by: Copilot 223556219+Copilot@users.noreply.github.com