feat(playground): show per-message cost using models.dev pricing#2255
Merged
Conversation
Adds a per-message USD cost next to the existing token breakdown on assistant messages. Cost is computed from models.dev pricing (USD/1M tokens), fetched once and cached on disk for 24h in the main process. Cached input tokens use the model's cache_read rate when available. Providers/models with no pricing entry (local providers, unknown models) render exactly as before. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Contributor
There was a problem hiding this comment.
Pull request overview
Adds per-message USD cost estimation to Playground assistant messages by fetching model token pricing from models.dev in the main process (disk-cached for 24h), exposing it via IPC, caching it in the renderer via React Query, and computing a cost breakdown from existing token usage metrics.
Changes:
- Added main-process pricing fetch + 24h disk cache and an IPC endpoint (
chat:get-model-pricing) to expose a provider/model pricing map. - Added renderer React Query hook to retrieve pricing and a pure
calculateCost/formatUsdutility with unit tests. - Updated
TokenUsageUI to render inline cost and a tooltip breakdown when pricing is available (and added component tests).
Reviewed changes
Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.
Show a summary per file
| File | Description |
|---|---|
| renderer/src/features/chat/lib/calculate-cost.ts | Pure cost calculation + USD formatting helpers. |
| renderer/src/features/chat/lib/tests/calculate-cost.test.ts | Unit tests for cost calculation and formatting. |
| renderer/src/features/chat/hooks/use-model-pricing.ts | React Query hook to fetch/cache pricing map from IPC. |
| renderer/src/features/chat/components/chat-message/token-usage.tsx | UI: inline cost display + tooltip cost breakdown using pricing map. |
| renderer/src/features/chat/components/chat-message/assistant-message.tsx | Passes model into TokenUsage so pricing lookup can be model-specific. |
| renderer/src/features/chat/components/chat-message/tests/token-usage.test.tsx | Component tests for cost rendering behavior. |
| preload/src/api/chat.ts | Adds getModelPricing() to the preload chat API + types. |
| main/src/ipc-handlers/chat/pricing.ts | Registers IPC handler for chat:get-model-pricing. |
| main/src/ipc-handlers/chat/index.ts | Wires pricing IPC handler registration into chat IPC registration. |
| main/src/ipc-handlers/chat/tests/index.test.ts | Updates registration test to include pricing handler. |
| main/src/chat/pricing.ts | Implements models.dev fetch, extraction, in-memory cache, and disk cache w/ TTL + offline fallback. |
- useModelPricing now only enables its React Query fetch when a remote provider+model pair is known (skips ollama/lmstudio and the empty state). Avoids a no-op IPC + models.dev fetch for local-only users. - Inline cost rendering no longer strips the dollar sign from the formatter output. The "<$0.01" case now reads correctly inline instead of "<0.01". - Add unit tests for the main-process pricing module: cold-start fetch + write, warm disk cache within TTL, stale cache returns immediately + background refresh, offline with no disk cache, and non-OK upstream response. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
peppescg
approved these changes
May 14, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a per-message USD cost next to the existing token breakdown on assistant messages in the playground chat. Pricing comes from models.dev (USD per 1M tokens), fetched once and cached on disk for 24h in the main process.
100 → 50 = 150 • $ 0.0012next to the existing token totals.Costsection with Input, Cached, Output, Total breakdown.cache_readrate when models.dev exposes it (e.g. Claude Sonnet 4.5, Haiku 4.5); otherwise treated as regular input.ollama,lmstudio) and unknown models render exactly as they did before.costentries (slash-prefixed IDs already match).How it works
main/src/chat/pricing.tsfetcheshttps://models.dev/api.json, caches{ fetchedAt, data }touserData/models-dev-cache.json, refreshes in the background when older than 24h, and falls back to the last cached copy when offline.chat:get-model-pricingreturns the extractedRecord<providerId, Record<modelId, ModelCost>>map.useModelPricingcaches the map in React Query (staleTime: 24h).calculateCost(usage, pricing)is a pure function — easy to unit-test, no React, no DOM.What deliberately does NOT change
MessageMetadata— cost is derived on render from existingtotalUsage+model+ the cached pricing map.ollama/lmstudio— they naturally have no pricing entry and fall through to "no cost rendered."Test plan
pnpm run lintpnpm run type-checkpnpm run test:nonInteractive(212 files, 2402 tests, all green)$x.xxxxand tooltip breakdown appear after the response completes.🤖 Generated with Claude Code