Skip to content

Read model prices and context windows from LiteLLM instead of hand-kept tables - #40

Merged
calvintvu merged 8 commits into
mainfrom
cvu/auto-pull-model-data
Aug 10, 2026
Merged

Read model prices and context windows from LiteLLM instead of hand-kept tables#40
calvintvu merged 8 commits into
mainfrom
cvu/auto-pull-model-data

Conversation

@calvintvu

@calvintvu calvintvu commented Aug 5, 2026

Copy link
Copy Markdown
Contributor

Context

we dont want to manually sync model information, we'll follow ccusage pattern and pull info from litellm

Changes

  • The collector reads prices and context windows from one embedded table, generated from LiteLLM (MIT). A new model release needs no code change.
  • The collector computes each cost and sends it with the session. The frontend holds no pricing table.
  • make models rewrites the table. The Models workflow runs it daily and opens a pull request, so a person reads every price change before it merges.
  • The workflow compares the model list before and after each refresh. The pull request names every model added and dropped, and a dropped model turns the run red.
  • Three prices now match LiteLLM: gpt-5.6-terra input is $2.00 per MTok, not $2.50. gpt-5.6-luna input is $0.20, not $1.00. The gpt-5.5-pro cache read is $3.00, not $30.00.
  • The context window of claude-sonnet-5 is 1,000,000, not 967,000. The context gauge shows a smaller percentage for those sessions.
  • LiteLLM carries no entry for claude-mythos-5 or gpt-5.5-cyber. Sessions on those models show the unpriced-model warning and no cost.
  • A Codex session takes its context window from the table when the transcript reports none.

Test

  • make check - passed. The collector carries no test files, so make test runs nothing.
  • npm run format:check, npx tsc -b, npm run lint, npm test (8 tests), npm run build - all passed
  • Manual: ran the dev binary and read /api/sessions. 191 sessions, total cost $1166.56, no unpriced model, subagent costs present, claude-sonnet-5 window 1,000,000.
  • Manual: ran the workflow comparison locally. A real refresh reported no change. A table with claude-opus-5 deleted reported the drop and exited 1.
  • The last commit deletes o4-mini-2025-04-16 from the table on purpose, to watch the workflow report a model as added. Revert it before merge.

Screenshots

Screenshot 2026-08-10 at 13 23 03 Screenshot 2026-08-10 at 13 23 23

@calvintvu calvintvu added github_actions Pull requests that update GitHub Actions code enhancement New feature or request labels Aug 5, 2026
@calvintvu

Copy link
Copy Markdown
Contributor Author

6e190b8 removes model o4-mini-2025-04-16 from models json, upon merge we can test workflow to see if we updated the models json correctly

@luhe19001 luhe19001 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This backend migration regresses cost and context-window handling for model IDs that were supported by the previous pricing table but are absent from the generated LiteLLM subset. For sessions using previously supported IDs such as claude-mythos-5, gpt-5.5-pro, or gpt-5.5-cyber, the lookup now fails after only removing a date suffix, so usage becomes unpriced and contributes $0. claude-mythos-5 also loses its context window. Please preserve local aliases or fallback entries for these IDs when moving pricing to the backend.

Comment thread collector/internal/session/models.go
@calvintvu
calvintvu force-pushed the cvu/auto-pull-model-data branch from 6e190b8 to 1d22057 Compare August 5, 2026 23:30
@calvintvu
calvintvu requested a review from luhe19001 August 6, 2026 00:03
Comment thread collector/internal/session/models.go Outdated
@calvintvu
calvintvu force-pushed the cvu/auto-pull-model-data branch from c633c22 to e61909b Compare August 6, 2026 23:06
@calvintvu
calvintvu requested a review from luhe19001 August 7, 2026 20:32
@calvintvu
calvintvu force-pushed the cvu/auto-pull-model-data branch from b2dc78b to 3f5832c Compare August 7, 2026 20:32
@calvintvu
calvintvu force-pushed the cvu/auto-pull-model-data branch from 3f5832c to 78bcbc5 Compare August 10, 2026 17:44
@calvintvu
calvintvu merged commit cc109b2 into main Aug 10, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request github_actions Pull requests that update GitHub Actions code

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants