feat(usage): report Grok and Kimi in the usage overviews - #387
Merged
Conversation
Both CLIs already write usage to disk, so the overview can read them the same way it reads Claude and Codex — no new telemetry, no provider APIs. Grok records one `shell.turn.inference_done` per model round trip in a single process-wide log (`~/.grok/logs/unified.jsonl`) rather than per-session files, so lines from concurrent sessions interleave and the model — which the usage event does not carry — is carried forward per session id. `prompt_tokens` is inclusive of the cached portion, so uncached input is the difference. `grok-4.5` matches the LiteLLM table, so Grok turns price normally. Kimi records a `StatusUpdate` per served response in each session's `wire.jsonl`, whose `token_usage` splits cleanly onto the existing totals and whose `message_id` de-duplicates refreshed statuses. Nothing in Kimi's logs names a model, and the CLI's currently configured model says nothing about what served a turn weeks ago, so Kimi turns are recorded under an unpriceable sentinel: real token counts, and an honest "unpriced" share instead of a fabricated cost. Validated against this machine's real state: 573 Grok records over 19 sessions (80.2M tokens, all attributed to grok-4.5) and the one Kimi session that has a wire log, whose 17,729 tokens match Kimi's own context counter exactly. USAGE_CONTRACT_VERSION goes to 4 because the bucket vocabulary changed; an environment on an older server is excluded from the merge and reported as partial coverage rather than failing the page. Known limits, both upstream of us: - Grok's log is a single file that is not rotated, so history only reaches back as far as it happens to retain (~2.5 days on this machine) while the other providers reach back the full window. - Kimi only writes `wire.jsonl` for some sessions, so its coverage is partial. Co-authored-by: Patrick Roza <42661+patroza@users.noreply.github.com>
`model changed` leads a session in Grok's log, so the model map covers every turn in practice — 573/573 on this machine. But the log is one append-only file, and if it is ever rotated between a session's announcement and a later turn, that turn arrives with no model in the new file's map. Dropping it lost real tokens outright. Those turns are now recorded under a bare-provider sentinel that prices as unpriceable, matching how Kimi's unnamed model is handled: the token count stays whole and the cost stays honest. Neither bare name is itself a priced model in the rate table. Found by the gpt-5.6-sol adversarial review. Co-authored-by: Patrick Roza <42661+patroza@users.noreply.github.com>
…ontract Adding a provider literal is additive: an environment on an older server reports no grok/kimi buckets, and its payload still decodes here. Bumping USAGE_CONTRACT_VERSION instead moved that environment into staleEnvironments, which drops *all* of its usage — Claude and Codex included — until every environment in a fleet has been upgraded. That trades a real undercount for a signal about providers the lagging environment may not even run, and it does not help an older client either: a v4 payload fails to decode on "grok" before any version logic runs. Also spans the day-breakdown empty state across the real column count. It was a literal 5 against Day + 2 providers + Total + Tokens, so going to four providers left the "No activity" cell short of the row. Adds the Kimi turn-accounting tests the review asked for: distinct responses sum because each re-bills its whole context, and a refreshed status carries the same message_id so it collapses. Found by the grok-4.5 adversarial review. Co-authored-by: Patrick Roza <42661+patroza@users.noreply.github.com>
omegent-app
Bot
force-pushed
the
feat/usage-grok-kimi
branch
from
August 9, 2026 19:32
f9db00a to
09a7a6d
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds Grok and Kimi to the Usage overviews, web and mobile. OpenCode is deliberately left
out — see the last section.
Both CLIs already write usage to disk, so this reads them exactly the way the page already reads
Claude and Codex. No new telemetry, no provider APIs, no changes to how anyone runs a turn.
Where the numbers come from
~/.grok/logs/unified.jsonl— one process-wide log, all sessions interleaved~/.kimi/sessions/<hash>/<session>/wire.jsonl— per sessionshell.turn.inference_done, one per model round tripStatusUpdate, one per served responsemodel changedgrok-4.5matches the LiteLLM table exactlyTwo details that would be double-counting bugs if read casually:
prompt_tokensis inclusive ofcached_prompt_tokens, so uncached input is thedifference, not the raw field. (Same trap Codex's
input_tokenshas, handled the same way.)per-session rollout allows — a single variable would credit one session's turns to whichever
session switched model last. It is carried per session id.
Kimi is deliberately unpriced. Nothing in Kimi's logs names a model, and the CLI's currently
configured model says nothing about what served a turn three weeks ago. Rather than attribute — and
therefore price — turns against a guess, they are recorded under a sentinel that
usagePricingalready treats as unpriceable. Kimi shows real token counts and an honest
unpricedshare instead ofa fabricated cost. The page already surfaces that share in its cost-quality panel.
Validated against real data, not just fixtures
Ran the actual reader over this machine's actual state:
grok-4.5. Modelcarry-forward resolved 573/573 —
model changedis emitted at session start, not only on aswitch, so no record goes unattributed.
Kimi's own
context.jsonlcounter (17600context +129output), which is a real cross-check onthe token math rather than a fixture agreeing with itself.
Known limits — both upstream of us, neither hidden
(
2026-08-07→2026-08-09) while Grok sessions go back to July 15. So a 30-day view will showthe full window for Claude/Codex and only the last few days for Grok. Making this complete means T3
persisting its own snapshots over time — a real feature, deliberately not smuggled in here.
wire.jsonlfor only some sessions (1 of 4 on this machine), so its coverage ispartial by construction.
Why not OpenCode
Agreeing with your instinct, and there is a concrete blocker behind it: the scanner only walks
*.jsonl, and OpenCode stores per-message JSON files rather than JSONL rollouts. Supporting it meansa different traversal, not another parser.
Contract version: deliberately not bumped
I bumped
USAGE_CONTRACT_VERSION3 → 4 first, and the review talked me out of it. Adding a providerliteral is additive — an environment on an older server reports no grok/kimi buckets and its payload
still decodes. Bumping would instead move that environment into
staleEnvironments, dropping allof its usage including Claude and Codex, until every environment in a fleet was upgraded. It also
does not protect an older client, which fails to decode
"grok"before any version check runs. Sothe bump bought nothing and cost a real undercount; it is reverted, with the reasoning recorded at
the constant.
Adversarial review
Both reviewers were given the real record shapes as ground truth and told to attack double-counting,
the cache/fingerprint model, session attribution, the sentinel, the contract version, exhaustiveness,
and test integrity. Neither substantiated a HIGH. Four findings were worth acting on; all are
fixed.
"grok"fails to decode before any version check runs. It bought nothing and cost a real undercount across a mixed-version fleetmessage_idso it collapsescolSpan={5}against a row that is now 7 columnsPROVIDER_ORDER.length + 3so it cannot drift againcontext.jsonlfiles are walked tooWhat both independently confirmed, having checked it against the real logs rather than my
description of them: the Grok inclusive-cache arithmetic (gpt-5.6-sol verified a real progression —
prompt
75,234/cached58,752then prompt76,314/cached75,136— that only makes sense ifprompt_tokensincludes cached), the per-session model map under interleaving, that a re-parsereplaces rather than accumulates cache entries, that multiple environments scanning one
~/.grokde-duplicate to a single owner via the host+provider+path+volume fingerprint, that the
kimisentinel cannot collide with a real LiteLLM model through slash-normalisation, and that the chart
test kept its regression value — a Claude value regressed to a cumulative
30still fails againstthe expected
20.Verification
Record<UsageProviderKind, …>maps (labels, colours, order, marks, web and mobile) were allupdated rather than silently defaulting.
only failure is the pre-existing
CodexTextGenerationlaunch-args one, which reproduces onunmerged
fork/dev.UsageProviderChart.test.ts's band assertion previously hard-coded a two-element array, so itbroke on any new provider. Rewritten to assert what it was actually protecting — that band values
are absolute rather than cumulative stack offsets — without pinning the provider count.
Co-authored by @patroza
opened by Patrick Roza in chat thread Discord · Discord · T3