Skip to content

feat(usage): report Grok and Kimi in the usage overviews - #387

Merged
patroza merged 3 commits into
fork/devfrom
feat/usage-grok-kimi
Aug 10, 2026
Merged

feat(usage): report Grok and Kimi in the usage overviews#387
patroza merged 3 commits into
fork/devfrom
feat/usage-grok-kimi

Conversation

@omegent-app

@omegent-app omegent-app Bot commented Aug 9, 2026

Copy link
Copy Markdown

Adds Grok and Kimi to the Usage overviews, web and mobile. OpenCode is deliberately left
out — see the last section.

Both CLIs already write usage to disk, so this reads them exactly the way the page already reads
Claude and Codex. No new telemetry, no provider APIs, no changes to how anyone runs a turn.

Where the numbers come from

Grok Kimi
File ~/.grok/logs/unified.jsonlone process-wide log, all sessions interleaved ~/.kimi/sessions/<hash>/<session>/wire.jsonl — per session
Usage record shell.turn.inference_done, one per model round trip StatusUpdate, one per served response
Model not on the usage record — carried forward per session id from model changed named nowhere at all
Cost priced: grok-4.5 matches the LiteLLM table exactly unpriced, on purpose

Two details that would be double-counting bugs if read casually:

  • Grok's prompt_tokens is inclusive of cached_prompt_tokens, so uncached input is the
    difference, not the raw field. (Same trap Codex's input_tokens has, handled the same way.)
  • Grok's log is process-wide, so the model cannot be carried forward in a scalar the way Codex's
    per-session rollout allows — a single variable would credit one session's turns to whichever
    session switched model last. It is carried per session id.

Kimi is deliberately unpriced. Nothing in Kimi's logs names a model, and the CLI's currently
configured
model says nothing about what served a turn three weeks ago. Rather than attribute — and
therefore price — turns against a guess, they are recorded under a sentinel that usagePricing
already treats as unpriceable. Kimi shows real token counts and an honest unpriced share instead of
a fabricated cost. The page already surfaces that share in its cost-quality panel.

Validated against real data, not just fixtures

Ran the actual reader over this machine's actual state:

  • Grok: 573 records across 19 sessions, 80.2M tokens, every one attributed to grok-4.5. Model
    carry-forward resolved 573/573model changed is emitted at session start, not only on a
    switch, so no record goes unattributed.
  • Kimi: the one session that has a wire log — 17,729 tokens. That figure independently matches
    Kimi's own context.jsonl counter (17600 context + 129 output), which is a real cross-check on
    the token math rather than a fixture agreeing with itself.

Known limits — both upstream of us, neither hidden

  • Grok's log is a single file that is never rotated. On this machine it retains ~2.5 days
    (2026-08-072026-08-09) while Grok sessions go back to July 15. So a 30-day view will show
    the full window for Claude/Codex and only the last few days for Grok. Making this complete means T3
    persisting its own snapshots over time — a real feature, deliberately not smuggled in here.
  • Kimi writes wire.jsonl for only some sessions (1 of 4 on this machine), so its coverage is
    partial by construction.

Why not OpenCode

Agreeing with your instinct, and there is a concrete blocker behind it: the scanner only walks
*.jsonl, and OpenCode stores per-message JSON files rather than JSONL rollouts. Supporting it means
a different traversal, not another parser.

Contract version: deliberately not bumped

I bumped USAGE_CONTRACT_VERSION 3 → 4 first, and the review talked me out of it. Adding a provider
literal is additive — an environment on an older server reports no grok/kimi buckets and its payload
still decodes. Bumping would instead move that environment into staleEnvironments, dropping all
of its usage including Claude and Codex, until every environment in a fleet was upgraded. It also
does not protect an older client, which fails to decode "grok" before any version check runs. So
the bump bought nothing and cost a real undercount; it is reverted, with the reasoning recorded at
the constant.

Adversarial review

Both reviewers were given the real record shapes as ground truth and told to attack double-counting,
the cache/fingerprint model, session attribution, the sentinel, the contract version, exhaustiveness,
and test integrity. Neither substantiated a HIGH. Four findings were worth acting on; all are
fixed.

Finding Disposition
grok-4.5, MEDIUM — bumping the contract version drops a lagging environment's entire usage, Claude and Codex included, not just the providers it cannot report Fixed — bump reverted. The reviewer is right that an added literal is additive: an old environment reports no grok/kimi buckets and its payload still decodes. And the bump does not protect old clients either, since a payload containing "grok" fails to decode before any version check runs. It bought nothing and cost a real undercount across a mixed-version fleet
gpt-5.6-sol, LOW — a Grok turn whose session never announced a model in that file is dropped, so a rotated log silently loses tokens Fixed. Those turns now record under an unpriceable bare-provider sentinel, matching Kimi: the token count stays whole, the cost stays honest. (grok-4.5 raised the same thing as its third follow-up)
grok-4.5, MEDIUM — Kimi's multi-turn and refresh semantics were asserted nowhere Fixed. Added tests locking both: distinct responses sum because each re-bills its whole context, and a refreshed status carries the same message_id so it collapses
grok-4.5, LOW — the day-breakdown empty state spans colSpan={5} against a row that is now 7 columns Fixed. Derived from PROVIDER_ORDER.length + 3 so it cannot drift again
gpt-5.6-sol, LOW — Grok's single append-only log defeats the bounded scan cache: it is always fresh, so every append re-parses the whole file Acknowledged, not fixed. Real, and it is the same property behind the retention limit above. Incremental byte-offset parsing is the fix and is a feature in its own right. Worth noting the file is self-limiting in practice — Grok truncates it, which is why it only holds ~2.5 days
grok-4.5, LOW — Kimi's context.jsonl files are walked too Harmless: the substring gate skips them; only the skipped-file counter sees them

What both independently confirmed, having checked it against the real logs rather than my
description of them: the Grok inclusive-cache arithmetic (gpt-5.6-sol verified a real progression —
prompt 75,234/cached 58,752 then prompt 76,314/cached 75,136 — that only makes sense if
prompt_tokens includes cached), the per-session model map under interleaving, that a re-parse
replaces rather than accumulates cache entries, that multiple environments scanning one ~/.grok
de-duplicate to a single owner via the host+provider+path+volume fingerprint, that the kimi
sentinel cannot collide with a real LiteLLM model through slash-normalisation, and that the chart
test kept its regression value — a Claude value regressed to a cumulative 30 still fails against
the expected 20.

Verification

  • Recursive typecheck clean across all 18 packages — which is also what proves the exhaustive
    Record<UsageProviderKind, …> maps (labels, colours, order, marks, web and mobile) were all
    updated rather than silently defaulting.
  • 2,357 tests pass, including 14 new parser tests written against the real record shapes. The
    only failure is the pre-existing CodexTextGeneration launch-args one, which reproduces on
    unmerged fork/dev.
  • UsageProviderChart.test.ts's band assertion previously hard-coded a two-element array, so it
    broke on any new provider. Rewritten to assert what it was actually protecting — that band values
    are absolute rather than cumulative stack offsets — without pinning the provider count.

Co-authored by @patroza

opened by Patrick Roza in chat thread Discord · Discord · T3

omegent-app Bot and others added 3 commits August 9, 2026 19:29
Both CLIs already write usage to disk, so the overview can read them the same
way it reads Claude and Codex — no new telemetry, no provider APIs.

Grok records one `shell.turn.inference_done` per model round trip in a single
process-wide log (`~/.grok/logs/unified.jsonl`) rather than per-session files,
so lines from concurrent sessions interleave and the model — which the usage
event does not carry — is carried forward per session id. `prompt_tokens` is
inclusive of the cached portion, so uncached input is the difference.
`grok-4.5` matches the LiteLLM table, so Grok turns price normally.

Kimi records a `StatusUpdate` per served response in each session's
`wire.jsonl`, whose `token_usage` splits cleanly onto the existing totals and
whose `message_id` de-duplicates refreshed statuses. Nothing in Kimi's logs
names a model, and the CLI's currently configured model says nothing about
what served a turn weeks ago, so Kimi turns are recorded under an unpriceable
sentinel: real token counts, and an honest "unpriced" share instead of a
fabricated cost.

Validated against this machine's real state: 573 Grok records over 19 sessions
(80.2M tokens, all attributed to grok-4.5) and the one Kimi session that has a
wire log, whose 17,729 tokens match Kimi's own context counter exactly.

USAGE_CONTRACT_VERSION goes to 4 because the bucket vocabulary changed; an
environment on an older server is excluded from the merge and reported as
partial coverage rather than failing the page.

Known limits, both upstream of us:
- Grok's log is a single file that is not rotated, so history only reaches
  back as far as it happens to retain (~2.5 days on this machine) while the
  other providers reach back the full window.
- Kimi only writes `wire.jsonl` for some sessions, so its coverage is partial.

Co-authored-by: Patrick Roza <42661+patroza@users.noreply.github.com>
`model changed` leads a session in Grok's log, so the model map covers every
turn in practice — 573/573 on this machine. But the log is one append-only
file, and if it is ever rotated between a session's announcement and a later
turn, that turn arrives with no model in the new file's map. Dropping it lost
real tokens outright.

Those turns are now recorded under a bare-provider sentinel that prices as
unpriceable, matching how Kimi's unnamed model is handled: the token count
stays whole and the cost stays honest. Neither bare name is itself a priced
model in the rate table.

Found by the gpt-5.6-sol adversarial review.

Co-authored-by: Patrick Roza <42661+patroza@users.noreply.github.com>
…ontract

Adding a provider literal is additive: an environment on an older server
reports no grok/kimi buckets, and its payload still decodes here. Bumping
USAGE_CONTRACT_VERSION instead moved that environment into staleEnvironments,
which drops *all* of its usage — Claude and Codex included — until every
environment in a fleet has been upgraded. That trades a real undercount for a
signal about providers the lagging environment may not even run, and it does
not help an older client either: a v4 payload fails to decode on "grok" before
any version logic runs.

Also spans the day-breakdown empty state across the real column count. It was
a literal 5 against Day + 2 providers + Total + Tokens, so going to four
providers left the "No activity" cell short of the row.

Adds the Kimi turn-accounting tests the review asked for: distinct responses
sum because each re-bills its whole context, and a refreshed status carries the
same message_id so it collapses.

Found by the grok-4.5 adversarial review.

Co-authored-by: Patrick Roza <42661+patroza@users.noreply.github.com>
@omegent-app
omegent-app Bot force-pushed the feat/usage-grok-kimi branch from f9db00a to 09a7a6d Compare August 9, 2026 19:32
@patroza
patroza merged commit 13a789e into fork/dev Aug 10, 2026
5 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant