Skip to content

True token accounting: count cache tokens, show minds their context economics #384

Description

@psamiton

Problem

The usage numbers Volute records are a small fraction of real token throughput:

  • The claude template reports only the final result's input_tokens + output_tokens to the daemon (templates/claude/src/lib/stream-consumer.ts:129-137). cache_read_input_tokens and cache_creation_input_tokens are excluded — they're summed separately into contextTokens for the compaction threshold (:54-59) and never reach the usage event.
  • For a long-running mind, cache reads/writes are most of the throughput: a 100k-context mind whose turn reports input_tokens: 4000 actually processed ~104k. Token budgets (turn-lifecycle.ts:257-261token-budget.ts recordUsage) and the dashboard both operate on the undercounted number, so budgets are ineffective exactly for the minds they're meant to bound.
  • Minds themselves have no way to understand their own context economics — what a cold vs warm turn costs, why compaction matters, what the 1h cache TTL means for idle gaps.

Implementation plan

1. Templates report full usage.

  • claude: include cache_read_input_tokens, cache_creation_input_tokens (and the per-TTL breakdown cache_creation.ephemeral_1h_input_tokens / ephemeral_5m_input_tokens when present) in the usage broadcast/emit in stream-consumer.ts.
  • codex: cached_input_tokens; pi: cacheRead/cacheWrite — the parsers already exist in templates/_base/src/lib/context-breakdown.ts; surface the same fields in their usage events.

2. Daemon records the full breakdown (turn-lifecycle.ts usage handling): persist all fields in turn metadata rather than collapsing to two numbers.

3. Budget semantics — decide in the PR (options, roughly in order of preference):

  • (a) Cost-weighted units: budget counts input + output×(out/in price ratio) + 0.1×cache_read + 2×cache_creation — closest to real spend, but needs per-model price awareness;
  • (b) Raw total: input + output + cache_read + cache_creation — simple, overweights cheap cache reads but at least monotone with activity;
  • (c) keep current fields for the budget but display the full breakdown everywhere — minimal change, fixes visibility only.
    Whichever is chosen, existing budgets' numeric meaning changes — call it out in release notes.

4. Dashboard: show cache hit ratio and warm/cold turn breakdown per mind — this is what makes the idle-compaction win visible.

5. Mind-facing context economics (the experience half):

  • Add a "Your context and what it costs" section to templates/_base/home/VOLUTE.md (or the orientation/memory skill): sessions resend history each turn; caching makes repeat turns cheap; the cache lasts ~1 hour idle; compaction trades detail for a light context; how to check your own usage.
  • Framed as self-knowledge, not restriction — a mind that understands its economics can choose to journal-and-compact before a quiet spell.

Testing

  • Unit: stream-consumer emits full usage fields; turn-lifecycle persists them; budget math per the chosen option.
  • Existing token-budget tests updated for the new semantics.

Part of the token-efficiency series. Principle: you can't steward what you can't see — and neither can the mind.

Related: #370 (budget UX — warning delivery/replay framing builds on these corrected numbers), #368 (environment transparency — VOLUTE_TOKEN_BUDGET startup line), #379.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P1High priority: real damage or blocks a trainarea:mind-experienceContinuity, transparency, notices, token economicsbugSomething isn't working

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions