Track cached input tokens in provider usage + Cost event#322
Merged
Conversation
zarvis reported no prompt-cache usage because providers never parsed the cached-token counts. Add `Usage.cached_tokens` and parse it from each provider that exposes it — codex-oauth (`input_tokens_details.cached_tokens`), OpenAI (`prompt_tokens_details.cached_tokens`), Anthropic + Claude CLI (`cache_read_input_tokens`). Surface it on `SessionEvent::Cost` as `tokens_cached` (serde default → backward-compatible), and show `cached=N` in the cost modeline.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Adds
Usage.cached_tokensand atokens_cachedfield onSessionEvent::Cost, populated from each provider that reports a cached-token count:usage.input_tokens_details.cached_tokensusage.prompt_tokens_details.cached_tokensusage.cache_read_input_tokensusage.cache_read_input_tokensThe cost modeline now shows
cached=Nalongsidein/out.Why
Benchmarking surfaced that zarvis reported 0 cached tokens while the Codex CLI on the same
store:falsebackend reported ~97% of its input as cached. The cause was purely a parsing gap — the providers only readinput_tokens/output_tokensand dropped the cached breakdown — so prompt-cache usage was invisible (and unmeasurable in trajectories/cost). This plumbs the existing data through so cache hit-rate can actually be observed per turn.Compatibility
tokens_cachedis#[serde(default)], so old transcripts/clients deserialize unchanged;Usage.cached_tokensdefaults to 0 for providers that don't report it (e.g. Ollama).Test plan
cargo build(workspace) +cargo test -p agentd-protocol -p agentd-adapter-zarvis -p agentd-adapter-claude— green.input_tokens/output_tokenspaths; field defaults to 0 when absent.🤖 Generated with Claude Code