Skip to content

fix(stream): correct cached input accounting - #31

Merged
patlux merged 2 commits into
patlux:mainfrom
IfkumRfnl:fix/cache-accounting
Aug 3, 2026
Merged

fix(stream): correct cached input accounting#31
patlux merged 2 commits into
patlux:mainfrom
IfkumRfnl:fix/cache-accounting

Conversation

@IfkumRfnl

Copy link
Copy Markdown
Contributor

Summary

  • Use inputTokenDetails.noCacheTokens for uncached input usage.
  • Fall back to subtracting cache reads and writes from total input.
  • Add regression coverage using the usage shape returned by Command Code.

Command Code's inputTokens includes cached tokens. Treating it as uncached input caused cache reads to be counted twice in pi's usage totals, cost estimates, and context accounting.

Testing

  • npm run typecheck
  • npm run test:unit
  • npm run test:stream
  • npm run format:check
  • Manually verified with deepseek/deepseek-v4-flash

Fixes #30

@patlux patlux left a comment

Copy link
Copy Markdown
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verified the token semantics against the Vercel AI SDK usage schema and the provider implementations used by Command Code. The fix correctly separates uncached input, cache reads, and cache writes. Added regression coverage for the fallback without noCacheTokens and for cacheWriteTokens; local tests and CI are green.

@patlux

patlux commented Aug 3, 2026

Copy link
Copy Markdown
Owner

Thank you @IfkumRfnl for reporting this with a concrete API example and providing the fix! I independently verified the usage semantics against the Vercel AI SDK and the provider implementations used by Command Code. I also added coverage for the fallback path and cache-write accounting. Everything is green.

@patlux
patlux merged commit b151aaa into patlux:main Aug 3, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cached input tokens are counted twice in usage totals

2 participants