fix: Reconcile token displays and persist metadata-only saves#459
Merged
fix: Reconcile token displays and persist metadata-only saves#459
Conversation
- Status bar T.<n> and Context: N% now derive from LastInputTokens, matching /context's "Current Context Size" so all three surfaces report the same metric (window fill, not cumulative billing). The Context: N% indicator always renders; HIGH/FULL labels still kick in at 75%/90%. - /context shortcut now falls back to the tokenizer polyfill when the provider does not return usage in its response, marking the value as ~<n> (estimated). The container's TokenizerService is reused so the shortcut, the optimizer, and the rollover manager share one instance. - JSONL storage's saveConversationUnlocked now appends a fresh metadata line on every save, including saves with no new entries. The previous early-return dropped metadata-only saves (e.g. AddTokenUsage updating token stats after the assistant message was already persisted), causing /conversations to lag by one save event. Added regression test TestJsonlStorage_MetadataOnlyUpdatePersists.
Switch the percentage shown by /context, the status bar Context: N% indicator, and the T.<n> raw counter to use cumulative TotalInputTokens instead of the most-recent prompt size. Users expect the meter to reflect "how much of the 1M-token allowance the session has spent" — e.g. 20K input cumulative on a 1M window now correctly shows 2%. The /context "Current Context Size" line is renamed to "Total Input Tokens" since it now reports cumulative usage. Tests updated to assert the new semantics with realistic numbers (20000 input → 2.0%).
ig-semantic-release-bot Bot
pushed a commit
that referenced
this pull request
Apr 27, 2026
## [0.104.2](v0.104.1...v0.104.2) (2026-04-27) ### 🐛 Bug Fixes * **services:** Prevent compact summary truncation at 200-token cap ([#457](#457)) ([36b9612](36b9612)), closes [#454](#454) * Reconcile token displays and persist metadata-only saves ([#459](#459)) ([8bc8767](8bc8767)) ### 📚 Documentation * Update agents MD ([#458](#458)) ([27bfaea](27bfaea)) ### 🧹 Maintenance * **nix:** Update package to v0.104.1 ([#456](#456)) ([784e4bc](784e4bc))
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
T.<n>andContext: N%now derive fromLastInputTokens, matching/context'sCurrent Context Size. All three surfaces report the same metric — window fill, not cumulative billing./context: When the provider does not return usage in its response, the shortcut falls back to the existingTokenizerServicepolyfill and marks the value as~<n> (estimated). The container's tokenizer is now created unconditionally and reused by the shortcut, the optimizer, and the rollover manager.saveConversationUnlockedpreviously skipped writing metadata when no new entries were added (if len(entries) > persistedCountgate). This dropped saves triggered byAddTokenUsageafter the assistant message was already persisted, causing/conversationsto lag by one save event. Now metadata is appended on every save; the reader already uses the last meta line.