feat(context): per-session context token tree across CLI, TUI, and dashboard#591
Merged
Conversation
Non-Composer sessions (e.g. GPT) record no context-window meter and keep the prompt in the agent stream, so the user bubble's own text is empty. Those turns hit the 0/0-token fallback with text_length 0 and were dropped entirely, so that model's traffic never appeared in the report. loadAgentToolsByComposer now also sums the user-role stream text length per conversation, and the meterless fallback estimates input from it (chars/4), credited once per conversation, when the bubble text is empty. Turns with no stream text are left untouched, so no phantom tokens are invented.
…n tree Reconstructs a zaly-style context breakdown for a Claude Code session from its transcript: messages and tokens by role, block type, and tool, split into full-session history vs the live window since the last compaction (following preservedSegment.headUuid). Block tokens are chars/4 estimates, images estimated from sniffed PNG or JPEG dimensions, reasoning derived per message from output_tokens minus visible output since transcripts strip thinking text. The exact context size comes from the last assistant message's API usage, and the gap to the estimate is shown as derived system prompt and tool overhead.
Reads the latest ai-title entry (summary entry as fallback for older sessions) from one tail and one head chunk of each transcript, so the list stays fast on 100MB files.
…trees Adds a Context page to the web dashboard: a navbar toggle, a provider picker (Claude Code / Codex), the 15 most recent sessions with titles, and per-session expandable details showing the context token tree, the exact live context vs window, and a live/full-history scope toggle. Server side adds /api/context/sessions and /api/context/tree with an mtime-keyed tree cache, plus a Codex rollout builder: response items feed the tree, compacted entries (with replacement_history) split live window from full history, reasoning comes exact from cumulative token_count totals, and model_context_window gives the real window.
codeburn context with no arguments now opens an ink TUI: arrow keys to move, enter to expand a session's token tree inline (with a spinner while the transcript reads), tab to switch between Claude Code and Codex, f to toggle live window vs full history, q to quit. Flag and id forms keep the plain output for scripts, and non-TTY runs fall back to the static list. Tree rows are shared with the static renderer via snapshotRows.
…d surfaces Window sizes are no longer guessed from token counts alone: opus-4-8 and [1m] model ids map to the 1M window (a 190K opus-4-8 session used to render as 95% of a 200K window), and Codex omits the percentage instead of borrowing Anthropic constants when the rollout lacks model_context_window. Codex compaction accounting now reads the encrypted compaction item and developer messages from replacement_history, so post-compaction windows stop undercounting. Both builders stream with largeLineAsBuffer so one oversized line cannot silently truncate the walk. The tree API stops sending session.filePath, serves the flattened rows so the dash renders the same tree as the CLI and TUI, resolves ids directly instead of re-scanning every session file per request, and the tree cache evicts LRU. Session discovery honors CLAUDE_CONFIG_DIRS, CLAUDE_CONFIG_DIR, and CODEX_HOME, stats files in parallel, and id lookups stat only matching files. CLI gains --provider codex and machine-readable --list --json; --full now opens the TUI in full scope instead of bypassing it. The TUI shows build errors instead of a frozen spinner and drops the ref-plus-counter repaint for plain state. The dashboard scopes the usage error banner and device sidebar to the Usage page.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
New
codeburn contextcommand plus a Context page in the web dashboard: for any Claude Code or Codex session, see what is actually filling the context window, broken down by role, block type, and tool, zaly-style but reconstructed from the session files on disk.codeburn contextopens an interactive TUI: arrows to move, enter expands a session's tree inline, tab switches Claude Code / Codex,ftoggles live window vs full history.codeburn context <id> [--provider codex] [--full] [--json]prints the same tree statically for scripts, and--list [--json]enumerates recent sessions with their titles.How it works
output_tokensminus the estimated visible output. Compaction boundaries (including the preserved segment carried across a compaction) split the live window from full history.token_countevents supply exact reasoning totals and the realmodel_context_window, andcompactedentries seed the new window from theirreplacement_history(including the encrypted compaction summary and developer messages).opus-4-8and[1m]ids map to 1M). When a Codex rollout does not state its window, the percentage is omitted rather than guessed.Notes
CLAUDE_CONFIG_DIRS,CLAUDE_CONFIG_DIR, andCODEX_HOME, stats files in parallel, and resolves ids without scanning every session file. The server caches built trees per file version with LRU eviction.