feat(terminal): add context token status

[omarluq]

No description provided.

[coderabbitai]

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 163f75ee-e0ac-4721-b5df-0cac2d516508

📥 Commits

Reviewing files that changed from the base of the PR and between b0ef5cb and 0a542c6.

📒 Files selected for processing (5)

• internal/assistant/sse.go
• internal/assistant/usage_test.go
• internal/terminal/app.go
• internal/terminal/token_usage_export_test.go
• internal/terminal/token_usage_test.go

🚧 Files skipped from review as they are similar to previous changes (3)

• internal/terminal/token_usage_export_test.go
• internal/assistant/sse.go
• internal/assistant/usage_test.go

---

📝 Walkthrough

Summary by CodeRabbit

• New Features

  • End-to-end token usage tracking for assistant requests (estimates merged with provider-reported usage).
  • Token-usage stream events and explicit emission during prompts.
  • Terminal status line now displays context tokens, context window, and percentage.

• Tests

  • Added comprehensive tests for usage parsing, merging, SSE handling, streaming propagation, and terminal rendering/merge behavior.

Walkthrough

This PR implements end-to-end token usage tracking in the assistant and terminal. It adds a TokenUsage model, parses and merges provider usage, estimates prompt/message usage in the runtime and emits usage events, threads usage through terminal async events and input handling, and displays token context information in the status line.

Changes

Token Usage Tracking

Layer / File(s)	Summary
Token Usage Model Definition internal/model/usage.go	TokenUsage struct with integer fields for context window/tokens and helper methods (EmptyTokenUsage, TotalTokens, HasAny, ContextPercent).
CompletionResult plumbing and client constants internal/assistant/client.go	Adds jsonOutputTokensKey, extends CompletionResult/providerResult with Usage model.TokenUsage, and updates textCompletionResult to accept usage.
Provider response parsing (Anthropic / OpenAI chat) internal/assistant/anthropic.go, internal/assistant/openai_chat.go	Provider response structs now include Usage fields and completion construction passes parsed usage into results.
OpenAI Responses loop & SSE accumulator internal/assistant/openai_responses.go, internal/assistant/sse.go	Tracks and accumulates usage across streamed iterations, ensures explicit empty usage fallback for SSE/fallback paths, and preserves usage when recomputing results from streamed items.
Usage estimation & parsing helpers internal/assistant/usage.go	Adds token estimation heuristic, mergeUsage, usageFromObject, and helpers to coerce numeric provider fields into model.TokenUsage.
emitUsage helper internal/assistant/usage_events.go	Adds Runtime.emitUsage to emit usage stream events and optionally forward them to runtime extensions.
Runtime estimation, emission, and integration internal/assistant/runtime.go, internal/assistant/usage_events.go	Builds model-facing messages, emits estimated usage before completion, merges estimated and provider-reported usage after completion, emits merged usage events, and returns response bundles with Usage.
Tool execution stream event refactor internal/assistant/tool_loop.go	Refactors tool start/result stream events into multi-line struct literals for consistency.
Assistant usage tests internal/assistant/usage_test.go, internal/assistant/runtime_test.go	Adds tests for parsing provider-shaped usage objects, mergeUsage semantics, SSE parsing preserving usage, and updates runtime test mocks to include Usage fields.
Terminal async events and app integration internal/terminal/async_events.go, internal/terminal/app.go	Async events now carry Usage pointers and are posted as pointers; handlers updated to accept pointers; App gains tokenUsage initialized/reset to EmptyTokenUsage; interrupt and lifecycle handling updated.
Auth event posting & input integration internal/terminal/auth_commands.go, internal/terminal/input.go	Auth and prompt events now posted as &asyncEvent{...} with Usage: nil; sendPrompt resets app token usage and applies returned usage before side effects.
Terminal merge logic and statusline formatting internal/terminal/token_usage.go, internal/terminal/runtime_buffers.go	Implements mergeTerminalUsage/applyTokenUsage, tokenStatusText/formatContextUsage, compactCount, and appends a token status segment to the status line when present.
Terminal tests & helpers internal/terminal/token_usage_export_test.go, internal/terminal/token_usage_test.go, internal/terminal/render_parity_test.go, internal/terminal/render_test.go	Exports test helpers for merging and App manipulation; adds unit tests validating merge semantics, clearing usage on reset/truncate, and statusline rendering of token context.

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Poem

🐰 A hop through tokens bright and gleaming,
Context windows now are streaming,
Usage flows from prompt to line,
Estimates and reports align,
Status bars hum with counts, all beaming.

🚥 Pre-merge checks | ✅ 3 | ❌ 1

❌ Failed checks (1 inconclusive)

Check name	Status	Explanation	Resolution
Description check	❓ Inconclusive	No description was provided by the author; the PR lacks explanatory context about the changes.	Add a description explaining the purpose, implementation details, and scope of the context token status feature.

✅ Passed checks (3 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately describes the primary change: adding context token status display in the terminal UI.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch feat/context-token-status

Tip

💬 Introducing Slack Agent: The best way for teams to turn conversations into code.

Slack Agent is built on CodeRabbit's deep understanding of your code, so your team can collaborate across the entire SDLC without losing context.

• Generate code and open pull requests
• Plan features and break down work
• Investigate incidents and troubleshoot customer tickets together
• Automate recurring tasks and respond to alerts with triggers
• Summarize progress and report instantly

Built for teams:

• Shared memory across your entire org—no repeating context
• Per-thread sandboxes to safely plan and execute work
• Governance built-in—scoped access, auditability, and budget controls

One agent for your entire SDLC. Right inside Slack.

👉 Get started

---

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@internal/assistant/sse.go`:
- Around line 31-37: When handling event["response"] in the SSE accumulator,
avoid overwriting previously captured usage: instead of blindly doing
accumulator.finalResponse = response, merge any existing usage from
accumulator.finalResponse["usage"] into the new response (e.g., if oldUsage :=
accumulator.finalResponse["usage"]; oldUsage != nil { response["usage"] =
oldUsage }) before assigning; keep using
ensureSSEFinalResponse(accumulator.finalResponse) when processing event["usage"]
so later usage updates still attach correctly to accumulator.finalResponse.

In `@internal/terminal/app.go`:
- Line 143: Reset tokenUsage when clearing message or session state: find the
places that reset message/session state (the same code paths that clear current
message, session, or message-related fields) and add tokenUsage =
model.TokenUsage{} so it is zeroed alongside other resets; ensure you update
every reset path referenced in this file (including the other reset locations
around where message/session switching occurs) so stale token usage isn't
retained after ResetMessage/ResetSession flows.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 6b7a0c65-ff80-484a-99e4-db0c59fde6f4

📥 Commits

Reviewing files that changed from the base of the PR and between 6aafdcb and b9f122d.

📒 Files selected for processing (22)

• internal/assistant/anthropic.go
• internal/assistant/client.go
• internal/assistant/openai_chat.go
• internal/assistant/openai_responses.go
• internal/assistant/runtime.go
• internal/assistant/runtime_test.go
• internal/assistant/sse.go
• internal/assistant/tool_loop.go
• internal/assistant/usage.go
• internal/assistant/usage_events.go
• internal/assistant/usage_test.go
• internal/model/usage.go
• internal/terminal/app.go
• internal/terminal/async_events.go
• internal/terminal/auth_commands.go
• internal/terminal/input.go
• internal/terminal/render_parity_test.go
• internal/terminal/render_test.go
• internal/terminal/runtime_buffers.go
• internal/terminal/token_usage.go
• internal/terminal/token_usage_export_test.go
• internal/terminal/token_usage_test.go