feat: add token usage counter per session#31
Conversation
enowdev
left a comment
There was a problem hiding this comment.
Nice feature idea, but I’m blocking this version because send_openai_compatible() now unconditionally adds stream_options: { include_usage: true } for every OpenAI-compatible provider (src-tauri/src/services/chat_service.rs:227-235). In this repo that path is also used for Ollama/custom gateways/other OpenAI-style backends, and many of them reject unknown request fields instead of ignoring them. That means this can break normal chat completions for providers that worked before. Please gate usage collection behind provider capability detection (or a fallback/retry path) before merging.
Tracks prompt and completion token counts from both OpenAI-compatible and Anthropic SSE streams and accumulates them per session in the frontend store. Backend: - TokenUsage / ChatUsageEvent / UsageAccumulator structs in chat_service.rs using serde::Serialize (no serde_json::json! to avoid clippy::disallowed_methods) - stream_openai_sse: requests stream_options.include_usage=true so the final SSE chunk carries usage; parsed in parse_openai_sse_line - stream_anthropic_sse: captures input_tokens from message_start and output_tokens from message_delta events - Emits chat-usage Tauri event after each completed completion - Also fixes stream_anthropic_sse to return error on missing message_stop (same as the pending PR enowdev#26) Frontend: - TokenUsage / ChatUsageEvent types added to types/index.ts - useChatStore: sessionUsage record, addTokenUsage (cumulative per-session sum), clearSessionUsage - AppShell: listens for chat-usage, calls addTokenUsage - ChatHeader: shows total token badge next to session title; tooltip shows split prompt / completion counts; formats as 4.2k for readability
aa5152e to
b8f8dca
Compare
enowdev
left a comment
There was a problem hiding this comment.
Conflict resolution is in and the usage-collection path is now gated so we do not send stream_options.include_usage to every OpenAI-compatible backend. This is in a much safer state for mixed provider setups.
# Conflicts: # src-tauri/src/services/chat_service.rs
Description
Tracks prompt and completion token counts from both OpenAI-compatible and Anthropic SSE streams, accumulates them per session, and displays the total as a badge in the chat header.
Backend (
src-tauri/src/services/chat_service.rs)TokenUsage,ChatUsageEvent, andUsageAccumulatorstructs using#[derive(serde::Serialize)]— noserde_json::json!()to stay clear ofclippy::disallowed_methodsstream_openai_sse: addsstream_options.include_usage: trueto the request so the final SSE chunk carries usage; captured inparse_openai_sse_linestream_anthropic_sse: capturesinput_tokensfrommessage_startandoutput_tokensfrommessage_deltaevents(String, Option<TokenUsage>)instead ofStringsend_message_inneremitschat-usageTauri event after each completed responseFrontend
src/types/index.ts:TokenUsageandChatUsageEventinterfacessrc/stores/useChatStore.ts:sessionUsage: Record<string, TokenUsage>,addTokenUsage(cumulative per-session sum),clearSessionUsagesrc/components/layout/AppShell.tsx: listens forchat-usageevent, callsaddTokenUsagesrc/components/layout/ChatHeader.tsx: token badge next to session title; tooltip shows split prompt/completion; formats as4.2kfor large numbersPreview
Type of Change
How Has This Been Tested?
bunx tsc --noEmitpasses (TypeScript) — only pre-existingbaseUrldeprecation warningcargo clippy -- -D warningspasses (Rust) — could not run in current environment due to missing GTK system deps on WSLManual verification:
UsageAccumulator::finish()returnsNonewhen both fields are 0 — no spurious events for providers that omit usageaddTokenUsageaccumulates across multiple turns per session, not reset per messagestream_options.include_usageis a standard OpenAI API field, silently ignored by providers that don't support itmessage_start.message.usage.input_tokensandmessage_delta.usage.output_tokensChecklist