Skip to content

Commit e1117fc

Browse files
🤖 fix: preserve usage data when stream is interrupted (#837)
## Problem When a stream is interrupted (e.g., by a queued message, user cancellation, or starting a new message), the usage data (breakdown by type and context window) resets to 0. ### Root Causes **1. AI SDK's `totalUsage` returns zeros on abort** In `cancelStreamSafely()`, we called `getStreamMetadata()` which awaits the AI SDK's `totalUsage` promise. When a stream is aborted mid-execution, this promise resolves with **zeros** (not `undefined`), so our fallback logic (`totalUsage ?? cumulativeUsage`) never triggered. **2. `usageStore` cache not invalidated on `stream-start`** The `MapStore` caches computed usage state. When a new stream starts after an abort: - `stream-abort` bumps `usageStore` ✓ - `stream-start` only bumped `states`, NOT `usageStore` ✗ - The stale cached value showed `liveUsage` as undefined ## Solution 1. **Use tracked `cumulativeUsage` directly** instead of AI SDK's unreliable `totalUsage` on abort. This is updated on each `finish-step` event (before tool execution), so it has accurate data even when interrupted mid-tool-call. 2. **Bump `usageStore` on `stream-start`** to invalidate the cache when a new stream begins. ### Trade-off Usage from the **interrupted step** is abandoned. The AI SDK's `finish-step` event fires *before* tool execution begins, so if we abort during tool execution, that step's usage was already recorded. However, if we abort during the model's response generation (before `finish-step`), that partial step's usage is lost. This is acceptable since: - We can't get reliable data from the SDK for interrupted steps - Cumulative usage from all *completed* steps is preserved - The alternative (zeros everywhere) is worse --- _Generated with `mux`_
1 parent f1a6cae commit e1117fc

File tree

3 files changed

+24
-3
lines changed

3 files changed

+24
-3
lines changed

src/browser/stores/WorkspaceStore.ts

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -144,6 +144,8 @@ export class WorkspaceStore {
144144
// Don't reset retry state here - stream might still fail after starting
145145
// Retry state will be reset on stream-end (successful completion)
146146
this.states.bump(workspaceId);
147+
// Bump usage store so liveUsage is recomputed with new activeStreamId
148+
this.usageStore.bump(workspaceId);
147149
},
148150
"stream-delta": (workspaceId, aggregator, data) => {
149151
aggregator.handleStreamDelta(data as never);

src/common/types/stream.ts

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,14 @@ export interface StreamAbortEvent {
5959
messageId: string;
6060
// Metadata may contain usage if abort occurred after stream completed processing
6161
metadata?: {
62+
// Total usage across all steps (for cost calculation)
6263
usage?: LanguageModelV2Usage;
64+
// Last step's usage (for context window display - inputTokens = current context size)
65+
contextUsage?: LanguageModelV2Usage;
66+
// Provider metadata for cost calculation (cache tokens, etc.)
67+
providerMetadata?: Record<string, unknown>;
68+
// Last step's provider metadata (for context window cache display)
69+
contextProviderMetadata?: Record<string, unknown>;
6370
duration?: number;
6471
};
6572
abandonPartial?: boolean;

src/node/services/streamManager.ts

Lines changed: 15 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -500,15 +500,27 @@ export class StreamManager extends EventEmitter {
500500
// while a new stream starts (e.g., old stream writing to partial.json)
501501
await streamInfo.processingPromise;
502502

503-
// Get usage and duration metadata (usage may be undefined if aborted early)
504-
const { usage, duration } = await this.getStreamMetadata(streamInfo);
503+
// For aborts, use our tracked cumulativeUsage directly instead of AI SDK's totalUsage.
504+
// cumulativeUsage is updated on each finish-step event (before tool execution),
505+
// so it has accurate data even when the stream is interrupted mid-tool-call.
506+
// AI SDK's totalUsage may return zeros or stale data when aborted.
507+
const duration = Date.now() - streamInfo.startTime;
508+
const hasCumulativeUsage = (streamInfo.cumulativeUsage.totalTokens ?? 0) > 0;
509+
const usage = hasCumulativeUsage ? streamInfo.cumulativeUsage : undefined;
510+
511+
// For context window display, use last step's usage (inputTokens = current context size)
512+
const contextUsage = streamInfo.lastStepUsage;
513+
const contextProviderMetadata = streamInfo.lastStepProviderMetadata;
514+
515+
// Include provider metadata for accurate cost calculation
516+
const providerMetadata = streamInfo.cumulativeProviderMetadata;
505517

506518
// Emit abort event with usage if available
507519
this.emit("stream-abort", {
508520
type: "stream-abort",
509521
workspaceId: workspaceId as string,
510522
messageId: streamInfo.messageId,
511-
metadata: { usage, duration },
523+
metadata: { usage, contextUsage, duration, providerMetadata, contextProviderMetadata },
512524
abandonPartial,
513525
});
514526

0 commit comments

Comments
 (0)