Commit e1117fc
authored
🤖 fix: preserve usage data when stream is interrupted (#837)
## Problem
When a stream is interrupted (e.g., by a queued message, user
cancellation, or starting a new message), the usage data (breakdown by
type and context window) resets to 0.
### Root Causes
**1. AI SDK's `totalUsage` returns zeros on abort**
In `cancelStreamSafely()`, we called `getStreamMetadata()` which awaits
the AI SDK's `totalUsage` promise. When a stream is aborted
mid-execution, this promise resolves with **zeros** (not `undefined`),
so our fallback logic (`totalUsage ?? cumulativeUsage`) never triggered.
**2. `usageStore` cache not invalidated on `stream-start`**
The `MapStore` caches computed usage state. When a new stream starts
after an abort:
- `stream-abort` bumps `usageStore` ✓
- `stream-start` only bumped `states`, NOT `usageStore` ✗
- The stale cached value showed `liveUsage` as undefined
## Solution
1. **Use tracked `cumulativeUsage` directly** instead of AI SDK's
unreliable `totalUsage` on abort. This is updated on each `finish-step`
event (before tool execution), so it has accurate data even when
interrupted mid-tool-call.
2. **Bump `usageStore` on `stream-start`** to invalidate the cache when
a new stream begins.
### Trade-off
Usage from the **interrupted step** is abandoned. The AI SDK's
`finish-step` event fires *before* tool execution begins, so if we abort
during tool execution, that step's usage was already recorded. However,
if we abort during the model's response generation (before
`finish-step`), that partial step's usage is lost. This is acceptable
since:
- We can't get reliable data from the SDK for interrupted steps
- Cumulative usage from all *completed* steps is preserved
- The alternative (zeros everywhere) is worse
---
_Generated with `mux`_1 parent f1a6cae commit e1117fc
File tree
3 files changed
+24
-3
lines changed- src
- browser/stores
- common/types
- node/services
3 files changed
+24
-3
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
144 | 144 | | |
145 | 145 | | |
146 | 146 | | |
| 147 | + | |
| 148 | + | |
147 | 149 | | |
148 | 150 | | |
149 | 151 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
59 | 59 | | |
60 | 60 | | |
61 | 61 | | |
| 62 | + | |
62 | 63 | | |
| 64 | + | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
63 | 70 | | |
64 | 71 | | |
65 | 72 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
500 | 500 | | |
501 | 501 | | |
502 | 502 | | |
503 | | - | |
504 | | - | |
| 503 | + | |
| 504 | + | |
| 505 | + | |
| 506 | + | |
| 507 | + | |
| 508 | + | |
| 509 | + | |
| 510 | + | |
| 511 | + | |
| 512 | + | |
| 513 | + | |
| 514 | + | |
| 515 | + | |
| 516 | + | |
505 | 517 | | |
506 | 518 | | |
507 | 519 | | |
508 | 520 | | |
509 | 521 | | |
510 | 522 | | |
511 | | - | |
| 523 | + | |
512 | 524 | | |
513 | 525 | | |
514 | 526 | | |
| |||
0 commit comments