Skip to content

fix(stats): savings calc no longer goes negative when user spends on media#36

Merged
1bcMax merged 1 commit intomainfrom
fix/savings-includes-media-cost
Apr 30, 2026
Merged

fix(stats): savings calc no longer goes negative when user spends on media#36
1bcMax merged 1 commit intomainfrom
fix/savings-includes-media-cost

Conversation

@KillerQueen-Z
Copy link
Copy Markdown
Collaborator

Summary

The "Saved vs Opus" hero on the panel can display a negative dollar amount as soon as a user spends meaningfully on `ImageGen` or `VideoGen`. Real example just hit:

$-8.79
You spent $20.4896 instead of $11.70

This happens because the comparison in `getStatsSummary()` mixes two different accounting universes.

Root cause

`src/stats/tracker.ts:268-272`:

```typescript
const opusCost =
(stats.totalInputTokens / 1_000_000) * OPUS_PRICING.input +
(stats.totalOutputTokens / 1_000_000) * OPUS_PRICING.output;
const saved = opusCost - stats.totalCostUsd;
```

  • `opusCost` only counts chat tokens (priced at Opus rates).
  • `stats.totalCostUsd` includes every recorded costUsd — chat plus image / video / music gen.
  • `recordUsage` writes media calls with `inputTokens=0, outputTokens=0` because per_image / per_second / per_track billing has no token concept, so they don't enter `opusCost` but they do enter `totalCostUsd`.

`saved` therefore equals `(chat-Opus-vs-chosen-delta) − media_cost`. Once media spend exceeds the chat delta, `saved` flips negative even though every chat call was strictly cheaper than Opus would have been.

Fix

Walk `byModel` once and split spend by whether the row accumulated tokens:

  • chatOnlyCost — rows that did (chat models)
  • mediaCost — rows that didn't (image / video / music)

Then construct the comparison so media appears on both sides of "you spent X instead of Y":

```typescript
const opusChatCost = (totalIn / 1M) * OPUS_PRICING.input + (totalOut / 1M) * OPUS_PRICING.output;
const opusCost = opusChatCost + mediaCost; // display baseline
const totalCostUsd = chatOnlyCost + mediaCost; // unchanged
const saved = Math.max(0, opusChatCost - chatOnlyCost);
```

Media nets to zero in the diff, so `saved` correctly reflects only the chat-side win — and the `Math.max(0, ...)` clamp handles the edge case where the user deliberately picks a more expensive chat model than Opus (e.g. Sonnet 4.6 with extended thinking).

`getStatsSummary` now also returns `chatOnlyCost` and `mediaCost` for any future panel feature that wants to show the breakdown.

Worked example

Before After
chat spend $2.00 $2.00
image spend $11.30 $11.30
Opus chat baseline $11.70 $11.70
Saved displayed −$1.60 $9.70
"You spent" $13.30 $13.30
"instead of" $11.70 $23.00 (= $11.70 + $11.30)

Both displayed totals now match the user's actual wallet activity, savings is the chat-side delta only, and the number is never negative.

Files

  • `src/stats/tracker.ts` — split chat/media in `getStatsSummary()`, expose both on the return type, clamp `saved` to >= 0
  • `src/panel/html.ts` — defensive: clamp `saved` and `pct` to >= 0 even if a stale `stats` object lacks `saved` (older panel sessions could otherwise still hit the negative path)

```
src/panel/html.ts | 11 +++++++++--
src/stats/tracker.ts | 38 +++++++++++++++++++++++++++++++++-----
2 files changed, 42 insertions(+), 7 deletions(-)
```

Out of scope

  • `src/stats/insights.ts` already has a `Math.max(0, ...)` clamp on its own savings calc, so it never displayed a negative number — but its math has the same chat/media conflation. Could be aligned in a follow-up.
  • No change to `recordUsage`, the on-disk schema, or any chat-only flow.

Test plan

```bash

Reproduce the negative case (before)

franklin --prompt 'use bytedance/seedance-2.0 to generate a 5-second clip of clouds'

repeat until media spend > chat-vs-Opus delta

franklin panel # open dashboard → savings hero → see negative dollar
```

After: same flow shows "You spent $X instead of $Y, saved $Z" where Z >= 0 and X / Y both include media.

…media

The "Saved vs Opus" hero on the panel could display a negative dollar
amount as soon as a user spent meaningfully on ImageGen or VideoGen
(e.g. \"$-8.79 — You spent $20.4896 instead of $11.70\").

## Root cause

`getStatsSummary()` in src/stats/tracker.ts compared two values that
live in different accounting universes:

- `opusCost` = (totalInputTokens + totalOutputTokens) * Opus token rate
  → only counts chat tokens
- `stats.totalCostUsd` = every recorded costUsd (chat + image + video +
  music)

`recordUsage` for media generation logs costUsd > 0 with both
inputTokens and outputTokens at 0 (per_image / per_second / per_track
billing has no token concept). So:

```
saved = opusCost - totalCostUsd
      = (chat-tokens-at-Opus-rates) - (chat + media)
      = (Opus-vs-chosen-chat-delta) - media_cost
```

Once `media_cost` exceeded the chat delta, `saved` flipped negative
even though the user genuinely saved money on every chat call —
the math just doesn't have room for media.

## Fix

Walk `byModel` once and split spend into:

- **chatOnlyCost** — rows that ever accumulated tokens
- **mediaCost** — rows that didn't

Then build the comparison so media appears identically on both sides
of "you spent X instead of Y", which makes the displayed totals match
the user's real wallet activity:

```
opusCost (display baseline) = opusChatCost + mediaCost
totalCostUsd (display actual) = chatOnlyCost + mediaCost   (unchanged)
saved = max(0, opusChatCost - chatOnlyCost)
```

The `Math.max(0, ...)` clamp handles the edge case where the user
deliberately picked a more expensive chat model than Opus
(e.g. Sonnet 4.6 with extended thinking on every request) — show
zero saved, never negative.

Also exposes `chatOnlyCost` and `mediaCost` on the returned summary
so future panel improvements can show the breakdown explicitly. The
panel hero is unchanged in shape — same "spent X instead of Y" line,
just with consistent numbers behind it.

## Worked example

Before:

```
totalInputTokens = 50k, totalOutputTokens = 30k
chat spend = $2.00 (cheap mix)
image spend = $11.30
totalCostUsd = $13.30, opusCost = $11.70
saved = $11.70 - $13.30 = -$1.60   ← bug
```

After:

```
chatOnlyCost = $2.00, mediaCost = $11.30
opusChatCost = $11.70
opusCost (display) = $11.70 + $11.30 = $23.00
totalCostUsd (display) = $13.30 (unchanged)
saved = max(0, $11.70 - $2.00) = $9.70
```

Hero now reads "You spent $13.30 instead of $23.00, saved $9.70" — every number reflects reality.

## Out of scope

- The `Insights` panel (`src/stats/insights.ts`) already had a
  `Math.max(0, ...)` clamp on its own savings calc, so it was never
  visibly negative — but its math has the same chat/media conflation.
  Could be aligned in a follow-up.
KillerQueen-Z added a commit that referenced this pull request Apr 30, 2026
Mirror of upstream PR #36 (fix/savings-includes-media-cost). The
"Saved vs Opus" panel hero would show negative dollar amounts as
soon as a user spent meaningfully on ImageGen / VideoGen, e.g.

    $-8.79
    You spent $20.4896 instead of $11.70

Root cause: getStatsSummary() compared an Opus-token baseline (chat
only — image/video log inputTokens=0/outputTokens=0) against
totalCostUsd (chat + media combined), so once media spend exceeded
the chat-vs-Opus delta the difference flipped negative.

Fix: split byModel into chatOnlyCost (rows with tokens) and mediaCost
(rows without). opusCost on the display side now equals
opusChatCost + mediaCost so "you spent X instead of Y" stays
apples-to-apples; saved = max(0, opusChatCost - chatOnlyCost) is
the chat-side delta only and is clamped non-negative.

Bumps vscode-extension to 0.5.1; updates README changelog.
@1bcMax 1bcMax merged commit 261b0e0 into main Apr 30, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant