Skip to content

fix(openai-compatible): clamp negative outputTokens.text from inconsistent provider usage - #18614

Merged
shaper merged 2 commits into
mainfrom
fix/openai-compatible-negative-text-tokens
Aug 9, 2026
Merged

fix(openai-compatible): clamp negative outputTokens.text from inconsistent provider usage#18614
shaper merged 2 commits into
mainfrom
fix/openai-compatible-negative-text-tokens

Conversation

@shaper

@shaper shaper commented Aug 9, 2026

Copy link
Copy Markdown
Collaborator

Problem

Providers can return internally inconsistent usage where completion_tokens_details.reasoning_tokens exceeds completion_tokens. Observed with Baseten serving moonshotai/Kimi-K3: the model spent its whole output budget on reasoning and hit the length stop before emitting any text, and the response reported:

"usage": {
  "prompt_tokens": 951,
  "completion_tokens": 6000,
  "total_tokens": 6952,
  "prompt_tokens_details": { "cached_tokens": 60 },
  "completion_tokens_details": { "reasoning_tokens": 6001 }
}

(total_tokens = 951 + 6001, so completion_tokens is the field undercounting the actual generation here.)

convertOpenAICompatibleChatUsage computes text = completion_tokens - reasoning_tokens, which yields outputTokens.text: -1 for this response. A negative token count is nonsensical and breaks downstream consumers that validate usage values (e.g. schemas requiring non-negative counts).

Fix

Clamp the text share at 0 — the text portion of completion tokens can never be negative. total and reasoning remain exactly as reported by the provider, so consumers that aggregate text + reasoning stay aligned with the actual generation (6001 reasoning tokens here).

Testing

  • Added convert-openai-compatible-chat-usage.test.ts with a regression case replaying the incident payload, plus null/normal cases.
  • Updated 5 snapshots that had baked in negative text values (-318, -229, -338, -201) from recorded xAI fixtures — the same class of provider-side usage inconsistency, previously recorded as expected output.
  • Reproduced end-to-end via a mock fetch replaying the incident payload through the Baseten provider on both the generateText and streamText paths: textTokens: -1 pre-fix, 0 post-fix.
  • Full @ai-sdk/openai-compatible suite (node + edge, 238 tests) and tsc --build pass.

@shaper
shaper requested a review from aayush-kapoor August 9, 2026 07:34
@shaper shaper added the backport Admins only: add this label to a pull request in order to backport it to the prior version label Aug 9, 2026
@shaper
shaper merged commit 83e6510 into main Aug 9, 2026
56 checks passed
@shaper
shaper deleted the fix/openai-compatible-negative-text-tokens branch August 9, 2026 17:39
github-actions Bot added a commit that referenced this pull request Aug 9, 2026
…stent provider usage (#18614)

## Problem

Providers can return internally inconsistent usage where
`completion_tokens_details.reasoning_tokens` exceeds
`completion_tokens`. Observed with Baseten serving `moonshotai/Kimi-K3`:
the model spent its whole output budget on reasoning and hit the length
stop before emitting any text, and the response reported:

```json
"usage": {
  "prompt_tokens": 951,
  "completion_tokens": 6000,
  "total_tokens": 6952,
  "prompt_tokens_details": { "cached_tokens": 60 },
  "completion_tokens_details": { "reasoning_tokens": 6001 }
}
```

(`total_tokens` = 951 + 6001, so `completion_tokens` is the field
undercounting the actual generation here.)

`convertOpenAICompatibleChatUsage` computes `text = completion_tokens -
reasoning_tokens`, which yields `outputTokens.text: -1` for this
response. A negative token count is nonsensical and breaks downstream
consumers that validate usage values (e.g. schemas requiring
non-negative counts).

## Fix

Clamp the text share at 0 — the text portion of completion tokens can
never be negative. `total` and `reasoning` remain exactly as reported by
the provider, so consumers that aggregate `text + reasoning` stay
aligned with the actual generation (6001 reasoning tokens here).

## Testing

- Added `convert-openai-compatible-chat-usage.test.ts` with a regression
case replaying the incident payload, plus null/normal cases.
- Updated 5 snapshots that had baked in negative `text` values (`-318`,
`-229`, `-338`, `-201`) from recorded xAI fixtures — the same class of
provider-side usage inconsistency, previously recorded as expected
output.
- Reproduced end-to-end via a mock fetch replaying the incident payload
through the Baseten provider on both the `generateText` and `streamText`
paths: `textTokens: -1` pre-fix, `0` post-fix.
- Full `@ai-sdk/openai-compatible` suite (node + edge, 238 tests) and
`tsc --build` pass.
@github-actions github-actions Bot removed the backport Admins only: add this label to a pull request in order to backport it to the prior version label Aug 9, 2026
@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

✅ Backport PR created: #18620

@github-actions

github-actions Bot commented Aug 9, 2026

Copy link
Copy Markdown
Contributor

🚀 Published in:

Package Version
@ai-sdk/baseten 2.1.6 github npm
@ai-sdk/cerebras 3.0.28 github npm
@ai-sdk/deepinfra 3.0.28 github npm
@ai-sdk/fireworks 3.0.30 github npm
@ai-sdk/google-vertex 5.0.48 github npm
@ai-sdk/huggingface 2.0.28 github npm
@ai-sdk/moonshotai 3.0.31 github npm
@ai-sdk/openai-compatible 3.0.28 github npm
@ai-sdk/togetherai 3.0.29 github npm
@ai-sdk/vercel 3.0.28 github npm

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants