Skip to content

Fix Grok 4.5 (xAI BYOK): accept null usage on the Responses stream open - #249

Merged
KB (KB-syntheticsciences) merged 2 commits into
mainfrom
fix/grok-45-responses-usage-null
Aug 3, 2026
Merged

Fix Grok 4.5 (xAI BYOK): accept null usage on the Responses stream open#249
KB (KB-syntheticsciences) merged 2 commits into
mainfrom
fix/grok-45-responses-usage-null

Conversation

@KB-syntheticsciences

Copy link
Copy Markdown
Collaborator

What does this PR do?

Fixes Grok 4.5 (xAI BYOK), which failed on every prompt with AI_TypeValidationError before emitting any content. Grok 4.2 and every other xAI model were unaffected.

provider.ts:394 routes only Grok 4.5 through xAI's Responses API (sdk.responses()); everything else uses sdk.languageModel() (chat). So only 4.5 is validated against the Responses schema.

xAI opens every stream with:

{"type":"response.created","response":{ …, "usage": null, "status":"in_progress", … }}

@ai-sdk/xai@2.0.51 declares that field as required, and the response.created / response.in_progress union members relax it with .partial({ usage: true }):

usage: xaiResponsesUsageSchema,   // required

response: xaiResponsesResponseSchema.partial({ usage: true, status: true })

.partial() makes a field optional — it accepts absent. xAI sends it present and null, and Zod's .optional() does not accept null; that needs .nullable()/.nullish(). So the very first SSE event fails every branch of the chunk union with path: ["response","usage"] — expected object, received null, and the stream aborts. That is why the failure was immediate and why the UI showed an empty / compacted context.

This is fixed upstream. @ai-sdk/xai@4.0.25 changes exactly one thing in that schema:

usage: xaiResponsesUsageSchema.nullish(),

This PR applies that same one-line change as a pinned dependency patch, using the convention already established here — bun patchedDependencies with the patch under tooling/patches/, alongside the existing @ai-sdk/anthropic and @ai-sdk/openai patches.

Why patch instead of upgrading the SDK?

@ai-sdk/xai@4.x requires @ai-sdk/provider@4.0.4 against the pinned 2.0.1two majors of the interface every provider implements — dragging ~20 @ai-sdk/* packages plus ai core, and anything implementing the model interfaces (transform.ts, the custom loaders in provider.ts).

That upgrade is worth doing and is being scheduled separately. It should not be the vehicle for a user-facing regression fix: bundling them makes the fix unverifiable (a regression in, say, Bedrock could not be attributed), and Grok 4.5 stays broken for the length of the migration.

The patch header and a comment at provider.ts:394 both record that 4.0.25 fixes this upstream and that the patch should be deleted when the SDK migration lands.

How did you verify your code works?

Live, against a real xAI key, rebuilt binary, isolated config and data:

openscience run -m xai/grok-4.5 "Reply with exactly: OK"
  • Before: AI_TypeValidationError naming ["response","usage"], exit 1.
  • After: OK, exit 0.

Confirmed twice — once by the implementer, once independently from a fresh build.

Chat path unaffected: the same probe against a non-4.5 xAI model succeeds before and after.

Offline regression test (backend/cli/test/provider/xai-responses-usage.test.ts) feeds a response.created chunk carrying "usage": null through the real provider via a stubbed fetch — no network, no key. Mutation-verified: with the patch reverted it fails with exactly the production error; restored, it passes.

Suite: 1723 pass / 2 fail — both on the repo's known pre-existing list (npm selects every supported native package contract…, and killTree SIGKILLs a detached group, re-verified passing standalone). Typecheck 7/7 green.

Notes for the reviewer

  • patchedDependencies keys on the exact version string (@ai-sdk/xai@2.0.51). Any bump within 2.x silently drops the patch with no install-time error, and Grok 4.5 breaks again. The regression test is the only guard — worth remembering when the SDK migration is planned.
  • The patch matches upstream 4.0.25 exactly rather than narrowing to response.created, so it also loosens response.done / response.completed. Consequence: a null usage on a terminal event would degrade to zero-usage rather than erroring. Deliberate — staying identical to upstream keeps the eventual migration a clean deletion.
  • Live coverage is a single-turn prompt: no tool calls, reasoning summaries, or web search. Those exercise other parts of the Responses schema, and since this bug was a schema mismatch on the very first event, further mismatches may exist deeper in a stream.
  • The patch was verified to survive a from-scratch bun install --frozen-lockfile.

Checklist

  • bun run typecheck passes
  • bun test (in backend/cli) passes — modulo the pre-existing failures noted above
  • bunx prettier --check is clean on changed files
  • Linked an issue — the report this came from is not filed as a GitHub issue
  • Screenshots for UI changes — none; backend only

Grok 4.5 is the only model routed through xAI's Responses API, and every
turn died immediately with AI_TypeValidationError at ["response","usage"].
xAI opens each stream with a response.created event carrying "usage": null;
@ai-sdk/xai@2.0.51 declares that field optional rather than nullable, so
the first SSE event fails every branch of the chunk union and the stream
aborts before any content arrives.

Patch the pinned dependency to make usage nullish, matching the upstream
fix in @ai-sdk/xai@4.0.25. Taking 4.x directly is not viable yet: it needs
@ai-sdk/provider@4.0.4 against our pinned 2.0.1, two majors of the shared
spec across ~20 packages. The patch header and the provider.ts comment
both record that it should be deleted when that migration lands.

Covered by an offline regression test that replays the null-usage
response.created through the real provider via a canned fetch.
@vercel

vercel Bot commented Aug 3, 2026

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
openscience Ready Ready Preview Aug 3, 2026 6:42pm

Request Review

@aayambansal

Copy link
Copy Markdown
Member

KB (@KB-syntheticsciences) looks good, please make sure ci-cd are green and not failing! merge after :)

@KB-syntheticsciences
KB (KB-syntheticsciences) merged commit 1b7978c into main Aug 3, 2026
12 checks passed
@aayambansal
Aayam Bansal (aayambansal) deleted the fix/grok-45-responses-usage-null branch August 4, 2026 16:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants