Fix Grok 4.5 (xAI BYOK): accept null usage on the Responses stream open - #249
Merged
Merged
Conversation
Grok 4.5 is the only model routed through xAI's Responses API, and every turn died immediately with AI_TypeValidationError at ["response","usage"]. xAI opens each stream with a response.created event carrying "usage": null; @ai-sdk/xai@2.0.51 declares that field optional rather than nullable, so the first SSE event fails every branch of the chunk union and the stream aborts before any content arrives. Patch the pinned dependency to make usage nullish, matching the upstream fix in @ai-sdk/xai@4.0.25. Taking 4.x directly is not viable yet: it needs @ai-sdk/provider@4.0.4 against our pinned 2.0.1, two majors of the shared spec across ~20 packages. The patch header and the provider.ts comment both record that it should be deleted when that migration lands. Covered by an offline regression test that replays the null-usage response.created through the real provider via a canned fetch.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Member
|
KB (@KB-syntheticsciences) looks good, please make sure ci-cd are green and not failing! merge after :) |
5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
Fixes Grok 4.5 (xAI BYOK), which failed on every prompt with
AI_TypeValidationErrorbefore emitting any content. Grok 4.2 and every other xAI model were unaffected.provider.ts:394routes only Grok 4.5 through xAI's Responses API (sdk.responses()); everything else usessdk.languageModel()(chat). So only 4.5 is validated against the Responses schema.xAI opens every stream with:
{"type":"response.created","response":{ …, "usage": null, "status":"in_progress", … }}@ai-sdk/xai@2.0.51declares that field as required, and theresponse.created/response.in_progressunion members relax it with.partial({ usage: true }):.partial()makes a field optional — it accepts absent. xAI sends it present and null, and Zod's.optional()does not acceptnull; that needs.nullable()/.nullish(). So the very first SSE event fails every branch of the chunk union withpath: ["response","usage"] — expected object, received null, and the stream aborts. That is why the failure was immediate and why the UI showed an empty / compacted context.This is fixed upstream.
@ai-sdk/xai@4.0.25changes exactly one thing in that schema:This PR applies that same one-line change as a pinned dependency patch, using the convention already established here — bun
patchedDependencieswith the patch undertooling/patches/, alongside the existing@ai-sdk/anthropicand@ai-sdk/openaipatches.Why patch instead of upgrading the SDK?
@ai-sdk/xai@4.xrequires@ai-sdk/provider@4.0.4against the pinned2.0.1— two majors of the interface every provider implements — dragging ~20@ai-sdk/*packages plusaicore, and anything implementing the model interfaces (transform.ts, the custom loaders inprovider.ts).That upgrade is worth doing and is being scheduled separately. It should not be the vehicle for a user-facing regression fix: bundling them makes the fix unverifiable (a regression in, say, Bedrock could not be attributed), and Grok 4.5 stays broken for the length of the migration.
The patch header and a comment at
provider.ts:394both record that 4.0.25 fixes this upstream and that the patch should be deleted when the SDK migration lands.How did you verify your code works?
Live, against a real xAI key, rebuilt binary, isolated config and data:
AI_TypeValidationErrornaming["response","usage"], exit 1.OK, exit 0.Confirmed twice — once by the implementer, once independently from a fresh build.
Chat path unaffected: the same probe against a non-4.5 xAI model succeeds before and after.
Offline regression test (
backend/cli/test/provider/xai-responses-usage.test.ts) feeds aresponse.createdchunk carrying"usage": nullthrough the real provider via a stubbedfetch— no network, no key. Mutation-verified: with the patch reverted it fails with exactly the production error; restored, it passes.Suite: 1723 pass / 2 fail — both on the repo's known pre-existing list (
npm selects every supported native package contract…, andkillTree SIGKILLs a detached group, re-verified passing standalone). Typecheck 7/7 green.Notes for the reviewer
patchedDependencieskeys on the exact version string (@ai-sdk/xai@2.0.51). Any bump within 2.x silently drops the patch with no install-time error, and Grok 4.5 breaks again. The regression test is the only guard — worth remembering when the SDK migration is planned.response.created, so it also loosensresponse.done/response.completed. Consequence: a nullusageon a terminal event would degrade to zero-usage rather than erroring. Deliberate — staying identical to upstream keeps the eventual migration a clean deletion.bun install --frozen-lockfile.Checklist
bun run typecheckpassesbun test(inbackend/cli) passes — modulo the pre-existing failures noted abovebunx prettier --checkis clean on changed files