`@cloudflare/tanstack-ai`: non-stream binding fetch discards OpenAI-shaped `binding.run` responses — structured output fails with "expected object, received string"

[tien]

Summary

With @cloudflare/tanstack-ai 0.2.1, a native Workers AI model, and the binding transport, chat({ outputSchema }) fails with a misleading validation error:

Error: Validation failed: Invalid input: expected object, received string
  code: 'structured-output-validation-failed'

The model is not at fault — it returns a valid, schema-conformant object. The adapter's non-stream response wrapper discards that output, because binding.run returns an OpenAI-shaped chat.completion body and the wrapper only understands the native { response: ... } shape. This is a regression against the equivalent workers-ai-provider + AI SDK setup, whose response extraction handles both shapes.

Reproducer

import { chat } from "@tanstack/ai";
import { createWorkersAiChat } from "@cloudflare/tanstack-ai";
import { z } from "zod";

const adapter = createWorkersAiChat("@cf/meta/llama-4-scout-17b-16e-instruct", {
  binding: env.AI,
  gateway: env.CF_AIG_ID,
});

await chat({
  adapter,
  systemPrompts: [systemPrompt],
  messages: [{ role: "user", content: userMessage }],
  outputSchema: z.object({ id: z.string().nullable() }),
});

Actual

Error: Validation failed: Invalid input: expected object, received string
  code: 'structured-output-validation-failed'
    at runAgenticStructuredOutput (@tanstack/ai/dist/esm/activities/chat/index.js:1671)
  cause: StandardSchemaValidationError
    at parseWithStandardSchema (@tanstack/ai/.../tools/schema-converter.js:157)
    at TextEngine.runStructuredFinalization (@tanstack/ai/.../chat/index.js:1303)

Root cause

wrapWorkersAiResultAsOpenAI (packages/tanstack-ai/src/utils/create-fetcher.ts), on the non-stream path of createWorkersAiBindingFetch:

const responseObj = typeof result === "object" && result !== null ? result : { response: String(result) };
const message = {
  role: "assistant",
  content: typeof responseObj.response === "string" ? responseObj.response : /* … */ "",
};

The wrapper only understands the native { response: ... } shape. But binding.run for this model (messages + response_format) returns an OpenAI-shaped chat.completion body — {"choices":[{"message":{"content":"{\"id\":\"A1\"}"}}], "object":"chat.completion", …} (vLLM passthrough fields like routed_experts included). That body has no .response key, so:

1. content becomes "" — the model's valid constrained output is discarded
2. in structuredOutput, JSON.parse("") throws → catch { data = rawText } → data = "", a string
3. → Validation failed: Invalid input: expected object, received string

The failure is deterministic for any model/route whose run endpoint returns the OpenAI shape.

Notably, the streaming transformer in the same file already handles this exact case:

if (parsed.choices !== undefined) { isOpenAiFormat = true; /* pass through */ }

The non-stream wrapper just never got the equivalent. It's the mirror image of #543 (native shape reaching a consumer that expects OpenAI shape — here an OpenAI shape reaching a wrapper that expects native).

Ruled out: the { name, strict, schema } envelope the adapter sends as response_format.json_schema. Tested live via the AI Gateway workers-ai/run/<model> REST endpoint and via binding.run directly — both the envelope and a bare JSON Schema engage constrained decoding correctly on this model, and the sibling workers-ai-provider distinction from #559 is not the issue here.

A second defect: the silent parse fallback

adapters/workers-ai.ts → structuredOutput does:

try {
  data = JSON.parse(rawText);
} catch {
  data = rawText;   // ← a failed parse becomes the raw string, with no signal
}

This is what turned an empty-content bug into a confusing type error about the user's schema. Whatever happens upstream of it, a non-JSON reply should not silently flow onward as a string — surfacing rawText in the error would have made the root cause obvious from the first stack trace.

Fix (verified locally)

Pass OpenAI-shaped bodies through untouched, mirroring the stream transformer (patched dist locally; chat({ outputSchema }) then returns the validated object against the live model):

function wrapWorkersAiResultAsOpenAI(result, model, salvageContext) {
  if (typeof result === "object" && result !== null && Array.isArray(result.choices)) {
    return new Response(JSON.stringify(result), { headers: { "Content-Type": "application/json" } });
  }
  // …existing native-shape wrapping…
}

Happy to turn this into a PR if useful.

Environment

• @cloudflare/tanstack-ai: 0.2.1
• @tanstack/ai: 0.42.0
• Model: @cf/meta/llama-4-scout-17b-16e-instruct
• Transport: direct binding + gateway id ({ binding: env.AI, gateway: env.CF_AIG_ID })
• Runtime: Cloudflare Workers

Related

• @cloudflare/tanstack-ai: Workers AI gateway-binding mode returns native response shape, breaks structured output (and streaming) #543 — the same shape mismatch on the gateway-binding path, in the opposite direction