Skip to content

fix(tanstack-ai): handle image ContentPart in workers-ai adapter#435

Merged
threepointone merged 2 commits intocloudflare:mainfrom
mdhruvil:mdhruvil/fix-tanstack-ai-handle-image-content-part
Mar 18, 2026
Merged

fix(tanstack-ai): handle image ContentPart in workers-ai adapter#435
threepointone merged 2 commits intocloudflare:mainfrom
mdhruvil:mdhruvil/fix-tanstack-ai-handle-image-content-part

Conversation

@mdhruvil
Copy link
Contributor

@mdhruvil mdhruvil commented Mar 4, 2026

Fix: Workers AI adapter drops type: "image" content parts

Closes #431

Problem

TanStack AI's multimodal content uses ContentPart types where images are represented as:

{
  type: "image",
  source: { type: "url" | "data", value: string, mimeType?: string }
}

The workers-ai adapter's buildUserContent function only handled the legacy type: "image_url" format (workers-ai.ts#L98-L122), which TanStack AI never sends. This caused all image parts to be silently dropped from user messages.

The detection logic on line 98 correctly checked for both type: "image_url" and type: "image", but the conversion logic on lines 112-122 only handled type: "image_url" — so images were detected as present but never converted.

Solution

Rewrote the message-building helpers to use TanStack AI's canonical types directly, following the patterns from the reference OpenAI adapter (as recommended by TanStack's community adapter guide).

Type changes

  • Removed the ad-hoc MessageLike interface — replaced with ModelMessage from @tanstack/ai
  • Removed the inline Array<{ type: string; content?: string; image_url?: unknown }> content type — replaced with ModelMessage["content"] which is string | null | Array<ContentPart>
  • Imported ContentPart and ModelMessage from @tanstack/ai (already a peer dependency)

convertContentPart (new)

Extracted a dedicated function that converts a single ContentPartOpenAI.Chat.ChatCompletionContentPart:

Input Output
type: "text" { type: "text", text: part.content }
type: "image", source.type: "url" { type: "image_url", image_url: { url: source.value } }
type: "image", source.type: "data" { type: "image_url", image_url: { url: "data:{mimeType};base64,{value}" } }
audio / video / document console.warn + skip (Workers AI chat API doesn't support these)

Includes a guard against source.value already containing a data: prefix (matching the reference adapter's pattern) to prevent malformed double-prefixed URIs.

Files changed

File Change
packages/tanstack-ai/src/adapters/workers-ai.ts Core fix: use ContentPart/ModelMessage, handle type: "image"
packages/tanstack-ai/test/message-builder.test.ts Update image tests to ContentPart format
packages/tanstack-ai/test/workers-ai-adapter.test.ts Update image integration test, add data source test

@changeset-bot
Copy link

changeset-bot bot commented Mar 4, 2026

🦋 Changeset detected

Latest commit: 7381171

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@cloudflare/tanstack-ai Patch

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@mdhruvil mdhruvil changed the title fix(tanstack-ai): handle image ContentPart in workers-ai adapter (#431) fix(tanstack-ai): handle image ContentPart in workers-ai adapter Mar 4, 2026
buildUserContent only handled type:"image_url" - TanStack AI
sends type:"image" with source field, so image parts were silently dropped.
- replace MessageLike with ModelMessage, ad-hoc content type with ContentPart
- extract convertContentPart(): image source.type "url"/"data" → OpenAI image_url
- guard against already-prefixed data URIs (matches reference OpenAI adapter)
- warn + skip unsupported modalities (audio/video/document)
- remove dead type:"image_url" codepath and type assertion
- update tests to ContentPart format, add edge cases
@threepointone threepointone force-pushed the mdhruvil/fix-tanstack-ai-handle-image-content-part branch from 561b1fc to 7381171 Compare March 18, 2026 23:32
@pkg-pr-new
Copy link

pkg-pr-new bot commented Mar 18, 2026

Open in StackBlitz

npx https://pkg.pr.new/cloudflare/ai/ai-gateway-provider@435
npx https://pkg.pr.new/cloudflare/ai/@cloudflare/tanstack-ai@435
npx https://pkg.pr.new/cloudflare/ai/workers-ai-provider@435

commit: 7381171

Copy link
Collaborator

@threepointone threepointone left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm, thanks!

@threepointone threepointone merged commit e46dfe3 into cloudflare:main Mar 18, 2026
3 checks passed
@github-actions github-actions bot mentioned this pull request Mar 18, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

@cloudflare/tanstack-ai workers-ai adapter drops image parts (type: "image")

2 participants