[Bug] dsh-llm-pi-ai sends non png/jpeg/gif images as-is; LM Studio rejects webp with 400 "'url' field must be a base64 encoded image."

[PeterXu1984]

Summary

When a user pastes an image into the GUI, dsh-llm-pi-ai serializes it as
data:<mimeType>;base64,<data> and sends it to the configured backend.
Strict OpenAI-compatible backends (LM Studio) only accept
image/png | jpeg | gif data URIs; webp, bmp, heic, etc. are rejected at
the protocol layer with HTTP 400 "'url' field must be a base64 encoded image.".

Environment

• dsh: 0.1.1-rc.2
• Provider: llm-pi-ai, api: openai-completions, baseURL → LM Studio
• Model: qwen/qwen3.8-27b with input: [text, image]

Steps to reproduce

1. Configure a local LM Studio provider (any vision-capable model).
2. Paste a webp image into a chat message (copied from a browser or a web
   page screenshot — very common).
3. Send. The request fails:

400 "'url' field must be a base64 encoded image."

Root cause

Verified directly against the backend — the rejection is a mime whitelist
check, not a base64 validity check:

data URI prefix	LM Studio result
data:image/png;base64,…	accepted (proceeds to engine decoding)
data:image/jpeg;base64,…	accepted
data:image/gif;base64,…	accepted
data:image/webp;base64,…	400 "'url' field must be a base64 encoded image."
data:image/bmp;base64,…	400 (same)
data:image/heic;base64,…	400 (same)

DSH correctly provides base64 bytes; the problem is only the mime type.
dsh-llm-pi-ai userContent() pushes
{ type: "image", data: <base64>, mimeType: <original> }, which pi-ai then
serializes to data:<original>;base64,… — so any format outside the whitelist
fails on every request.

Suggested fix

In the pi-ai adapter (or in dsh-llm-pi-ai conversion), transcode image
content that is not image/png|jpeg|gif to PNG before sending. sharp is
already available in the profile dependency tree (verified:
sharp(webpBuf).png().toBuffer() round-trips correctly).

At minimum, document this backend limitation.

Workaround (local)

We transcode in userContent() before pushing the image block (sharp, with a
best-effort fallback that keeps original bytes); happy to PR this if useful.

[denial123789]

The reproduced matrix exposes a capability gap that is slightly broader than conversion: rc.2 has three different image capability owners, but only one is modeled on the provider route.

• attachment-local advertises/stores PNG, JPEG, WebP, and GIF. That is a storage contract.
• input: [text, image] is a model modality gate. It says nothing about MIME, animation, dimensions, encoded bytes, or the provider wire shape.
• llm-pi-ai/src/context.ts reads the durable object and forwards its original mediaType; there is no format negotiation before pi-ai serialization.

So I agree that conversion can contain LM Studio incompatibility, but I would not use a catch-and-send-original fallback. Once the route is known to reject WebP, a converter failure should remain a typed local failure; sending the original turns a useful decoder/resource error back into the misleading remote 400.

I would also keep the original content-addressed attachment immutable and create a derived rendition keyed by source digest plus target MIME, encoder/version, size limits, animation policy, orientation/color rules, and quality. Animated WebP/GIF needs an explicit first-frame/all-frame/reject policy; PNG conversion is not semantically neutral. The route should declare supported MIME types so unsupported input can fail before egress when conversion is unavailable or disallowed.

I mapped the current rc.2 path, safe rendition boundary, decoder limits, and a provider conformance matrix here: https://sandbaseai.github.io/deepseek-harness-handbook/pi-ai-image-media-type.html

The LM Studio accept/reject matrix remains attributed to this report; I independently verified the DSH storage and conversion source.

Disclosure: I maintain the independent community handbook linked above.