VLM pipeline returns 400 Bad Request against local OpenAI-compatible server (LM Studio) – LM Studio logs “Channel Error”

[homerokzam]

Bug

When running VlmPipeline with ApiVlmOptions pointing to a local OpenAI-compatible endpoint (LM Studio, /v1/chat/completions), the pipeline fails with requests.exceptions.HTTPError: 400 Client Error: Bad Request. On the LM Studio side, the server prints Error: Channel Error. I’m unsure whether Docling’s VLM request shape is incompatible with LM Studio’s OpenAI shim, or if I’m misconfiguring ApiVlmOptions.

Steps to reproduce

Minimal snippet (imports and flow trimmed for brevity; replace "some.pdf" with any PDF):

from docling.datamodel.base_models import InputFormat from docling.datamodel.pipeline_options import VlmPipelineOptions from docling.datamodel.pipeline_options_vlm_model import ApiVlmOptions, ResponseFormat from docling.document_converter import DocumentConverter, PdfFormatOption from docling.pipeline.vlm_pipeline import VlmPipeline

def lms_vlm_options(model: str, prompt: str, format: ResponseFormat):
return ApiVlmOptions(
url="http://localhost:1234/v1/chat/completions",
params=dict(model=model),
prompt=prompt,
timeout=90,
scale=1.0,
response_format=format,
)

pipeline_options = VlmPipelineOptions(
vlm_options=lms_vlm_options(
model="smoldocling-256m-preview-mlx-docling-snap",
prompt="Convert this page to docling.",
format=ResponseFormat.DOCTAGS,
),
enable_remote_services=True,
)

converter = DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(
pipeline_cls=VlmPipeline,
pipeline_options=pipeline_options,
),
}
)

doc = converter.convert(source="some.pdf").document
md = doc.export_to_markdown()
print(md[:500])


Expected behavior
• The VLM request succeeds and Docling returns the converted document/markdown, or
• Docling surfaces a clear, actionable error indicating what part of the request is incompatible (e.g., payload shape for images, unsupported parameter, wrong endpoint).

Actual behavior
Client side:
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://localhost:1234/v1/chat/completions

LM Studio server log:
[LM STUDIO SERVER] Running chat completion on conversation with 1 messages.
[ERROR] Error: Channel Error

Docling version

docling 2.50.0

Python version

Python 3.12.7

[dosubot]

The 400 Bad Request and "Channel Error" from LM Studio are likely caused by Docling's VLM pipeline sending a request payload or parameters that LM Studio's OpenAI-compatible endpoint doesn't support. Docling constructs the payload with a multimodal messages array containing a base64-encoded image and prompt, and passes all key-value pairs from ApiVlmOptions.params (including custom fields like response_format, scale, etc.) directly to the endpoint. If LM Studio doesn't support these fields or strict multimodal formats, it will reject the request with a 400 error details.

To troubleshoot:

• Try removing nonstandard fields (like response_format, scale, etc.) from params and only include those required by LM Studio (usually just model).
• Make sure LM Studio supports OpenAI's multimodal vision API and the payload shape Docling sends (image + prompt in messages).
• If LM Studio doesn't support multimodal requests, Docling's VLM pipeline won't work with it until LM Studio adds support or Docling adds a compatibility mode reference.

If you need strict compatibility, consider testing with a backend known to support OpenAI's multimodal API, like vllm or llama-server.

To reply, just mention @dosu.

---

Docs are dead. Just use Dosu.

[homerokzam]

@dosu thanks for the quick reply!

A few clarifications and follow-ups from my side:
• I’m indeed pointing Docling to LM Studio’s OpenAI-compatible endpoint at http://localhost:1234/v1/chat/completions (LM Studio 0.3.24).
• In my code, I only set params. The other fields (response_format, scale, prompt, timeout) are passed as attributes of ApiVlmOptions, not inside params.

Environment
• Python: 3.12.7
• Docling: 2.50.0
• LM Studio: 0.3.24
• Model: smoldocling-256m-preview-mlx-docling-snap
• Endpoint: http://localhost:1234/v1/chat/completions
• ResponseFormat.DOCTAGS, enable_remote_services=True

What I’m trying to understand
1. Even if I don’t include response_format/scale in params, does Docling still forward those fields to the server (e.g., merge into the request body or query string)?
2. What is the exact request payload shape that Docling’s VLM pipeline sends for a PDF page (i.e., how images + prompt are encoded in messages[].content[])? A small JSON example would help me validate compatibility with LM Studio’s OpenAI shim.
3. Does Docling’s VLM pipeline officially support Chat Completions (/v1/chat/completions) for multimodal, or should I target Responses API (/v1/responses) for vision use cases?

Troubleshooting plan (based on your guidance)
• Keep params minimal ({"model": "…"}"), and if Docling forwards non-standard fields elsewhere, is there a toggle to disable forwarding (e.g., ignore unknown fields)?
• If Docling requires an OpenAI multimodal implementation that LM Studio doesn’t fully support, I can test with a backend known to support image parts (e.g., vLLM, llama-server) to confirm the pipeline.
• If Docling prefers /v1/responses for VLM, I can switch the url accordingly — please confirm if that’s the recommended path.

Repro snippet (same as before, pared down)

from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import VlmPipelineOptions
from docling.datamodel.pipeline_options_vlm_model import ApiVlmOptions, ResponseFormat
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.pipeline.vlm_pipeline import VlmPipeline

def lms_vlm_options(model: str, prompt: str, format: ResponseFormat):
return ApiVlmOptions(
url="http://localhost:1234/v1/chat/completions",
params=dict(model=model), # minimal
prompt=prompt,
timeout=90,
scale=1.0, # not in params
response_format=format, # not in params
)

pipeline_options = VlmPipelineOptions(
vlm_options=lms_vlm_options(
model="smoldocling-256m-preview-mlx-docling-snap",
prompt="Convert this page to docling.",
format=ResponseFormat.DOCTAGS,
),
enable_remote_services=True,
)

converter = DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(
pipeline_cls=VlmPipeline,
pipeline_options=pipeline_options,
),
}
)

doc = converter.convert(source="some.pdf").document
md = doc.export_to_markdown()
print(md[:500])


Observed
requests.exceptions.HTTPError: 400 Client Error: Bad Request for url: http://localhost:1234/v1/chat/completions

LM Studio log:

[LM STUDIO SERVER] Running chat completion on conversation with 1 messages.
[ERROR] Error: Channel Error

[dosubot]

Here’s what’s happening under the hood:

1. Field forwarding: Only fields you put in params (like model) are sent in the request body. Attributes like response_format, scale, prompt, and timeout are not included in the payload unless you explicitly add them to params. timeout is used only for the HTTP request itself, not sent to the server. There’s no toggle to filter out unknown fields—just control what you put in params (source, source).

2. Request payload shape: For each PDF page, Docling sends a payload like:

   {
     "messages": [
       {
         "role": "user",
         "content": [
           {
             "type": "image_url",
             "image_url": { "url": "data:image/png;base64,<...>" }
           },
           {
             "type": "text",
             "text": "<your prompt>"
           }
         ]
       }
     ],
     "model": "<model-name>"
   }

   Only fields in params (like model) are merged into the payload. The image is base64-encoded as a PNG (source).

3. Endpoint support: Docling does not distinguish between /v1/chat/completions and /v1/responses—it just sends the payload to the URL you provide. Compatibility depends on whether the backend supports OpenAI’s multimodal chat completions schema. LM Studio may not fully support this, which can cause the 400 error you’re seeing (source).

Best practice: Keep params minimal (just model and any required OpenAI fields). If LM Studio doesn’t support this multimodal schema, try vLLM or llama-server, which are known to work with Docling’s remote VLM pipeline (source).

Let me know if you want to see the full request/response logs or need help adapting for a specific backend.

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.

[homerokzam]

@dosu thanks — that clarifies a lot.

Just to be explicit about my setup: Docling is the one assembling and sending the payload; in my code I only supply params={"model": ...} and I do not add response_format/scale/… to params. So the request reaching the backend should be the multimodal chat completions JSON you described (image as a data:image/png;base64,… part + prompt + model in the body).

[homerokzam]

@dosu

Just to close the loop: I tested again using the exact same code and setup, but changed the model from
smoldocling-256m-preview-mlx-docling-snap

to
smoldocling-256m-preview-mlx

With that change, the pipeline worked correctly — the PDF was converted and I got Markdown output as expected.

So it looks like the issue is specific to the -docling-snap variant under LM Studio, not with Docling’s request shape itself.

[dosubot]

That's exactly right—the issue is specific to the -docling-snap model variant. That variant uses the VLLM inference framework (which expects CUDA support), while the working smoldocling-256m-preview-mlx model uses the MLX framework (optimized for MPS/Apple Silicon). LM Studio 0.3.24 may not fully support VLLM-based models, which explains the 400 error and "Channel Error" logs you saw. If you ever need to run the -docling-snap variant, you'd likely need a backend with full VLLM support (like vLLM or llama-server) rather than LM Studio in its current form details.

If you want to dig deeper into why the snap variant fails under LM Studio, I can help look at logs or configuration. Otherwise, your current setup with the MLX variant is the right approach. Let me know if you want to troubleshoot further!

To reply, just mention @dosu.

---

Share context across your team and agents. Try Dosu.