Vision models retrun error "Model does not support images. Please use a model that does"

[mikhailkis]

Which version of LM Studio?
Example: LM Studio 0.3.8

Which operating system?
Windows 10 x64

What is the bug?
All models that supposed to process images failes with error: "Error: Model does not support images. Please use a model that does." when I try send an image

Screenshots
If applicable, add screenshots to help explain your problem.

Logs
[2025-01-28 10:23:43.616] [error] [LMSInternal][Client=LM Studio][Endpoint=sendMessage] Error in RPC handler: Error: Rehydrated error
support images. Please use a model that does.
at _0x398b8f. (D:\LMSTUDIO\LM Studio\resources\app.webpack\main\index.js:408:104227)
at _0x1f1ce6.subscriber (D:\LMSTUDIO\LM Studio\resources\app.webpack\main\index.js:81:1785)
at _0x1f1ce6.notifier (D:\LMSTUDIO\LM Studio\resources\app.webpack\main\index.js:242:135159)

• Caused By: Error: Channel Error
  at (D:\LMSTUDIO\LM Studio\resources\app.webpack\main\index.js:103:62933)
  at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
• Caused By: Error: received prediction-error
  at _0x398b8f. (D:\LMSTUDIO\LM Studio\resources\app.webpack\main\index.js:408:104227)
  at _0x19257a._0x2b6fbc (D:\LMSTUDIO\LM Studio\resources\app.webpack\main\index.js:24:270226)
  at _0x19257a.emit (node:events:519:28)
  at _0x19257a.onChildMessage (D:\LMSTUDIO\LM Studio\resources\app.webpack\main\index.js:24:244659)
  at _0x19257a.onChildMessage (D:\LMSTUDIO\LM Studio\resources\app.webpack\main\index.js:24:284260)
  at ForkUtilityProcess. (D:\LMSTUDIO\LM Studio\resources\app.webpack\main\index.js:24:243659)
  at ForkUtilityProcess.emit (node:events:519:28)
  at ForkUtilityProcess.a.emit (node:electron/js2c/browser_init:2:71823)
• Caused By: Error: Model does not support images. Please use a model that does.
  at _0x1ed565. (D:\LMSTUDIO\LM Studio\resources\app.webpack\lib\llmworker.js:7:761123)
  at async _0x334b09..predictTokens (D:\LMSTUDIO\LM Studio\resources\app.webpack\lib\llmworker.js:9:43638)
  at async Object.predictTokens (D:\LMSTUDIO\LM Studio\resources\app.webpack\lib\llmworker.js:14:10581)
  at async Object.handleMessage (D:\LMSTUDIO\LM Studio\resources\app.webpack\lib\llmworker.js:14:2019)

To Reproduce
Steps to reproduce the behavior:

1. Go to 'Chat' with any vision model (for example xtuner/llava-phi-3-mini-gguf)
2. Click on 'Add Image'
3. Press 'Enter'
4. See error

[quincy451]

I have run into this as well, any progress.

[arhontes-netizen]

Same here, it works in the GUI but fails via API. On two models so far: openbmb/MiniCPM-o-2_6-gguf/Model-7.6B-Q4_0.gguf and openbmb/MiniCPM-V-2_6-gguf/ggml-model-Q3_K_L.gguf.

The API call used is:

def vLLM_prompt(input_image_path, model_name, prompt):

    client = OpenAI(base_url="http://localhost:1234/v1", api_key="lm-studio")

    with open(input_image_path, "rb") as image:
        base64_string = base64.b64encode(image.read()).decode("utf-8")

    base64_value = f"data:image/png;base64,{base64_string}"

    response = client.chat.completions.create(
        model=model_name,
        messages=[{
            "role": "user",
            "content": [
                {"type": "text", "text": prompt},
                {"type": "image_url", "image_url": {"url": base64_value}}
            ]}
        ],
        stream=False
    )

    output = response.choices[0].message.content

    return output

The error message is:

BadRequestError: Error code: 400 - {'error': 'Model does not support images. Please use a model that does.'}

[arhontes-netizen]

Update: In my case it was a matter of too low context length. I use dynamic tokenization logic and multiply the token value a few times to get a reasonable context length.

But manually setting it to several thousand seems to work, though I have no clue why, since the base64 strings can have hundreds of thousands or millions of tokens.

Not sure what counts as part of it and what does not.

[quincy451]

No joy for me...I moved maxtokens to 65535 and still no response...I tried changing to the model your most likely using and still no response... My code looks like this...I might be doing something wrong...############################################# New function to handle image + text############################################def send_image_and_prompt(    system_prompt,    user_prompt,    image_path,    host="127.0.0.1",    port=1234,#    model="qwen2-vl-7b-instruct",    model="minicpm-o-2_6",    temperature=0.7,    max_tokens=65535,  # <-- Increase default here as well    stream=False):    """    Sends system_prompt, user_prompt, and an image to a service    that can handle visual+language tasks.    """    if not os.path.exists(image_path):        raise FileNotFoundError(f"Image file not found: {image_path}")     # Example: base64-encode the image.    with open(image_path, "rb") as f:        image_bytes = f.read()    encoded_image = base64.b64encode(image_bytes).decode("utf-8")     # The URL or endpoint for the vision model may differ from the text model:    api_url = f"http://{host}:{port}/v1/chat/completions"     payload = {        "model": model,        "messages": [            {"role": "system", "content": system_prompt},            {"role": "user", "content": user_prompt},        ],        "image": encoded_image,        "temperature": temperature,        "max_tokens": max_tokens,        "stream": stream    }     headers = {"Content-Type": "application/json"}    try:        response = requests.post(api_url, headers=headers, json=payload, timeout=60)        response.raise_for_status()        data = response.json()         choices = data.get("choices", [])        if choices:            generated_text = choices[0]["message"]["content"]        else:            generated_text = "[No content returned]"                return generated_text     except requests.RequestException as e:        raise RuntimeError(f"Request to visual endpoint failed: {str(e)}")    except ValueError as e:        raise RuntimeError(f"Error processing response from visual endpoint: {str(e)}") On Monday, February 10, 2025 at 12:37:20 PM CST, arhontes-netizen ***@***.***> wrote: Update: In my case it was a matter of too low context length. I use dynamic tokenization logic and multiply the token value a few times to get a reasonable context length. But manually setting it to several thousand seems to work, though I have no clue why, since the base64 strings can have hundreds of thousands or millions of tokens. Not sure what counts as part of it and what does not. — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you commented.Message ID: ***@***.***>

[Jaron-Wilson]

im using the api, the new python version, tried how lmstudio says to do it and get this error: ing. (Type 'quit' to exit)
{'status': 'error', 'message': "Failed to analyze image: 'Client' object has no attribute 'files'"}
with lms.Client() as client:
# Prepare the image for analysis
image_handle = client.files.prepare_image(image_path)
model = client.llm.model("qwen2-vl-2b-instruct")
chat = lms.Chat()
chat.add_user_message("Describe this image please", images=[image_handle])
prediction = model.respond(chat)
return {"status": "success", "response": prediction.text}