Qwen3 VL doesn't work with images

[Willian7004]

Which version of LM Studio?
LM Studio 0.3.30(Build2)
runtime: V1.55.0

Which operating system?
Kali GNU/Linux Rolling x86_64

..............                                      william@william
            ..,;:ccc,.                              ---------------
          ......''';lxO.                            OS: Kali GNU/Linux Rolling x86_64
.....''''..........,:ld;                            Host: NHx0ERQ_EPS
           .';;;:::;,,.x,                           Kernel: Linux 6.16.8+kali-amd64
      ..'''.            0Xxoc:,.  ...               Uptime: 2 hours, 56 mins
  ....                ,ONkc;,;cokOdc',.             Packages: 2332 (dpkg), 7 (flatpak), 2 (snap)
 .                   OMo           ':ddo.           Shell: zsh 5.9
                    dMc               :OO;          Display (BOE0A2D): 2560x1440 @ 1.15x in 16", 165 Hz
                    0M.                 .:o.        DE: GNOME 48.4
                    ;Wd                             WM: Mutter (X11)
                     ;XO,                           WM Theme: Kali-Dark
                       ,d0Odlc;,..                  Theme: Fusion [Qt], adw-gtk3-blue-dark [GTK2/3/4]
                           ..',;:cdOOd::,.          Icons: Flat-Remix-Blue-Dark [Qt], Flat-Remix-Blue-Dark [GTK2/3/4]
                                    .:d;.':;.       Font: Cantarell (11pt) [GTK2/3/4]
                                       'd,  .'      Cursor: Adwaita (24px)
                                         ;l   ..    Terminal: GNOME Terminal 3.58.0
                                          .o        Terminal Font: Fira Code Medium (10pt)
                                            c       CPU: AMD Ryzen 5 5600H (12) @ 4.28 GHz
                                            .'      GPU 1: NVIDIA GeForce RTX 3070 Mobile / Max-Q [Discrete]
                                             .      GPU 2: AMD Radeon Vega Series / Radeon Vega Mobile Series [Integrated]
                                                    Memory: 29.34 GiB / 62.16 GiB (47%)
                                                    Swap: 8.00 KiB / 59.60 GiB (0%)
                                                    Disk (/): 76.05 GiB / 870.98 GiB (9%) - btrfs
                                                    Disk (/sda1): 255.21 GiB / 931.51 GiB (27%) - btrfs
                                                    Local IP (Meta): 198.18.0.1/30
                                                    Locale: zh_CN.UTF-8

What is the bug?
Qwen3 VL clashs without additional imformation.

Screenshots

Logs

2025-11-01 14:13:27 [DEBUG]
 llama_context: constructing llama_context
llama_context: n_seq_max     = 1
llama_context: n_ctx         = 16384
llama_context: n_ctx_per_seq = 16384
llama_context: n_batch       = 512
llama_context: n_ubatch      = 512
llama_context: causal_attn   = 1
llama_context: flash_attn    = enabled
llama_context: kv_unified    = false
llama_context: freq_base     = 5000000.0
llama_context: freq_scale    = 1
llama_context: n_ctx_per_seq (16384) < n_ctx_train (262144) -- the full capacity of the model will not be utilized
2025-11-01 14:13:27 [DEBUG]
 llama_context:  CUDA_Host  output buffer size =     0.58 MiB
2025-11-01 14:13:27 [DEBUG]
 llama_kv_cache:      CUDA0 KV buffer size =  1224.00 MiB
2025-11-01 14:13:27 [DEBUG]
 llama_kv_cache: size = 1224.00 MiB ( 16384 cells,  36 layers,  1/1 seqs), K (q8_0):  612.00 MiB, V (q8_0):  612.00 MiB
2025-11-01 14:13:27 [DEBUG]
 llama_context:      CUDA0 compute buffer size =   352.77 MiB
llama_context:  CUDA_Host compute buffer size =    40.02 MiB
llama_context: graph nodes  = 1267
llama_context: graph splits = 2
2025-11-01 14:13:27 [DEBUG]
 common_init_from_params: added <|endoftext|> logit bias = -inf
common_init_from_params: added <|im_end|> logit bias = -inf
common_init_from_params: added <|fim_pad|> logit bias = -inf
common_init_from_params: added <|repo_name|> logit bias = -inf
common_init_from_params: added <|file_sep|> logit bias = -inf
common_init_from_params: setting dry_penalty_last_n to ctx_size = 16384
common_init_from_params: warming up the model with an empty run - please wait ... (--no-warmup to disable)
2025-11-01 14:13:27 [DEBUG]
 GgmlThreadpools: llama threadpool init = n_threads = 4
2025-11-01 14:13:27 [DEBUG]
 clip_model_loader: model name:   Qwen3-Vl-8B-Instruct
clip_model_loader: description:  
clip_model_loader: GGUF version: 3
clip_model_loader: alignment:    32
clip_model_loader: n_tensors:    352
clip_model_loader: n_kv:         31

clip_model_loader: has vision encoder
2025-11-01 14:13:27 [DEBUG]
 clip_ctx: CLIP using CUDA0 backend
2025-11-01 14:13:27 [DEBUG]
 load_hparams: projector:          qwen3vl_merger
load_hparams: n_embd:             1152
load_hparams: n_head:             16
load_hparams: n_ff:               4304
load_hparams: n_layer:            27
load_hparams: ffn_op:             gelu
load_hparams: projection_dim:     4096

--- vision hparams ---
load_hparams: image_size:         1024
load_hparams: patch_size:         16
load_hparams: has_llava_proj:     0
load_hparams: minicpmv_version:   0
load_hparams: proj_scale_factor:  0
load_hparams: n_wa_pattern:       0
load_hparams: spatial_merge_size: 2

load_hparams: model size:         1108.69 MiB
load_hparams: metadata size:      0.12 MiB
2025-11-01 14:13:27 [DEBUG]
 alloc_compute_meta:      CUDA0 compute buffer size =     3.00 MiB
alloc_compute_meta:        CPU compute buffer size =     0.19 MiB
2025-11-01 14:13:50 [DEBUG]
 Sampling params:	repeat_last_n = 64, repeat_penalty = 1.100, frequency_penalty = 0.000, presence_penalty = 0.000
	dry_multiplier = 0.000, dry_base = 1.750, dry_allowed_length = 2, dry_penalty_last_n = -1
	top_k = 40, top_p = 0.950, min_p = 0.050, xtc_probability = 0.000, xtc_threshold = 0.100, typical_p = 1.000, top_n_sigma = -1.000, temp = 0.100
	mirostat = 0, mirostat_lr = 0.100, mirostat_ent = 5.000
2025-11-01 14:13:50 [DEBUG]
 Sampling: 
logits -> logit-bias -> penalties -> dry -> top-n-sigma -> top-k -> typical -> top-p -> min-p -> xtc -> temp-ext -> dist 
Generate: n_ctx = 16384, n_batch = 512, n_predict = -1, n_keep = 16
2025-11-01 14:13:50 [DEBUG]
 Total prompt tokens: 1038
Prompt tokens to decode: 1038
BeginProcessingPrompt
2025-11-01 14:13:50 [DEBUG]
 PromptProcessing: 0.578035
2025-11-01 14:13:50 [DEBUG]
 Embedding image for model arch: qwen3vl
2025-11-01 14:13:50 [DEBUG]
 ggml_backend_cuda_buffer_type_alloc_buffer: allocating 1216.06 MiB on device 0: cudaMalloc failed: out of memory
ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 1275133952
llama.cpp abort:1851: GGML_ASSERT((char *)addr + ggml_backend_buffer_get_alloc_size(buffer, tensor) <= (char *)ggml_backend_buffer_get_base(buffer) + ggml_backend_buffer_get_size(buffer)) failed
2025-11-01 14:13:50 [DEBUG]
 /home/william/.lmstudio/extensions/backends/llama.cpp-linux-x86_64-nvidia-cuda12-avx2-1.55.0/libggml-base.so(+0x1816b) [0x7fe6384b216b]
/home/william/.lmstudio/extensions/backends/llama.cpp-linux-x86_64-nvidia-cuda12-avx2-1.55.0/libggml-base.so(ggml_print_backtrace+0x210) [0x7fe6384b25d0]
/home/william/.lmstudio/extensions/backends/llama.cpp-linux-x86_64-nvidia-cuda12-avx2-1.55.0/libggml-base.so(ggml_abort+0x152) [0x7fe6384b27a2]
/home/william/.lmstudio/extensions/backends/llama.cpp-linux-x86_64-nvidia-cuda12-avx2-1.55.0/libggml-base.so(ggml_backend_tensor_alloc+0xdd) [0x7fe6384cdddd]
/home/william/.lmstudio/extensions/backends/llama.cpp-linux-x86_64-nvidia-cuda12-avx2-1.55.0/libggml-base.so(ggml_gallocr_alloc_graph+0x2b0) [0x7fe6384c7810]
/home/william/.lmstudio/extensions/backends/llama.cpp-linux-x86_64-nvidia-cuda12-avx2-1.55.0/libggml-base.so(ggml_backend_sched_alloc_graph+0x14f) [0x7fe6384cda1f]
/home/william/.lmstudio/extensions/backends/llama.cpp-linux-x86_64-nvidia-cuda12-avx2-1.55.0/libmtmd.so(_Z23clip_image_batch_encodeP8clip_ctxiPK20clip_image_f32_batchPf+0xba) [0x7fe6385c86da]
/home/william/.lmstudio/extensions/backends/llama.cpp-linux-x86_64-nvidia-cuda12-avx2-1.55.0/libmtmd.so(mtmd_encode+0x99) [0x7fe638559a39]
/home/william/.lmstudio/extensions/backends/llama.cpp-linux-x86_64-nvidia-cuda12-avx2-1.55.0/libllm_engine.so(_ZN10llm_engine10LlamaModel10embedImageERNS_15LlamaImageStateE+0xe2) [0x7fe6386a80b2]
/home/william/.lmstudio/extensions/backends/llama.cpp-linux-x86_64-nvidia-cuda12-avx2-1.55.0/libllm_engine.so(_ZN10llm_engine10LlamaModel25getImageAtProcessingLevelENS_20ImageProcessingLevelEi+0xbd) [0x7fe6386a857d]
/home/william/.lmstudio/extensions/backends/llama.cpp-linux-x86_64-nvidia-cuda12-avx2-1.55.0/libllm_engine.so(_ZN10llm_engine7LlamaV316_predictInternalERKNS_15PredictionInputERKNS_16PredictionConfigESt8functionIFvRKNS_14SamplingOutputEEES7_IFvNS_9EventTypeERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEE+0xbd7) [0x7fe638696077]
/home/william/.lmstudio/extensions/backends/llama.cpp-linux-x86_64-nvidia-cuda12-avx2-1.55.0/libllm_engine.so(_ZN10llm_engine7LlamaV37predictERKNS_15PredictionInputERKNS_16PredictionConfigESt8functionIFvRKNS_14SamplingOutputEEES7_IFvNS_9EventTypeERKNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEEE+0x434) [0x7fe63869b284]
2025-11-01 14:13:50 [DEBUG]
 /home/william/.lmstudio/extensions/backends/llama.cpp-linux-x86_64-nvidia-cuda12-avx2-1.55.0/llm_engine_cuda12.node(_ZN13PredictWorker7ExecuteEv+0xbc) [0x7fe63873d66c]
/home/william/.lmstudio/extensions/backends/llama.cpp-linux-x86_64-nvidia-cuda12-avx2-1.55.0/llm_engine_cuda12.node(_ZN4Napi11AsyncWorker18OnAsyncWorkExecuteEP10napi_env__Pv+0x36) [0x7fe63873cb46]
2025-11-01 14:13:50 [DEBUG]
 /home/william/.lmstudio/.internal/utils/node(_ZZN4node14ThreadPoolWork12ScheduleWorkEvENUlP9uv_work_sE_4_FUNES2_+0x59) [0xc86539]
/home/william/.lmstudio/.internal/utils/node() [0x18c3860]
/lib/x86_64-linux-gnu/libc.so.6(+0x92b7b) [0x7fe63b09cb7b]
/lib/x86_64-linux-gnu/libc.so.6(+0x1107b8) [0x7fe63b11a7b8]

To Reproduce

1. This doesn't happen when images are inputted.
2. Using Vulkan or CUDA12 runtime will reproduce this, and the prompt processing sticks on 1% when using CPU runtime.
3. The model I use is IQ4_XS version of unsloth/Qwen3-VL-8B-Instruct-GGUF and unsloth/Qwen3-VL-8B-Thinking-GGUF .

[Willian7004]

Qwen3 VL 4b works.

[Sal0ID]

For me it works only with low res images ( e.g. 770x175 ) but if I try something bigger ( even 773x541 ) it throws the same error.

[Willian7004]

Qwen3 VL 4b works.  

I tested again and the 4b version doesn't work again. Maybe it only works at the first time.

[Sal0ID]

I figured it out:

You need more VRAM to create image embedding.

Line from your log file:

"""
ggml_backend_cuda_buffer_type_alloc_buffer: allocating 1216.06 MiB on device 0: cudaMalloc failed: out of memory
ggml_gallocr_reserve_n: failed to allocate CUDA0 buffer of size 1275133952
"""

[rjb2112]

Similar issue, I believe, in GUI.
LM Studio 0.3.31
Mac OS 15.7.2
Qwen3 VL fails when image attached using image attach tool. .JPG of chess board attached using tool. Simple query "What is this?" Results in:
"Failed to send message
Context overflow policy error: TruncateMiddle context overflow policy is not currently supported for prompts with images. Please increase the model's loaded context length, shorten the prompt, or utilize a different context overflow policy."
Same error occurs when just trying to load image with no query.

2025-11-07 15:21:49 [INFO]
[Plugin(lmstudio/rag-v1)] stdout: [PromptPreprocessor][Request (QahM+a)] Preprocess request completed.
2025-11-07 15:21:49 [DEBUG]
[lmstudio-llama-cpp] Error in predictTokens: Context overflow policy error: TruncateMiddle context overflow policy is not currently supported for prompts with images. Please increase the model's loaded context length, shorten the prompt, or utilize a different context overflow policy.
2025-11-07 15:21:49 [DEBUG]
1 Error predicting: _0x3844d5 [Error]: Context overflow policy error: TruncateMiddle context overflow policy is not currently supported for prompts with images. Please increase the model's loaded context length, shorten the prompt, or utilize a different context overflow policy.
at _0x597699..predictTokens (/Applications/LM Studio.app/Contents/Resources/app/.webpack/lib/llmworker.js:85:30177)
at async _0x3a5274.predictTokens (/Applications/LM Studio.app/Contents/Resources/app/.webpack/lib/llmworker.js:113:17873)
at async _0x3a5274.handleMessage (/Applications/LM Studio.app/Contents/Resources/app/.webpack/lib/llmworker.js:113:7072) {
cause: undefined,
suggestion: undefined,
errorData: undefined,
data: undefined,
displayData: undefined,
title: "Context overflow policy error: TruncateMiddle context overflow policy is not currently supported for prompts with images. Please increase the model's loaded context length, shorten the prompt, or utilize a different context overflow policy."
}

[Willian7004]

It works with runtime v1.57.1

[rjb2112]

I was just about to tuck in for the night and saw this message, so ran to the machine to try it. AWESOME! Thank you for your efforts! Have a great weekend! - Rob

…

On Fri, Nov 14, 2025 at 11:43 PM Willian7004 ***@***.***> wrote: *Willian7004* left a comment (lmstudio-ai/lmstudio-bug-tracker#1176) <#1176 (comment)> It works with runtime v1.57.1 — Reply to this email directly, view it on GitHub <#1176 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AGCLI6IKNBAVVHL2YINPEDD3424R3AVCNFSM6AAAAACK2ZHN4WVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTKMZWGA2DINBTG4> . You are receiving this because you commented.Message ID: ***@***.***>