ISSUE DESCRIPTION
I encountered a severe performance problem when using llama-server with MTMD and a Vision model (Qwen3.5‑9B‑Vision). The same model and the same image run extremely fast in llama-cli, but become unusably slow in llama-server, where Vision encoding takes over 82 seconds before any tokens are generated.
This problem affects only llama-server (MTMD path).
llama-cli processes the same image almost instantly.
Image Used for Test
7.44 Mb , ~4k resolution
https://freenaturestock.com/bird-calling-from-a-tree/
ENVIRONMENT
CPU: AMD Ryzen 9 7950X
GPU: 7900 xtx
OS: FreeBSD/Linux (both checked, same delay)
llama.cpp: latest commit (fresh clone today)
Model:
Qwen3.5-9B-Q4_K_M.gguf
mmproj-Qwen3.5-9B-BF16.gguf
COMMAND USED
llama-server \
--jinja \
--threads 30 \
--gpu-layers 999 \
--ctx-size 131072 \
--batch-size 2048 \
--ubatch-size 512 \
--temperature 0.7 \
--top-k 20 \
--top-p 0.8 \
--min-p 0.0 \
--repeat-penalty 1.0 \
--presence-penalty 1.5 \
--flash-attn off \
--reasoning off \
--reasoning-budget 0 \
--host 0.0.0.0 \
--port 8080 \
--mmproj mmproj-Qwen3.5-9B-BF16.gguf \
-m Qwen3.5-9B-Q4_K_M.gguf
LOG OUTPUT (critical part)
encoding image slice...
image slice encoded in 82047 ms
Full Context log
srv process_chun: processing image...
encoding image slice...
image slice encoded in 82047 ms
decoding image batch 1/2, n_tokens_batch = 2048
image decoded (batch 1/2) in 945 ms
decoding image batch 2/2, n_tokens_batch = 2008
image decoded (batch 2/2) in 821 ms
srv process_chun: image processed in 83813 ms
Root Cause
mtmd_encode_chunk(ctx, chunk) -> called from mtmd_helper_eval_chunk_single()
This function performs the Vision encoding step for MTMD, and in the current implementation it appears to run the full Vision encoder (mmproj + vision tower) on CPU in BF16 for every image slice sequentially without GPU acceleration This explains why llama-cli is fast (different pipeline, GPU-friendly), while llama-server (MTMD) is extremely slow.
Version
version: 8994 (aab68217b)
Actual Behaviour
llama-cli: fast, image processed almost instantly
llama-server (MTMD): 82 seconds spent inside mtmd_encode_chunk()
Vision inference becomes unusable
Expected Behaviour
Vision encoding in llama-server should be comparable to llama-cli or at least not take 82 seconds per image MTMD Vision path should use GPU when available BF16 mmproj should not run on CPU if GPU layers are enabled.
ISSUE DESCRIPTION
I encountered a severe performance problem when using llama-server with MTMD and a Vision model (Qwen3.5‑9B‑Vision). The same model and the same image run extremely fast in llama-cli, but become unusably slow in llama-server, where Vision encoding takes over 82 seconds before any tokens are generated.
This problem affects only llama-server (MTMD path).
llama-cli processes the same image almost instantly.
Image Used for Test
7.44 Mb , ~4k resolution
https://freenaturestock.com/bird-calling-from-a-tree/
ENVIRONMENT
CPU: AMD Ryzen 9 7950X
GPU: 7900 xtx
OS: FreeBSD/Linux (both checked, same delay)
llama.cpp: latest commit (fresh clone today)
Model:
Qwen3.5-9B-Q4_K_M.gguf
mmproj-Qwen3.5-9B-BF16.gguf
COMMAND USED
LOG OUTPUT (critical part)
encoding image slice...
image slice encoded in 82047 ms
Full Context log
Root Cause
mtmd_encode_chunk(ctx, chunk) -> called from mtmd_helper_eval_chunk_single()
This function performs the Vision encoding step for MTMD, and in the current implementation it appears to run the full Vision encoder (mmproj + vision tower) on CPU in BF16 for every image slice sequentially without GPU acceleration This explains why llama-cli is fast (different pipeline, GPU-friendly), while llama-server (MTMD) is extremely slow.
Version
version: 8994 (aab68217b)Actual Behaviour
llama-cli: fast, image processed almost instantly
llama-server (MTMD): 82 seconds spent inside mtmd_encode_chunk()
Vision inference becomes unusable
Expected Behaviour
Vision encoding in llama-server should be comparable to llama-cli or at least not take 82 seconds per image MTMD Vision path should use GPU when available BF16 mmproj should not run on CPU if GPU layers are enabled.