A benchmark comparison of three popular GGUF quantizations of Qwen 3.6-35B-A3B running on the GMKtec EVO-X2 mini-PC, using llama.cpp Vulkan backend. Includes a cross-platform comparison with NVIDIA DGX Spark CUDA results.
Related work
- Previous: Gemma 4 vs Qwen 3.5 on EVO-X2 (Qiita in Japanese: link)
- Yesterday: Qwen 3.6 vs 3.5 on DGX Spark
- This work (Japanese version): Qiita article
- Tested 3 Qwen 3.6 quantizations on EVO-X2: bartowski Q4_K_M, Unsloth UD-Q4_K_M, and Unsloth MXFP4_MOE.
- bartowski Q4_K_M is the best daily-driver on EVO-X2 (fastest tg, fastest VLM, smallest size).
- MXFP4_MOE wins at prompt processing (
pp) but loses at token generation — great for RAG, bad for chat. - Long-context scaling on Vulkan is 15× worse than CUDA (−37% vs −2.4% degradation from pp512 to pp16384).
- Thinking mode is the big news: Qwen 3.5 regressed −10.73 pt with Thinking ON; Qwen 3.6 now gains +0.89 pt on the same hardware — a 11.62 pt reversal.
- With Thinking ON, Qwen 3.6 bartowski Q4_K_M reaches 96.87% on JCommonsenseQA, beating Gemma 4 Q4_K_M Thinking (95.80%) — a new best text-quality setup on EVO-X2.
- VLM multimodal quality is excellent — all 3 quantizations hit 100% JSON/PPE parse rate and can even read text displayed on monitor screens inside photos.
| Item | Value |
|---|---|
| Hardware | GMKtec EVO-X2 (AMD Ryzen AI Max+ 395, Radeon 8060S / gfx1151) |
| Memory | 128 GB LPDDR5X (UMA), BIOS VGM set to 96 GB |
| OS | Ubuntu 25.10 (kernel 6.18.20) |
| llama.cpp | b8838 (Vulkan backend) |
| Comparison platform | DGX Spark (NVIDIA GB10) + llama.cpp b8672 CUDA |
The previous article used llama.cpp b8672; this one is on b8838 to match Qwen 3.6's release window. The pp2048 numbers for Qwen 3.5 Q4_K_M went from 730 (b8672) to 624 (b8838), but that falls within the prior measurement's noise band (±104 t/s).
A plain libvulkan-dev install is no longer enough:
fatal error: spirv/unified1/spirv.hpp: No such file or directory
Somewhere between b8672 and b8838, SPIR-V header dependency was added. You need spirv-headers too:
sudo apt install -y libvulkan-dev spirv-headers
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp && git checkout b8838
cmake -B build-vulkan-b8838 -DGGML_VULKAN=ON -DCMAKE_BUILD_TYPE=Release
cmake --build build-vulkan-b8838 --config Release -j$(nproc)python3 -m venv ~/venv-bench
source ~/venv-bench/bin/activate
pip install huggingface_hub datasets httpx
# Unsloth (UD-Q4_K_M + MXFP4_MOE + mmproj)
hf download unsloth/Qwen3.6-35B-A3B-GGUF \
--include "*UD-Q4_K_M*.gguf" \
--include "*MXFP4_MOE*.gguf" \
--include "mmproj-F16.gguf" \
--local-dir ~/models/Qwen3.6-35B-A3B/
# bartowski plain Q4_K_M
hf download bartowski/Qwen_Qwen3.6-35B-A3B-GGUF \
--include "Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf" \
--local-dir ~/models/Qwen3.6-35B-A3B/gfx1151 is bf16: 0, so use mmproj-F16.gguf rather than the BF16 variant.
llama-bench -m <model.gguf> -ngl 99 -fa 1 -mmp 0 -p 2048 -n 32 -ub 2048 -r 3
| Model | Size | pp2048 (t/s) | tg32 (t/s) |
|---|---|---|---|
| Qwen 3.5 Q4_K_M (prev article / b8672) | 19.71 GiB | 730 | 72.9 |
| Qwen 3.6 bartowski Q4_K_M | 19.91 GiB | 625 | 67.5 |
| Qwen 3.6 Unsloth UD-Q4_K_M | 20.60 GiB | 642 | 60.9 |
| Qwen 3.6 Unsloth MXFP4_MOE | 20.21 GiB | 815 | 50.8 |
pp (prompt processing) processes your input prompt on the GPU; tg (token generation) generates one token at a time. Chat feels faster with high tg; RAG/long-context workloads care about pp.
The striking result is that MXFP4_MOE is +30% faster on pp but −25% slower on tg compared to Q4_K_M. DGX Spark showed MXFP4 winning both, but on EVO-X2 Vulkan the asymmetry is dramatic. bartowski Q4_K_M has the highest tg (67.5 t/s) and smallest size (19.91 GiB), making it the natural daily-chat choice.
For Q4_K_M, pp is essentially unchanged (measurement noise) and tg drops from 72.9 to 67.5 (−7%). Yesterday's DGX Spark article showed ±0.5% difference, so EVO-X2 sees slightly worse tg — likely because Qwen 3.6 has a 3:1 GDN:Attention layer ratio, and GDN falls back to CPU on Vulkan.
Yesterday's DGX Spark article highlighted improved long-context pp scaling for Qwen 3.6. What about EVO-X2 Vulkan?
| Model | pp512 | pp4096 | pp16384 | Degradation (512→16K) |
|---|---|---|---|---|
| Qwen 3.6 bartowski Q4_K_M | 1017 | 638 | 636 | −37% |
| Qwen 3.6 Unsloth UD-Q4_K_M | 1022 | 626 | 613 | −40% |
| Qwen 3.6 Unsloth MXFP4_MOE | 955 | 627 | 618 | −35% |
| (reference) DGX Spark CUDA MXFP4_MOE | 2268 | 2292 | 2215 | −2.4% |
DGX Spark is essentially flat (−2.4%), while EVO-X2 Vulkan loses 35–40% going from 512 to 16,384 tokens. Past pp4096 it saturates, reflecting the -ub 2048 ubatch limit.
The "Qwen 3.6 improves long-context degradation by 2.6×" story from the DGX Spark article does not transfer to EVO-X2 Vulkan. Budget ~600 t/s when designing 16K-token RAG pipelines on EVO-X2.
Likely causes: Vulkan's FLASH_ATTN_EXT scaling, MoE batch efficiency, and the 10-of-40 Gated Attention layers. llama.cpp Issue #21284 discusses gfx1151-specific prefill optimization that might land upstream and change this picture.
Curiously, at pp512, MXFP4_MOE (955) is slower than Q4_K_M (1017). MXFP4 wins at pp2048 (ubatch-sized batch) but loses at sub-ubatch lengths — the format's advantage is batch-size dependent.
Quality on Japanese common-sense reasoning using JCommonsenseQA v1.1: 1,119 5-choice questions, 3-shot prompt.
| Model | Accuracy | Avg tok/s |
|---|---|---|
| Qwen 3.5 Q4_K_M (prev article) | 94.64% | — |
| Qwen 3.6 bartowski Q4_K_M | 95.98% | 97.6 |
| Qwen 3.6 Unsloth UD-Q4_K_M | 96.07% | 88.9 |
| Qwen 3.6 Unsloth MXFP4_MOE | 95.53% | 86.8 |
From 94.64% → 95.98%. The Qwen 3.6 bartowski Q4_K_M on EVO-X2 matches the DGX Spark Qwen 3.6 MXFP4_MOE result exactly (95.98%), confirming that model quality is insensitive to hardware and quantization at this level.
The three quantizations span 95.53–96.07% — a 0.54 pt (6-question) spread. Unsloth UD's +0.09 pt edge over bartowski is noise. UD claims SOTA quantization, but its 10% tok/s penalty makes it unattractive for EVO-X2 daily use; plain Q4_K_M is the rational choice.
The previous EVO-X2 article reported that Qwen 3.5 Q4_K_M with Thinking ON lost 10.73 pt — "thinking harder" actually hurt accuracy, as the reasoning trace drifted away from the correct initial intuition.
Running the same benchmark on Qwen 3.6 produces the opposite: +0.89 pt improvement. An 11.62 pt swing. The gain is also larger than DGX Spark's (+0.45 pt), so on this metric EVO-X2 Q4_K_M beats DGX Spark MXFP4_MOE (1–2 question difference, well within noise, but not behind).
| Model / Quantization | Platform | Thinking OFF | Thinking ON | Delta |
|---|---|---|---|---|
| Qwen 3.5 Q4_K_M | EVO-X2 (prev article) | 94.64% | 83.91% | −10.73 pt |
| Qwen 3.5 MXFP4_MOE | DGX Spark | 96.16% | 89.28% | −6.88 pt |
| Qwen 3.6 MXFP4_MOE | DGX Spark (yesterday) | 95.98% | 96.43% | +0.45 pt |
| Qwen 3.6 bartowski Q4_K_M | EVO-X2 (this article) | 95.98% | 96.87% | +0.89 pt |
The DGX Spark article already showed Thinking Preservation helping on that platform. EVO-X2 confirms the fix is platform- and quantization-independent.
In the previous article, Gemma 4 Q4_K_M Thinking ON scored 95.80%. Qwen 3.6 bartowski Q4_K_M Thinking ON at 96.87% beats it by 1.07 pt.
The previous article's "Gemma 4 is the best on EVO-X2" held for nothink usage. For Thinking-heavy workloads, Qwen 3.6 bartowski Q4_K_M is now the new leader.
Thinking ON generates a <think>...</think> reasoning trace before answering:
| Mode | Avg latency | Avg tok/s |
|---|---|---|
| nothink (answer only) | 0.520s | 97.6 |
| think (reasoning + answer) | 9.169s | 65.2 |
17.6× latency increase. The lower tok/s (vs 97.6 for nothink) reflects longer outputs (400–1000 tokens with thinking vs a few characters without). Full run: about 3 hours.
Running Qwen 3.6 with mmproj-F16 in production-grade setup.
Previously I ran llama-server manually. This time I set up systemd services for easy switching:
# /etc/systemd/system/llama-server-q36-q4km.service
[Unit]
Description=llama-server Qwen3.6-35B-A3B bartowski Q4_K_M + Vision
After=network.target
Conflicts=llama-server-q6k.service llama-server-q4km.service
[Service]
Type=simple
User=nabe
ExecStart=/home/nabe/llama.cpp/build-vulkan-b8838/bin/llama-server \
-m /home/nabe/models/Qwen3.6-35B-A3B/Qwen_Qwen3.6-35B-A3B-Q4_K_M.gguf \
--mmproj /home/nabe/models/Qwen3.6-35B-A3B/mmproj-F16.gguf \
-ngl 99 -fa on --no-mmap \
-c 32768 -np 2 --port 8080 --host 0.0.0.0 \
--jinja --reasoning off --reasoning-budget 0
Restart=on-failure
[Install]
WantedBy=multi-user.targetThe Conflicts= line ensures the existing daily Qwen 3.5 service stops automatically. One service per quantization makes sudo systemctl start llama-server-q36-mxfp4 a one-command model switch.
Unsloth discussions/1 reported that Qwen 3.6 VLM segfaults during image processing under llama.cpp. Yesterday's DGX Spark article noted the issue did not reproduce; I checked EVO-X2 Vulkan with all three quantizations, and no segfault occurred (bartowski Q4_K_M + mmproj-F16, UD-Q4_K_M, and MXFP4_MOE all work).
vlm_bench.py hits the llama-server OpenAI-compatible endpoint with base64-encoded images. Core:
def call_vlm(client, api_url, model, image_path, prompt, max_tokens=1024):
b64 = image_to_base64(image_path)
mime = get_mime(image_path)
messages = [
{"role": "user", "content": [
{"type": "image_url", "image_url": {"url": f"data:{mime};base64,{b64}"}},
{"type": "text", "text": prompt}
]}
]
resp = client.post(api_url, json={
"model": model, "messages": messages,
"max_tokens": max_tokens, "temperature": 0.0,
})
return resp.json()Prompt (Japanese): "Describe this image in detail in Japanese. Include location, people, objects, and atmosphere."
Five test images from Japanese trade shows and conferences. Outputs are typically 1000+ tokens of structured Japanese description.
bartowski Q4_K_M is fastest on every single image; MXFP4_MOE is slowest. Long outputs amplify the tg-speed gap from llama-bench.
Full Japanese captions are in results/vlm_qwen36_bartowski_q4km.json. Brief English summaries of what the model extracted:
| Image | What the caption correctly extracted |
|---|---|
 |
1Finity Inc. booth at CEATEC 2025. Read the product plate "1FINITY T250 Hitless Protection Demo", noted yellow fiber-optic cables indicating a live demo, parsed fine text on a QR code stand ("distribution material QR code / survey takes 1–2 minutes"), mentioned the "DWDM Solution" brochure. Latency: 18.2s. |
 |
Delta Electronics, identified the "AI Containerized Datacenter" display, inferred cooling design from the "Hot Aisle" label, quoted the tagline "Delta Supports You All the Way", and described visitors' clothing and actions in detail. Latency: 13.5s. |
 |
Netskope Japan booth. Quoted the banner copy accurately, then read the Gemini browser window on the SONY monitor — identifying the user's prompt ("Tell me about Netskope's good points"), and understanding that Gemini's response explained SASE. Also parsed the social-media hashtag campaign poster. Latency: 16.3s — the most information-dense image. |
 |
GMO INTERNET × NVIDIA joint exhibit. Identified the NVIDIA Blackwell Ultra GPU / HGX™ B300 platform through the transparent display case, noted staff uniforms ("GMO 1" black shirt), and spotted the "Lenovo" booth in the background. Latency: 13.9s. |
 |
STech I Forum 2025 with Yoichi Ochiai presenting. Listed 10+ job titles from the screen (Media Artist, PhD, Associate Professor at University of Tsukuba, CEO of Pixie Dust Technologies, Expo 2025 producer, etc.) without missing any. Picked up thematic keywords like "Digital Nature" and "3D Visual-Tactile Display". Latency: 18.5s — heaviest Japanese text load. |
What stood out most was the Netskope photo. The model read the Gemini chat interface displayed on the booth's monitor — understanding it as a Gemini session, identifying the user's Japanese input query, and summarizing the AI's response about SASE. Reading small text inside a browser window rendered in a photograph, and interpreting it as an LLM conversation, is a notable level of multimodal comprehension.
Getting this kind of structured Japanese output from a single image, running locally on compact hardware, is genuinely useful.
Prompt: extract structured JSON (location, event type, subjects, technologies, people count, atmosphere) from the image.
Example output for the DELTA booth photo:
{
"location": "Exhibition venue (likely an international technology trade show)",
"event_type": "Business exhibition / product launch",
"subjects": [
"Delta's AI containerized datacenter",
"Server rack system",
"Cooling system (Hot Aisle marker visible)"
],
"technologies": ["Delta", "AI Containerized Datacenter", "Hot Aisle"],
"people_count": 0,
"atmosphere": "Professional, modern, futuristic IT infrastructure display"
}(Translated from the Japanese output; see results/vlm_qwen36_bartowski_q4km.json for the original.)
All three quantizations achieved 100% JSON parse success — matching Qwen 3.5's reliability.
A practical construction/manufacturing use case. The model analyzes worker PPE compliance in JSON form.
Example output for the forklift worker image:
{
"workers_count": 1,
"ppe_items": [{
"worker_id": 1,
"hard_hat": true,
"safety_vest": true,
"safety_glasses": false,
"gloves": true,
"safety_shoes": true,
"other": ["headphones", "earplugs (possible)"]
}],
"compliance_score": "mid",
"observations": "Worker wears basic PPE (hard hat, vest, gloves, shoes) but no safety glasses. Headphones also observed."
}Each piece of equipment is returned as a boolean; the overall compliance is rated and the observations field flags the missing safety glasses. Great for automated safety monitoring prototypes.
All three quantizations hit 100% parse success again.
| Task | Qwen 3.5 Q4_K_M | Qwen 3.6 bartowski | Qwen 3.6 UD | Qwen 3.6 MXFP4 |
|---|---|---|---|---|
| Caption avg | 15.56s | 16.06s | 17.98s | 19.19s (slowest) |
| JSON extraction | 5.54s | 5.16s (fastest) | 5.72s | 5.66s |
| PPE detection | 6.00s | 5.91s (fastest) | 6.34s | 6.23s |
| Parse rate | 100% | 100% | 100% | 100% |
bartowski Q4_K_M is fastest or tied across all three tasks. Qwen 3.5 → 3.6 shows a 3% Caption slowdown (tg regression); JSON/PPE stays similar because of short outputs.
| Criterion | bartowski Q4_K_M | Unsloth UD-Q4_K_M | Unsloth MXFP4_MOE |
|---|---|---|---|
| Interactive speed (tg) | ◎ 67.5 | ○ 60.9 | △ 50.8 |
| VLM overall | ◎ | ○ | △ |
| JCQ nothink | 95.98% | 96.07% (+0.09 pt) | 95.53% |
| JCQ Thinking ON | ◎ 96.87% | (not tested) | (not tested) |
| RAG / long prompt (pp) | ○ | ○ | ◎ |
| Model size | ◎ 19.91 GiB | △ 20.60 GiB | ○ 20.21 GiB |
Pick bartowski Q4_K_M for chat and VLM. Switch to MXFP4_MOE only for pp-heavy batch workloads (RAG, long-document summarization). Thinking ON doesn't regress, so even reasoning-heavy use is safe.
UD-Q4_K_M doesn't justify itself on EVO-X2: the claimed +0.09 pt edge is noise and it's consistently 10% slower.
Previously I concluded that Gemma 4 dominated Qwen 3.5 on EVO-X2 across pp, VLM, and quality. Does Qwen 3.6 change that?
| Metric | Gemma 4 Q4_K_M (prev) | Qwen 3.6 bartowski Q4_K_M |
|---|---|---|
| pp2048 | 1,348 | 625 |
| tg32 | 65.4 | 67.5 |
| JCQ nothink | 96.16% | 95.98% |
| JCQ Thinking ON | 95.80% | 96.87% |
| VLM Caption | 10.29s | 16.06s |
| VLM JSON parse rate | 100% | 100% |
| Size | 15.6 GiB | 19.91 GiB |
Gemma 4 keeps its speed advantage: 2.2× faster pp, 1.5× faster VLM captioning, smaller model.
But Qwen 3.6 beats Gemma 4 by 1.07 pt on Thinking ON. Previously I noted Gemma 4's Thinking regression was unusually small (−0.71 pt). Qwen 3.6 improves (+0.89 pt) and ends up above.
On EVO-X2, the "just use Gemma 4" era ends. It's now:
- Speed / pp / VLM throughput → Gemma 4 Q4_K_M. Still no alternative for high pp workloads, bulk caption generation, or RAG.
- Reasoning / Thinking-mode / complex dialog → Qwen 3.6 bartowski Q4_K_M. New best quality for agents and analysis tasks.
With systemd Conflicts= service switching, both stay one systemctl start away.
- Compared 3 Qwen 3.6 quantizations on EVO-X2; bartowski Q4_K_M wins as daily driver.
- Unsloth UD-Q4_K_M doesn't justify its cost on this platform.
- MXFP4_MOE is niche — good for pp-heavy batch, bad for chat.
- Vulkan long-context pp degradation is 15× worse than CUDA — plan accordingly.
- VLM: 100% parse reliability, and the quality is strong enough to read on-screen text inside photos.
- Qwen 3.5 → 3.6: JCQ nothink +1.34 pt, Thinking ON −10.73 → +0.89 (11.62 pt swing).
- Thinking ON at 96.87% beats Gemma 4 Thinking (95.80%) — new top text quality on EVO-X2.
The previous "Gemma 4 only" conclusion now splits into a two-way use-case routing. For future updates, llama.cpp Issue #21284 (gfx1151 prefill optimization) and Ubuntu 26.04 LTS (scheduled for 4/23) RADV improvements may shift the picture again.
qwen36-evo-x2/
├── README.md # This article
├── LICENSE # MIT
├── scripts/
│ ├── jcq_bench.py # JCommonsenseQA runner
│ ├── vlm_bench.py # Vision-language runner
│ └── charts/ # Chart generation scripts
├── results/ # Benchmark JSON outputs
├── charts/ # PNG charts (6 files)
└── images/ # Test images (5 for caption, 3 for PPE)
- Previous article (Japanese, on EVO-X2): Qiita / GitHub
- Yesterday's article on DGX Spark (Japanese): Qiita
- This article in Japanese: Qiita
- Qwen 3.6-35B-A3B on HuggingFace
- Unsloth Qwen3.6-35B-A3B-GGUF
- bartowski Qwen3.6-35B-A3B-GGUF
- llama.cpp Issue #21284 (gfx1151 prefill optimization)
- JCommonsenseQA v1.1
MIT