Post your Arc results #1

Torchit1 · 2026-09-26T11:38:38Z

Torchit1
Sep 26, 2026
Maintainer

Running the arc-b580 branch? Post your card, driver, OS, context size and the numbers you get (llama-bench tg128/pp512, or the server's t/s). Results from other setups help most.

lavdrimismajili · 2026-09-27T21:17:56Z

lavdrimismajili
Sep 27, 2026

SYCL Flash Attention on Arc 140T (Xe-LPG): oneDNN SDPA returns wrong results for quantized KV with permuted views

Summary

On Intel Arc 140T (Xe-LPG+, Arrow Lake-H), the SYCL backend's oneDNN SDPA path returns numerically incorrect results when the KV cache is quantized, the layout is non-contiguous (permute=[0,2,1,3]), and the tensor is a padded view (kv_view=1). The native TILE kernel handles the same shape correctly. This matches upstream issue ggml-org#27769.

Separately, MTP + ngram-mod speculative decoding on SYCL leaks device memory and crashes with OUT_OF_RESOURCES at first prompt. Both are backend defects, not configuration errors.

Hardware / Software

GPU: Intel Arc 140T (12GB shared), Xe-LPG+
Driver: Level Zero 1.15.39183+4, sycl-ls reports 12.74
oneAPI: 2026.1.1 (DPC++ 20260101)
oneDNN: 3.13.3 (built from source, pointed via DNNL_ROOT / DNNL_DIR)
llama.cpp: ebf104ca2 (build 10767), PrismML / Mintplex fork with SYCL ternary kernels
Backend: SYCL, GGML_SYCL_F16=ON, GGML_SYCL_DNN=ON
Model: Ternary-Bonsai 2 27B PTQ1_0 (Qwen3.8-based, 1.75 bpw ternary, group 128)

`test-backend-ops` Results

With oneDNN and MKL FA enabled

OK   hsk=256,hsv=256,nh=2,nr23=[16,2],kv=1025,nb=1, q8_0, permute=[0,1,2,3]
FAIL hsk=256,hsv=256,nh=2,nr23=[16,1],kv=1025,nb=64, q8_0, permute=[0,2,1,3]
     ERR = 1.183620451 > 0.000500000
OK   hsk=256,hsv=256,nh=4,nr23=[6,1],kv=1025,nb=512, q4_0, permute=[0,1,2,3]
OK   hsk=128,hsv=128,nh=8,nr23=[4,1],kv=1025,nb=64,  q4_0, permute=[0,1,2,3]
OK   hsk=256,hsv=256,nh=4,nr23=[6,1],kv=1025,nb=512, q8_0, permute=[0,1,2,3]
OK   hsk=128,hsv=128,nh=8,nr23=[4,1],kv=1025,nb=64,  q8_0, permute=[0,1,2,3]

5/6 tests passed

With oneDNN and MKL FA disabled

6/6 tests passed

Same binary, same model, same flags. Only difference is GGML_SYCL_FA_ONEDNN=0 and GGML_SYCL_ENABLE_MKL_FA=0.

Reproduction

set GGML_SYCL_ENABLE_VMM=0
set GGML_SYCL_FA_ONEDNN=1
set GGML_SYCL_ENABLE_MKL_FA=1
build-sycl\bin\test-backend-ops.exe test -o FLASH_ATTN_EXT -b SYCL0 -p "kv=1025,"

Expected: 5/6, one FAIL with ERR ≈ 1.2–1.5
With both disabled: 6/6 PASS

Observed Error Magnitudes Across Runs

oneDNN version	ERR
oneAPI bundled (3.11)	1.26
3.13.3 (source build)	1.33
3.13.3	1.28
3.13.3	1.48
3.13.3	1.18
3.13.3	1.23

Upgrading oneDNN from 3.11 to 3.13.3 did not fix the bug. Error stays in the 1.2–1.5 range, roughly 2500× the 0.0005 tolerance.

`llama-bench` Results

Scope: Ternary-Bonsai 2 27B PTQ1_0 only. These numbers apply to this specific model on this specific fork. They should not be extrapolated to other models or other builds.

KV	ONEDNN	MKL	pp512	tg128
f16	0	0	167.39 ± 0.89	7.90 ± 0.03
q8_0	0	0	170.46 ± 1.33	7.85 ± 0.03
q8_0	1	1	174.03 ± 0.06	7.88 ± 0.01
f16	1	1	212.10 ± 0.84	7.93 ± 0.04

The F16 + oneDNN row is the fastest correct configuration for this model. F16 removes the quantized-KV precondition, so the broken kernel path is not taken.

The q8_0 + oneDNN row is fast but produces wrong attention output per the test-backend-ops failure above. It should not be used for correctness-sensitive work.

Server Findings

MTP memory leak

llama-server with --spec-type draft-mtp,ngram-mod crashes at ggml-sycl.cpp:2853 with UR_RESULT_ERROR_OUT_OF_RESOURCES on the first task launch. This persists with GGML_SYCL_ENABLE_VMM=0. It is separate from the FA bug.

VMM

GGML_SYCL_ENABLE_VMM=0 is required to avoid an earlier OUT_OF_RESOURCES failure at graph reserve. This aligns with the VMM disable-by-default behavior in PR ggml-org#24673 and PR ggml-org#27689.

ngram-mod behavior

ngram-mod only fires when the model echoes a long contiguous block from the prompt. In one session:

ngram-mod: #calls(b,g,a) = 3  356  2
#gen drafts = 2
#acc drafts = 2
#gen tokens = 512
#acc tokens = 301
mean acc len = 151.50

Hit rate 2/356 calls. When it fires, it carries 300+ tokens in 2 drafts. When it does not, MTP alone carries generation at ~4–7 t/s.

ngram-mod window size

n_max	Behavior
256	Catches long verbatim copies; mean accepted length up to 36
48	Catches short targeted edits; mean accepted length up to 25, acceptance up to 89%

Neither wins universally. A middle value (128) may capture both patterns.

Conclusion

The oneDNN SDPA path is numerically broken on Xe-LPG for quantized KV + permute=[0,2,1,3] + kv_view=1. Matches issue SYCL: flash attention returns a silently wrong answer for a quantised, non-contiguous KV view (Arc 140T, Xe-LPG) ggml-org/llama.cpp#27769, not fixed by oneDNN 3.13.3.
F16 KV avoids the bug and is the fastest correct configuration for this model on this fork.
MTP + ngram-mod on SYCL has a separate memory leak causing OUT_OF_RESOURCES.
GGML_SYCL_ENABLE_VMM=0 is required to avoid an earlier allocation failure.

Generated with DeepSeek <3

5 replies

Torchit1 Sep 27, 2026
Maintainer Author

Thanks for the detailed write-up, it was really useful. I reproduced the test-backend-ops failures on my B580 too: FLASH_ATTN_EXT with a quantized KV cache in the permuted layout fails from about 1K keys up, and it only passes with both oneDNN and MKL turned off. So it's not specific to the 140T.

I found the cause. The test builds its KV view with a gap between tokens (it allocates twice the heads and views half of them), and the oneDNN and MKL paths work out the token stride from the head count instead of reading it. llama.cpp's real cache has no gap, so actual models weren't affected. With Bonsai 27B, q4_0 KV and flash attention, perplexity at 4K context is the same on all three attention paths (oneDNN 8.727, MKL 8.731, the native TILE kernel 8.733), and needle-in-a-haystack recall is exact at 32K, 64K and 120K. I've added a guard so a layout with a gap falls back to the correct kernel. The full FLASH_ATTN_EXT suite now passes (4137/4137), plus a new sweep of the real cache layout from 1K to 8K keys. That fix will be in the next push. F16 KV is still a fine choice on the 140T if you have the memory for it.

On the MTP crash, I suspect memory rather than a leak. The draft context needs its own KV and compute buffers, and 12 GB of shared memory gets tight fast. Could you post your exact llama-server command? On the B580 I use LLAMA_ARG_SPEC_DRAFT_UBATCH=512 plus -ctkd q4_0 -ctvd q4_0 for the draft context, and on the 140T I'd start at -c 16384 and work up from there.

Also, the ternary XMX path only runs on Xe2 (B-series, Lunar Lake). On the 140T it switches itself off, so around 8 t/s is about what that memory bandwidth allows for a 27B model.

lavdrimismajili Sep 28, 2026

Thanks for the detailed write-up, it was really useful. I reproduced the test-backend-ops failures on my B580 too: FLASH_ATTN_EXT with a quantized KV cache in the permuted layout fails from about 1K keys up, and it only passes with both oneDNN and MKL turned off. So it's not specific to the 140T.

I found the cause. The test builds its KV view with a gap between tokens (it allocates twice the heads and views half of them), and the oneDNN and MKL paths work out the token stride from the head count instead of reading it. llama.cpp's real cache has no gap, so actual models weren't affected. With Bonsai 27B, q4_0 KV and flash attention, perplexity at 4K context is the same on all three attention paths (oneDNN 8.727, MKL 8.731, the native TILE kernel 8.733), and needle-in-a-haystack recall is exact at 32K, 64K and 120K. I've added a guard so a layout with a gap falls back to the correct kernel. The full FLASH_ATTN_EXT suite now passes (4137/4137), plus a new sweep of the real cache layout from 1K to 8K keys. That fix will be in the next push. F16 KV is still a fine choice on the 140T if you have the memory for it.

On the MTP crash, I suspect memory rather than a leak. The draft context needs its own KV and compute buffers, and 12 GB of shared memory gets tight fast. Could you post your exact llama-server command? On the B580 I use LLAMA_ARG_SPEC_DRAFT_UBATCH=512 plus -ctkd q4_0 -ctvd q4_0 for the draft context, and on the 140T I'd start at -c 16384 and work up from there.

Also, the ternary XMX path only runs on Xe2 (B-series, Lunar Lake). On the 140T it switches itself off, so around 8 t/s is about what that memory bandwidth allows for a 27B model.

Is it possible to run the ternary XMX for Xe-LPG+?
Since i can run qwen 3.8 27b at q4 K M at a cool 10t/s avg (12t/s) coding but takes all my ram to do it on Vulkan

Torchit1 Sep 28, 2026
Maintainer Author

Good question. Xe-LPG+ does have XMX (it's the main thing Arrow Lake-H added over Meteor Lake), so it's possible in principle, but not with the current code. The ternary and attention kernels are written for Xe2's XMX: 16-lane sub-groups and Xe2's 2D block loads. Xe-LPG+'s XMX comes from Alchemist (8-lane DPAS), so it would need separate kernel variants, and I don't have an Xe-LPG+ machine to build or test them on, so it's not something I'm planning. That's why the path switches itself off on the 140T.

A correction to what I said before, too: 8 t/s isn't the bandwidth ceiling. If you get 10-12 t/s from a ~16 GB Q4_K_M 27B, Bonsai at 5.85 GB should do much better than 8, so the plain SYCL kernels are leaving a lot on the table on the 140T. Worth trying the Vulkan build of this branch with the same Bonsai file (steps in the guide under "Build and run (Vulkan)") and posting both numbers here. One heads-up: PrismML lists PTQ1_0 on Intel Vulkan as having been slow and occasionally hanging after about 1,900 tokens, so if it stalls, that's the likely cause.

If you or anyone with Xe-LPG+ hardware wants to have a go at the XMX side, the Xe2 check is ggml_sycl_device_is_xe2() and the kernels are in ggml/src/ggml-sycl/. I'm happy to review a PR.

lavdrimismajili Sep 28, 2026

disabling VMM made it possible to run the model whereas before it didn't even start hence the crashes, I would like to see and try to fix the SYCL kernel but i don't know where to start since i don't know, maybe if you could guide me on the start, i could do something or find something.

Torchit1 Sep 30, 2026
Maintainer Author

Happy to point you in. One thing first: for generation speed, XMX probably isn't where the win is. Generating a token reads every weight once, so it's limited by memory bandwidth, and XMX mainly helps prompt reading and MTP draft checks. On the 140T the bigger opportunity is the plain PTQ1_0 mat-vec kernel it already uses, which is also much easier to work on.

Where to start:

Get a baseline. Run test-backend-ops perf -o MUL_MAT -b SYCL0 -p ptq1_0 (kernel only) and llama-bench -m -p 0 -n 128 (whole model). Work out the GB/s the kernel reaches from its timing and matrix size, and compare it with your RAM's bandwidth. That gap is what you're trying to close.
The kernel is in ggml/src/ggml-sycl/mmvq.cpp: the PTQ1_0 reorder path around line 800 (reorder_vec_dot_q_sycl<GGML_TYPE_PTQ1_0>) and the launch code around lines 1150 and 1460. Things worth trying on Xe-LPG: sub-group size (8 vs 16), rows per work-group, and making each work-item load weights in wider, contiguous chunks.
After every change, check it's still correct: test-backend-ops test -o MUL_MAT -b SYCL0 must pass, and a quick llama-perplexity run should match before and after.
Posting the Vulkan numbers for the same Bonsai file would give a useful reference point.

If you do want to try XMX later, the gate is ggml_sycl_device_is_xe2() in ggml/src/ggml-sycl/ptq1-t2.cpp (line 67), and those kernels live in ptq1-t2.cpp and ternsycl/. They'd need 8-lane DPAS variants for Xe-LPG+. I can't test on that hardware, but I'm happy to review whatever you come up with.

PotatoNut · 2026-09-30T12:05:07Z

PotatoNut
Sep 30, 2026

System info:
Asrock B580 12 GB - Graphics Driver 32.0.101.9030
CPU: Ryzen 7 7700
RAM 32 GB 6000 mt/s cl 30
OS Windows 11

Context size:
Unknown, the prompt incorrectly marks this text as 4 tokens:

You have two sand timers, which can show 4 minutes and 7 minutes respectively. Use both the sand timers(at a time or one after other or any other combination) and measure a time of 9 minutes.

In most local models I have tried, this task end up getting brute forced and often is in an infinite loop.

Result from the Terminal:

9.32.264.104 I slot release: id 0 | task 391 | stop processing: n_tokens = 6385, truncated = 0
9.32.264.147 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.059
9.32.383.713 I slot launch_slot_: id 0 | task 637 | processing task, is_child = 0
9.35.515.056 I slot print_timing: id 0 | task 637 | n_gen = 168, tg = 55.00 t/s, tg_3s = 55.33 t/s
9.38.545.317 I slot print_timing: id 0 | task 637 | n_gen = 332, tg = 54.56 t/s, tg_3s = 54.12 t/s
9.41.549.918 I slot print_timing: id 0 | task 637 | n_gen = 518, tg = 56.99 t/s, tg_3s = 61.91 t/s
9.44.593.857 I slot print_timing: id 0 | task 637 | n_gen = 691, tg = 56.95 t/s, tg_3s = 56.83 t/s
9.47.642.063 I slot print_timing: id 0 | task 637 | n_gen = 862, tg = 56.78 t/s, tg_3s = 56.10 t/s
9.50.655.480 I slot print_timing: id 0 | task 637 | n_gen = 1029, tg = 56.56 t/s, tg_3s = 55.42 t/s
9.53.671.737 I slot print_timing: id 0 | task 637 | n_gen = 1202, tg = 56.67 t/s, tg_3s = 57.36 t/s
9.56.720.684 I slot print_timing: id 0 | task 637 | n_gen = 1350, tg = 55.65 t/s, tg_3s = 48.54 t/s
9.59.748.650 I slot print_timing: id 0 | task 637 | n_gen = 1515, tg = 55.52 t/s, tg_3s = 54.49 t/s
10.02.756.850 I slot print_timing: id 0 | task 637 | n_gen = 1680, tg = 55.45 t/s, tg_3s = 54.85 t/s
10.05.764.248 I slot print_timing: id 0 | task 637 | n_gen = 1866, tg = 56.03 t/s, tg_3s = 61.85 t/s
10.09.312.509 I slot print_timing: id 0 | task 637 | n_gen = 2056, tg = 55.79 t/s, tg_3s = 53.55 t/s
10.12.340.111 I slot print_timing: id 0 | task 637 | n_gen = 2231, tg = 55.94 t/s, tg_3s = 57.80 t/s
10.15.891.599 I slot print_timing: id 0 | task 637 | n_gen = 2419, tg = 55.70 t/s, tg_3s = 52.94 t/s
10.18.932.694 I slot print_timing: id 0 | task 637 | n_gen = 2581, tg = 55.54 t/s, tg_3s = 53.27 t/s
10.21.969.807 I slot print_timing: id 0 | task 637 | n_gen = 2752, tg = 55.59 t/s, tg_3s = 56.30 t/s
10.25.025.671 I slot print_timing: id 0 | task 637 | n_gen = 2917, tg = 55.49 t/s, tg_3s = 53.99 t/s
10.28.509.225 I slot print_timing: id 0 | task 637 | n_gen = 3102, tg = 55.35 t/s, tg_3s = 53.11 t/s
10.31.516.060 I slot print_timing: id 0 | task 637 | n_gen = 3289, tg = 55.69 t/s, tg_3s = 62.19 t/s
10.34.572.797 I slot print_timing: id 0 | task 637 | n_gen = 3469, tg = 55.85 t/s, tg_3s = 58.89 t/s
10.37.605.463 I slot print_timing: id 0 | task 637 | n_gen = 3622, tg = 55.60 t/s, tg_3s = 50.45 t/s
10.40.637.741 I slot print_timing: id 0 | task 637 | n_gen = 3759, tg = 55.14 t/s, tg_3s = 45.18 t/s
10.43.685.465 I slot print_timing: id 0 | task 637 | n_gen = 3922, tg = 55.07 t/s, tg_3s = 53.48 t/s
10.46.719.495 I slot print_timing: id 0 | task 637 | n_gen = 4110, tg = 55.35 t/s, tg_3s = 61.96 t/s
10.49.769.676 I slot print_timing: id 0 | task 637 | n_gen = 4303, tg = 55.66 t/s, tg_3s = 63.27 t/s
10.52.773.426 I slot print_timing: id 0 | task 637 | n_gen = 4465, tg = 55.60 t/s, tg_3s = 53.93 t/s
10.55.787.927 I slot print_timing: id 0 | task 637 | n_gen = 4614, tg = 55.37 t/s, tg_3s = 49.43 t/s
10.58.802.819 I slot print_timing: id 0 | task 637 | n_gen = 4750, tg = 55.01 t/s, tg_3s = 45.11 t/s
11.01.829.267 I slot print_timing: id 0 | task 637 | n_gen = 4931, tg = 55.18 t/s, tg_3s = 59.81 t/s
11.04.860.209 I slot print_timing: id 0 | task 637 | n_gen = 5098, tg = 55.17 t/s, tg_3s = 55.10 t/s
11.08.293.106 I slot print_timing: id 0 | task 637 | n_gen = 5270, tg = 54.99 t/s, tg_3s = 50.10 t/s
11.11.318.003 I slot print_timing: id 0 | task 637 | n_gen = 5396, tg = 54.58 t/s, tg_3s = 41.65 t/s
11.14.371.772 I slot print_timing: id 0 | task 637 | n_gen = 5567, tg = 54.63 t/s, tg_3s = 56.00 t/s
11.17.401.228 I slot print_timing: id 0 | task 637 | n_gen = 5699, tg = 54.31 t/s, tg_3s = 43.57 t/s
11.20.433.590 I slot print_timing: id 0 | task 637 | n_gen = 5868, tg = 54.35 t/s, tg_3s = 55.73 t/s
11.21.715.172 I slot print_timing: id 0 | task 637 | prompt eval time = 95.04 ms / 4 tokens ( 23.76 ms per token, 42.09 tokens per second)
11.21.715.178 I slot print_timing: id 0 | task 637 | eval time = 109236.30 ms / 5972 tokens ( 18.29 ms per token, 54.66 tokens per second)
11.21.715.179 I slot print_timing: id 0 | task 637 | total time = 109331.34 ms / 5976 tokens
11.21.715.179 I slot print_timing: id 0 | task 637 | graphs reused = 3952
11.21.715.267 I slot print_timing: id 0 | task 637 | draft acceptance = 0.37763 ( 4220 accepted / 11175 generated), mean len = 3.43
11.21.715.364 I slot release: id 0 | task 637 | stop processing: n_tokens = 6353, truncated = 0
18.41.181.604 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 0.867 (> 0.100 thold), f_keep = 0.059
18.41.291.246 I slot launch_slot_: id 0 | task 4353 | processing task, is_child = 0
18.44.649.408 I slot print_timing: id 0 | task 4353 | n_gen = 146, tg = 47.94 t/s, tg_3s = 48.27 t/s

0 replies

Torchit1 · 2026-10-01T08:13:15Z

Torchit1
Oct 1, 2026
Maintainer Author

Thanks for testing, and for including the full log. 55 t/s sustained across almost 6,000 generated tokens on Windows is a solid result. The "4 tokens" isn't wrong, it's prompt caching. Your prompt was 381 tokens and 377 of them were already in the server's cache from the previous turn (f_sim_best = 1.000), so only 4 new ones needed processing. That conversation used about 6,350 tokens of context (n_tokens = 6353). The limit is set by -c, which is 131,072 (128K) in run-bonsai.bat, and the server prints it at startup as n_ctx. A couple of questions so I can read the numbers properly: did you use run-bonsai.bat as is, or your own command? The .bat starts with thinking turned off, so the model does its working in the answer itself. And did it get the timer puzzle right (it's 9 minutes: start both, flip the 7 when it runs out at 7, then flip it again when the 4 runs out at 8, and it finishes at 9)?

…

Sent from my Galaxy

-------- Original message -------- From: PotatoNut ***@***.***> Date: 30/9/26 10:05 pm (GMT+10:00) To: "Torchit1/llama.cpp" ***@***.***> Cc: Jesse Symons ***@***.***>, Author ***@***.***> Subject: Re: [Torchit1/llama.cpp] Post your Arc results (Discussion #1) System info: Asrock B580 12 GB - Graphics Driver 32.0.101.9030 CPU: Ryzen 7 7700 RAM 32 GB 6000 mt/s cl 30 OS Windows 11 Context size: Unknown, the prompt incorrectly marks this text as 4 tokens: You have two sand timers, which can show 4 minutes and 7 minutes respectively. Use both the sand timers(at a time or one after other or any other combination) and measure a time of 9 minutes. In most local models I have tried, this task end up getting brute forced and often is in an infinite loop. Result from the Terminal: 9.32.264.104 I slot release: id 0 | task 391 | stop processing: n_tokens = 6385, truncated = 0 9.32.264.147 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 1.000 (> 0.100 thold), f_keep = 0.059 9.32.383.713 I slot launch_slot_: id 0 | task 637 | processing task, is_child = 0 9.35.515.056 I slot print_timing: id 0 | task 637 | n_gen = 168, tg = 55.00 t/s, tg_3s = 55.33 t/s 9.38.545.317 I slot print_timing: id 0 | task 637 | n_gen = 332, tg = 54.56 t/s, tg_3s = 54.12 t/s 9.41.549.918 I slot print_timing: id 0 | task 637 | n_gen = 518, tg = 56.99 t/s, tg_3s = 61.91 t/s 9.44.593.857 I slot print_timing: id 0 | task 637 | n_gen = 691, tg = 56.95 t/s, tg_3s = 56.83 t/s 9.47.642.063 I slot print_timing: id 0 | task 637 | n_gen = 862, tg = 56.78 t/s, tg_3s = 56.10 t/s 9.50.655.480 I slot print_timing: id 0 | task 637 | n_gen = 1029, tg = 56.56 t/s, tg_3s = 55.42 t/s 9.53.671.737 I slot print_timing: id 0 | task 637 | n_gen = 1202, tg = 56.67 t/s, tg_3s = 57.36 t/s 9.56.720.684 I slot print_timing: id 0 | task 637 | n_gen = 1350, tg = 55.65 t/s, tg_3s = 48.54 t/s 9.59.748.650 I slot print_timing: id 0 | task 637 | n_gen = 1515, tg = 55.52 t/s, tg_3s = 54.49 t/s 10.02.756.850 I slot print_timing: id 0 | task 637 | n_gen = 1680, tg = 55.45 t/s, tg_3s = 54.85 t/s 10.05.764.248 I slot print_timing: id 0 | task 637 | n_gen = 1866, tg = 56.03 t/s, tg_3s = 61.85 t/s 10.09.312.509 I slot print_timing: id 0 | task 637 | n_gen = 2056, tg = 55.79 t/s, tg_3s = 53.55 t/s 10.12.340.111 I slot print_timing: id 0 | task 637 | n_gen = 2231, tg = 55.94 t/s, tg_3s = 57.80 t/s 10.15.891.599 I slot print_timing: id 0 | task 637 | n_gen = 2419, tg = 55.70 t/s, tg_3s = 52.94 t/s 10.18.932.694 I slot print_timing: id 0 | task 637 | n_gen = 2581, tg = 55.54 t/s, tg_3s = 53.27 t/s 10.21.969.807 I slot print_timing: id 0 | task 637 | n_gen = 2752, tg = 55.59 t/s, tg_3s = 56.30 t/s 10.25.025.671 I slot print_timing: id 0 | task 637 | n_gen = 2917, tg = 55.49 t/s, tg_3s = 53.99 t/s 10.28.509.225 I slot print_timing: id 0 | task 637 | n_gen = 3102, tg = 55.35 t/s, tg_3s = 53.11 t/s 10.31.516.060 I slot print_timing: id 0 | task 637 | n_gen = 3289, tg = 55.69 t/s, tg_3s = 62.19 t/s 10.34.572.797 I slot print_timing: id 0 | task 637 | n_gen = 3469, tg = 55.85 t/s, tg_3s = 58.89 t/s 10.37.605.463 I slot print_timing: id 0 | task 637 | n_gen = 3622, tg = 55.60 t/s, tg_3s = 50.45 t/s 10.40.637.741 I slot print_timing: id 0 | task 637 | n_gen = 3759, tg = 55.14 t/s, tg_3s = 45.18 t/s 10.43.685.465 I slot print_timing: id 0 | task 637 | n_gen = 3922, tg = 55.07 t/s, tg_3s = 53.48 t/s 10.46.719.495 I slot print_timing: id 0 | task 637 | n_gen = 4110, tg = 55.35 t/s, tg_3s = 61.96 t/s 10.49.769.676 I slot print_timing: id 0 | task 637 | n_gen = 4303, tg = 55.66 t/s, tg_3s = 63.27 t/s 10.52.773.426 I slot print_timing: id 0 | task 637 | n_gen = 4465, tg = 55.60 t/s, tg_3s = 53.93 t/s 10.55.787.927 I slot print_timing: id 0 | task 637 | n_gen = 4614, tg = 55.37 t/s, tg_3s = 49.43 t/s 10.58.802.819 I slot print_timing: id 0 | task 637 | n_gen = 4750, tg = 55.01 t/s, tg_3s = 45.11 t/s 11.01.829.267 I slot print_timing: id 0 | task 637 | n_gen = 4931, tg = 55.18 t/s, tg_3s = 59.81 t/s 11.04.860.209 I slot print_timing: id 0 | task 637 | n_gen = 5098, tg = 55.17 t/s, tg_3s = 55.10 t/s 11.08.293.106 I slot print_timing: id 0 | task 637 | n_gen = 5270, tg = 54.99 t/s, tg_3s = 50.10 t/s 11.11.318.003 I slot print_timing: id 0 | task 637 | n_gen = 5396, tg = 54.58 t/s, tg_3s = 41.65 t/s 11.14.371.772 I slot print_timing: id 0 | task 637 | n_gen = 5567, tg = 54.63 t/s, tg_3s = 56.00 t/s 11.17.401.228 I slot print_timing: id 0 | task 637 | n_gen = 5699, tg = 54.31 t/s, tg_3s = 43.57 t/s 11.20.433.590 I slot print_timing: id 0 | task 637 | n_gen = 5868, tg = 54.35 t/s, tg_3s = 55.73 t/s 11.21.715.172 I slot print_timing: id 0 | task 637 | prompt eval time = 95.04 ms / 4 tokens ( 23.76 ms per token, 42.09 tokens per second) 11.21.715.178 I slot print_timing: id 0 | task 637 | eval time = 109236.30 ms / 5972 tokens ( 18.29 ms per token, 54.66 tokens per second) 11.21.715.179 I slot print_timing: id 0 | task 637 | total time = 109331.34 ms / 5976 tokens 11.21.715.179 I slot print_timing: id 0 | task 637 | graphs reused = 3952 11.21.715.267 I slot print_timing: id 0 | task 637 | draft acceptance = 0.37763 ( 4220 accepted / 11175 generated), mean len = 3.43 11.21.715.364 I slot release: id 0 | task 637 | stop processing: n_tokens = 6353, truncated = 0 18.41.181.604 I slot get_availabl: id 0 | task -1 | selected slot by LCP similarity, f_sim_best = 0.867 (> 0.100 thold), f_keep = 0.059 18.41.291.246 I slot launch_slot_: id 0 | task 4353 | processing task, is_child = 0 18.44.649.408 I slot print_timing: id 0 | task 4353 | n_gen = 146, tg = 47.94 t/s, tg_3s = 48.27 t/s — Reply to this email directly, view it on GitHub<#1?email_source=notifications&email_token=BFCOGHKSHQACRT73CBGAEGL5RTZIXA5CNFSNUABIM5UWIORPF5TWS5BNNB2WEL2ENFZWG5LTONUW63SDN5WW2ZLOOQXTCOBWG44DOMZRUZZGKYLTN5XKMYLVORUG64VFMV3GK3TUVRTG633UMVZF6Y3MNFRWW#discussioncomment-18678731>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/BFCOGHL7GV24JKPK5D6FH435RTZIXAVCNFSNUABJKJSXA33TNF2G64TZHMYTGOBZGAZTEMRQGM5UI2LTMN2XG43JN5XDWMJQHA4TGMRTGWQXMAQ>. You are receiving this because you authored the thread.Message ID: ***@***.***>

1 reply

PotatoNut Oct 1, 2026

Thanks for the explanation.
I used Ternary-Bonsai-2-27B-PTQ1_0-mtp-lean with the run-bonsai.bat unchanged. No thinking.

The puzzle is particularly hard for the AI models. Gemma, Ornith, Ling, Laguna, those all failed the task, but were great at many other tests I tried with similar difficulty. Ternary-Bonsai end up the answering with this:

I think the answer is 9 minutes can be measured by:

Start the 4-minute timer.
When it finishes, start the 7-minute timer.
When the 7-minute timer finishes (11 min), flip the 4-minute timer.
When the 4-minute timer finishes (15 min), flip the 7-minute timer.
When the 7-minute timer finishes (22 min), stop.
I think I have to conclude that I cannot solve this correctly.

While it didn't produce the correct answer, it did think of the correct combo for the solution (4+4+1) Flip both, 4 runs out flip again, 7 runs out - flip, 4 runs out - flip the 7 again as 1 min is on the other end.

I think I have to accept that my manual calculation is flawed. Let's think logically:

We need 9 minutes.
Possible combinations:

7 + 2
4 + 4 + 1
4 + 5
7 + 4 - 2
4 + 7 - 2

With the flag {\"enable_thinking\":true}" the model now produces the correct solution at the first try. Here is the Terminal result:

0.28.078.851 I slot launch_slot_: id 0 | task 0 | processing task, is_child = 0
0.32.497.094 I slot print_timing: id 0 | task 0 | n_gen = 125, tg = 41.17 t/s, tg_3s = 41.50 t/s
0.35.504.700 I slot print_timing: id 0 | task 0 | n_gen = 243, tg = 40.20 t/s, tg_3s = 39.23 t/s
0.38.516.855 I slot print_timing: id 0 | task 0 | n_gen = 360, tg = 39.75 t/s, tg_3s = 38.84 t/s
0.41.561.645 I slot print_timing: id 0 | task 0 | n_gen = 468, tg = 38.67 t/s, tg_3s = 35.47 t/s
0.44.573.647 I slot print_timing: id 0 | task 0 | n_gen = 609, tg = 40.30 t/s, tg_3s = 46.81 t/s
0.47.620.590 I slot print_timing: id 0 | task 0 | n_gen = 723, tg = 39.81 t/s, tg_3s = 37.41 t/s
0.50.620.848 I slot print_timing: id 0 | task 0 | n_gen = 839, tg = 39.65 t/s, tg_3s = 38.66 t/s
0.53.622.580 I slot print_timing: id 0 | task 0 | n_gen = 1020, tg = 42.22 t/s, tg_3s = 60.30 t/s
0.53.771.588 I slot print_timing: id 0 | task 0 | prompt eval time = 1406.25 ms / 414 tokens ( 3.40 ms per token, 294.40 tokens per second)
0.53.771.593 I slot print_timing: id 0 | task 0 | eval time = 24286.36 ms / 1031 tokens ( 23.58 ms per token, 42.41 tokens per second)
0.53.771.594 I slot print_timing: id 0 | task 0 | total time = 25692.61 ms / 1445 tokens
0.53.771.594 I slot print_timing: id 0 | task 0 | graphs reused = 500
0.53.771.668 I slot print_timing: id 0 | task 0 | draft acceptance = 0.26190 ( 528 accepted / 2016 generated), mean len = 2.05
0.53.771.730 I slot release: id 0 | task 0 | stop processing: n_tokens = 1446, truncated = 0

Navigation Menu

Post your Arc results #1

Uh oh!

{{title}}

Uh oh!

Replies: 3 comments 6 replies

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{editor}}'s edit

{{editor}}'s edit

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Uh oh!

{{title}}

Uh oh!

Select a reply

Uh oh!

Post your Arc results #1

Uh oh!

Torchit1 Sep 26, 2026 Maintainer

Replies: 3 comments · 6 replies

Uh oh!

lavdrimismajili Sep 27, 2026

SYCL Flash Attention on Arc 140T (Xe-LPG): oneDNN SDPA returns wrong results for quantized KV with permuted views

Summary

Hardware / Software

test-backend-ops Results

With oneDNN and MKL FA enabled

With oneDNN and MKL FA disabled

Reproduction

Observed Error Magnitudes Across Runs

llama-bench Results

Server Findings

MTP memory leak

VMM

ngram-mod behavior

ngram-mod window size

Conclusion

Uh oh!

Torchit1 Sep 27, 2026 Maintainer Author

Uh oh!

Uh oh!

lavdrimismajili Sep 28, 2026

Uh oh!

Torchit1 Sep 28, 2026 Maintainer Author

Uh oh!

lavdrimismajili Sep 28, 2026

Uh oh!

Torchit1 Sep 30, 2026 Maintainer Author

Uh oh!

PotatoNut Sep 30, 2026

Uh oh!

Torchit1 Oct 1, 2026 Maintainer Author

Uh oh!

PotatoNut Oct 1, 2026

Torchit1
Sep 26, 2026
Maintainer

Replies: 3 comments 6 replies

lavdrimismajili
Sep 27, 2026

`test-backend-ops` Results

`llama-bench` Results

Torchit1 Sep 27, 2026
Maintainer Author

Torchit1 Sep 28, 2026
Maintainer Author

Torchit1 Sep 30, 2026
Maintainer Author

PotatoNut
Sep 30, 2026

Torchit1
Oct 1, 2026
Maintainer Author