arc-b580-v0.10.1

Fewer round trips and faster long-context attention on the B580: up to 6.5% faster than v0.10, and 12-15% faster than v0.8. Greedy outputs are byte-identical to v0.10.

• Against v0.10 on the B580 (same public models and flags): MTP with a 32K trimmed draft head 80.4 -> 84.7 t/s (+5.4%); MTP on mtp-lean 78.3 -> 80.5 (+2.8%). Live settings (MTP + n-gram drafts, 128K context, same head): short prompts 75.9 -> 78.6 (+3.6%), 48K 78.7 -> 82.4 (+4.7%), 115K 83.5 -> 88.9 (+6.5%).
• One read-back for the draft chain: the one-graph draft chain now returns all its draft tokens in one device-to-host copy instead of one per token. Each small copy costs about 0.2 ms on this SYCL stack, so this alone is worth about 2% with a draft head. On by default (LLAMA_MTP_CHAIN_TIGHT is now 31; bit 16 is this change).
• 256-register file for the q4_0 decode attention at 3+ query tokens (the MTP and n-gram verify batches): no register spills, same occupancy, and the kernel is 6-17% faster at 128K. GGML_SYCL_FA_DEC_GRF=0 turns it off.
• Three general improvements from PrismML's llama.cpp fork (github.com/PrismML-Eng/llama.cpp), with their authorship kept: KV seq_rm limited to the used cell range (ggml-org#311), reuse of evicted prompt-checkpoint buffers in the server (ggml-org#312), and top-k taken straight from the logits when it starts the sampler chain (ggml-org#313).
• Opt-in: GGML_SYCL_FA_DEC_B2D=1 uses 2D block loads for the decode attention's K/V reads (within noise on the B580).

Windows x64 build of the arc-b580 branch for Intel Arc (SYCL). EXPERIMENTAL: built by CI and not yet tested on a
Windows machine. Needs a current Intel Arc driver (an old driver's ze_loader.dll crashes at start).
Unzip, put the model next to it, and run run-bonsai.bat. See README.txt and
docs/bonsai-arc-b580.md. Please report results in https://github.com/Torchit1/llama.cpp/discussions/1

arc-b580-v0.10

Faster MTP decoding on the B580: about 9% faster than v0.8, with 432 MiB more free VRAM at 128K context. Outputs are identical to v0.8.

• Against v0.8 on the B580 (same public models and settings): MTP 71.9 -> 78.3 t/s (+8.9%); with a 32K trimmed draft head 73.8 -> 80.4 t/s (+9%). Live settings (MTP + n-gram drafts, 128K context): short prompts 69.9 -> 75.9 t/s (+8.6%), 48K 73.7 -> 78.7 (+6.8%), 115K 78.8 -> 83.5 (+6.0%).
• Rollback by replay (LLAMA_GDN_REPLAY, on): after each MTP verify, the delta-net layers replay the accepted tokens instead of keeping a saved copy of the state for every drafted token. Replay mode allocates only two state planes, which frees 432 MiB of VRAM at 128K context. Independent implementation of the technique described by llamADALL. LLAMA_GDN_REPLAY=0 turns it off.
• K/V-only MTP prompt pass (LLAMA_MTP_KV_ONLY, on): over a prompt, the draft head computes only the K/V rows it keeps, so long prompts read faster with MTP. Independent implementation of the technique described by GRIMOIRE. LLAMA_MTP_KV_ONLY=0 turns it off.
• One-graph MTP draft chain (LLAMA_MTP_CHAIN): all of a step's drafts in one decode, with the draft-vocabulary embeddings on the GPU. On by default for a GGUF with a trimmed draft head; LLAMA_MTP_CHAIN=1 forces it, =0 turns it off. Independent implementation of the technique described by llamADALL.
• New: scripts/bonsai/add_draft_head.py and freq_public.json add a trimmed 32K draft head to an MTP GGUF, using token counts from public text. See "Adding a trimmed draft head" in docs/bonsai-arc-b580.md.
• Tuned verify tiles on the B580 (GGML_SYCL_PTQ1_T2_TUNED): a sweep of the TernSYCL small-batch kernel at the model's shapes found faster bit-identical tiles for the 5-token MTP verify (FFN down 0.78x the time, FFN gate+up 0.90x, output head 0.88x, attention q+k+v 0.92x). On by default only on a B580; =1 forces them elsewhere, =0 turns them off.
• The SYCL backend uses the legacy Level Zero adapter unless SYCL_UR_USE_LEVEL_ZERO_V2 is set: plain decode about +1% on the B580.
• Fewer host synchronizations (LLAMA_SYNC_TRIM=3, LLAMA_MTP_CHAIN_TIGHT=15). LLAMA_SYNC_TRIM=0 restores the upstream behaviour.
• Diagnostics, off by default: LLAMA_MTP_DRAFT_TIMING, GGML_SYCL_OP_TIMING, GGML_SYCL_SYNC_TRACE.
• No code from llamADALL or GRIMOIRE is used. Every switch and credit is listed in docs/bonsai-arc-b580.md.

Windows x64 build of the arc-b580 branch for Intel Arc (SYCL). EXPERIMENTAL: built by CI and not yet tested on a
Windows machine. Needs a current Intel Arc driver (an old driver's ze_loader.dll crashes at start).
Unzip, put the model next to it, and run run-bonsai.bat. See README.txt and
docs/bonsai-arc-b580.md. Please report results in https://github.com/Torchit1/llama.cpp/discussions/1

arc-b580-v0.8

Faster plain generation on the B580 from fused kernels around the ternary mat-vecs, plus faster decode attention at long context.

• Plain generation (no speculation), public mtp-lean model, B580: short prompt ~41 -> 46 t/s, 48K context ~36 -> 40, 115K ~31.5 -> 35.
• Each token's activation is Hadamard-rotated and int8-quantized once and reused by every mat-vec that reads it. The residual add + RMS norm, the FFN SwiGLU and the delta-net pre-norm are folded into that rotation. The delta-net conv chain and its gate block each run as one kernel. Mat-vecs that share an input (q/k/v, gate/up) run in one launch. Small verify batches (up to 32 rows) use a smaller TernSYCL tile.
• Decode attention prefetches the next tile's rows: with MTP, +4% at 48K and +7.5% at 115K.
• n-gram drafts start at 64 tokens, double after a full accept and halve after a rejection (LLAMA_ARG_SPEC_NGRAM_ADAPT=0 restores a fixed length).
• Vendored TernSYCL updated to 8b0ae42.
• Every new piece can be switched off with an environment variable for comparison (list in docs/bonsai-arc-b580.md, "Switches"). KL divergence is unchanged within noise. The fusions have exact tests: test-backend-ops -o ARCLAB_* (75/75 on the B580).

Windows x64 build of the arc-b580 branch for Intel Arc (SYCL). EXPERIMENTAL: built by CI and not yet tested on a
Windows machine. Needs a current Intel Arc driver (an old driver's ze_loader.dll crashes at start).
Unzip, put the model next to it, and run run-bonsai.bat. See README.txt and
docs/bonsai-arc-b580.md. Please report results in https://github.com/Torchit1/llama.cpp/discussions/1

arc-b580-v0.7

• No undocumented IGC builtins left in the SYCL backend: the TernSYCL kernels are moved to upstream main (inline vISA), and the f16 attention DPAS uses inline vISA too. v0.6 still had intel_sub_group_i8_i2_matrix_mad_k32 and _builtin_IB* in the Bonsai path, so this completes the fix for drivers that don't provide them (reported on Lunar Lake / Arc 140V).
• Long n-gram draft checks stay on the XMX attention kernel up to 256 tokens (was 32), with no f16 copy of the KV cache. MTP at long context on the B580: 48K ~73 -> ~76 t/s, 115K ~60 -> ~65 t/s. Accuracy (KLD) unchanged. GGML_SYCL_FA_DEC_DPAS=32 restores the old cap.
• The SYCL backend is built with one device image per kernel (GGML_SYCL_DEVICE_CODE_SPLIT_PER_KERNEL, default ON), so the Xe2 XMX kernels are only compiled when used. This should help GPUs without XMX, but it isn't confirmed yet. The DLL is larger (about 98 MB).

Windows x64 build of the arc-b580 branch for Intel Arc (SYCL). EXPERIMENTAL: built by CI and not yet tested on a
Windows machine. Needs a current Intel Arc driver (an old driver's ze_loader.dll crashes at start).
Unzip, put the model next to it, and run run-bonsai.bat. See README.txt and
docs/bonsai-arc-b580.md. Please report results in https://github.com/Torchit1/llama.cpp/discussions/1

arc-b580-v0.6

What's new in v0.6

• Long-context decode is faster on the B580: plain decode +12% at 48K and +21% at 115K. MTP is +2–4%. Output and accuracy are unchanged.
• Fix: crash at the first decode on GPUs whose driver lacks the intel_sub_group_i8_i4_matrix_mad_k32 builtin (reported on Lunar Lake / Arc 140V). The kernels now use inline vISA instead.
• Guide: when to turn thinking on (reasoning puzzles).

Windows x64 build of the arc-b580 branch for Intel Arc (SYCL). EXPERIMENTAL: built by CI and not yet tested on a
Windows machine. Needs a current Intel Arc driver (an old driver's ze_loader.dll crashes at start).
Unzip, put the model next to it, and run run-bonsai.bat. See README.txt and
docs/bonsai-arc-b580.md. Please report results in https://github.com/Torchit1/llama.cpp/discussions/1

arc-b580-v0.5

Windows x64 build of the arc-b580 branch for Intel Arc (SYCL). EXPERIMENTAL: built by CI and not yet tested on a
Windows machine. Needs a current Intel Arc driver (an old driver's ze_loader.dll crashes at start).
Unzip, put the model next to it, and run run-bonsai.bat. See README.txt and
docs/bonsai-arc-b580.md. Please report results in https://github.com/Torchit1/llama.cpp/discussions/1

arc-b580-v0.4

Windows x64 build of the arc-b580 branch for Intel Arc (SYCL). EXPERIMENTAL: built by CI and not yet tested on a
Windows machine. Needs a current Intel Arc driver (an old driver's ze_loader.dll crashes at start).
Unzip, put the model next to it, and run run-bonsai.bat. See README.txt and
docs/bonsai-arc-b580.md. Please report results in https://github.com/Torchit1/llama.cpp/discussions/1

arc-b580-v0.3

Windows x64 build of the arc-b580 branch for Intel Arc (SYCL). EXPERIMENTAL: built by CI and not yet tested on a
Windows machine. Needs a current Intel Arc driver (an old driver's ze_loader.dll crashes at start).
Unzip, put the model next to it, and run run-bonsai.bat. See README.txt and
docs/bonsai-arc-b580.md. Please report results in https://github.com/Torchit1/llama.cpp/discussions/1

arc-b580-v0.2

Windows x64 build of the arc-b580 branch for Intel Arc (SYCL). EXPERIMENTAL: built by CI and not yet tested on a
Windows machine. Needs a current Intel Arc driver (an old driver's ze_loader.dll crashes at start).
Unzip, put the model next to it, and run run-bonsai.bat. See README.txt and
docs/bonsai-arc-b580.md. Please report results in https://github.com/Torchit1/llama.cpp/discussions/1

arc-b580-v0.1

Windows x64 build of the arc-b580 branch for Intel Arc (SYCL). EXPERIMENTAL: built by CI and not yet tested on a
Windows machine. Needs a current Intel Arc driver (an old driver's ze_loader.dll crashes at start).
Unzip, put the model next to it, and run run-bonsai.bat. See README.txt and
docs/bonsai-arc-b580.md. Please report results in https://github.com/Torchit1/llama.cpp/discussions/1