Skip to content

arcint 1.6.1

Latest

Choose a tag to compare

@marfrit marfrit released this 07 Nov 15:02
· 2 commits to main since this release

Gemma 4 on the fast path on Intel Arc, a correctness fix for upstream's f16 decode matvec, and NVIDIA's looped Dory1.0-4B. The full record is in CHANGELOG.md.

  • Gemma 4 E4B on the fast path at both KV types (patches 0106, 0110). Flash attention at head size 512 on Intel, which ggml declines on every Intel device. On the B60 at a 6.5k-token prompt, against 1.6.0: q8_0:q4_0 KV prefill 176-246 -> 1,905-2,053 t/s, decode 65 -> 75.5 t/s; f16 KV prefill about 1,285 -> 2,113-2,381 t/s, decode 79 -> 81. F32 weights are multiplied through an f16 copy in prompts: another +3.7 to +3.9 % prefill, decode unchanged (GGML_OPENCL_F32W_F16).
  • A correctness fix in upstream's f16 one-column matvec (patch 0107). kernel_mul_mat_f16_f32_l4_x8 staged its vector one float4 a sub-group lane, which fills only half of it on 32-wide sub-groups. Gemma 4 with an f16 KV cache and flash attention off decoded wrong: KL against a CPU reference 3.37 nats at batch 1, a thought that fell into a loop. Fixed, KL 0.0044 on both cards. It applies to any model whose decode reaches that kernel.
  • Dory1.0-4B is admitted (patches 0108, 0109): NVIDIA's looped nGPT, ported from llama.cpp pull request 30306, with the OpenCL copy from BF16 its scale vectors need. --llama-dory-loops and --llama-dory-kv set the loop count and the recurrent KV layout, checked at start. B60: 36.4 t/s at 2 loops, 22.0 at 4; A770: 23.7 and 14.3; KL against the CPU reference 0.009-0.016.

Release gate against 1.6.0, same window (measured-here): the agent passes, its 255k needle right. The coder and Flash-Next fail on the spread rule only, by the operator's acceptance: the coder's means equal with identical texts in every run and the acceptance task 10/10 x3; Flash-Next's prefill equal and its decode answers differing in every run of both arms. No unit reports a new downgrade.

The attached .deb is built from this tag's tarball for Debian 13 (trixie) on x86_64 and needs Intel's compute runtime.