Skip to content

b10256

Latest

Choose a tag to compare

@github-actions github-actions released this 04 Aug 06:39
6c8dcaa

sycl: parallelize the non-contiguous concat kernel (#25852)

  • sycl: parallelize the non-contiguous concat kernel

Launch geometry only: the non-contiguous concat kernel launched a single-lane
work-group (1, 1, 1), now it will launch a (1, 1, SYCL_CONCAT_BLOCK_SIZE) one.

SYCL_CONCAT_BLOCK_SIZE is defined in ggml/src/ggml-sycl/presets.hpp.

llama-bench (Arc Pro B70, Qwen3.6-27B-UD-Q4_K_XL, -fa on, q8_0 KV),
on top of upstream master: pp2048 920 -> 1006 t/s (+9.4%)

  • sycl: cap non-contiguous concat block at ne0

  • sycl: make non-contiguous concat block width env-tunable (GGML_SYCL_CONCAT_BLOCK_SIZE)

  • Revert "sycl: make non-contiguous concat block width env-tunable (GGML_SYCL_CONCAT_BLOCK_SIZE)"

This reverts commit 2709909.

Website:

macOS/iOS:

Linux:

Android:

Windows:

openEuler:

  • DISABLED
  • openEuler x86 (310p)
  • openEuler x86 (910b, ACL Graph)
  • openEuler aarch64 (310p)
  • openEuler aarch64 (910b, ACL Graph)

UI: