Skip to content

xdna.cpp v1.0.0 - AMD XDNA1 NPU Backend for llama.cpp

Latest

Choose a tag to compare

@random-unknown-username random-unknown-username released this 31 Aug 15:08
· 6 commits to main since this release

Highlights in v1.0.0

  • Native AMD XDNA1 (AIE2) Support: Direct vector execution on Phoenix/Hawk Point NPUs (4 columns x 4 rows = 16 tiles).
  • Bounded 128 MB Staging Ring: Eliminates unevictable kernel BO memory locking, enabling 15+ GB models (e.g. Qwen3.8-27B) to decode cleanly on 32 GB RAM without swap thrashing.
  • Heterogeneous Hybrid Scheduling: Accelerates medium and large matrices (e.g. Qwen2.5-3B at 16.5 tok/s, +68.8% over CPU) while preserving low-latency SIMD execution for small row blocks.
  • Drop-in llama.cpp Plugin: Dynamic libggml-xdna.so plugin loadable via --device XDNA0.