llama.cpp backend for oneDNN

[lxgo]

Intel Arc GPUs represent a popular target architecture for llama.cpp that is already supported by multiple backends running on various accelerator frameworks including Vulkan, SYCL, and OpenVINO.

I have recently developed and publicly released a new llama.cpp backend based on oneDNN framework. This backend is powered by Arhat LLM, a library of deep neural network primitives designed for implementation of LLM inference. Arhat LLM is built on top of oneDNN and can be used either in combination with llama.cpp or as a standalone library.

I use OpenCL 3.0 to implement GGML operations that are not natively supported by oneDNN. The respective OpenCL kernels have been ported from various GGML backends like OpenCL, CUDA, and Vulkan. For several computationally complex operations like matrix multiplication or flash attention I ported multiple kernels from different backends and compared performance of the resulting code. In the most critical cases I implemented further performance tuning specific for Intel GPU target platform. (Kernel tuning is an ongoing effort as there is certainly room for further optimization.)

I have validated Arhat LLM on integrated Intel Arc GPUs of Core Ultra 2 Series processors (former Arrow Lake and Lunar Lake).

The source code of Arhat LLM and llama.cpp backend for oneDNN is available on GitHub:

https://github.com/fragata-ai/arhat-llm