Repository navigation
Troubleshooting
Start with llama-npu check, then find the matching problem below.
Open a new terminal, or run source ~/.bashrc. Still missing? Use the full path:
~/.local/share/ggml-rknpu2/llama-npu.
The board isn't an RK3588, RK3576, RK3566, RK3568 or RK3562, or it runs a 32-bit system. llama-npu still works, on the CPU only.
The kernel has no Rockchip NPU driver. Most board makers' own images include it. On Armbian:
sudo armbian-config → System → Kernel → pick a vendor (or legacy) kernel, then reboot.
Mainline kernels, including their new rocket NPU driver, don't work with Rockchip's runtime.
Your user isn't allowed to use the NPU. The check prints the exact fix; usually it's:
sudo usermod -aG render $USER # then log out and back inFix the lines above it first. If they're all OK, run llama-npu devices -v, look for lines
mentioning rknpu or rknn, and open an issue with that output.
Run llama-npu update. It downloads runtime 2.3.2 next to the program.
Check the board's internet: curl -I https://huggingface.co. Some models ask you to accept their
licence on Hugging Face first; they need a token: export HF_TOKEN=your-token.
A build from one board was copied to a different chip. Use the installer, or build on the board you run it on.
Out of memory. Choose a smaller model or shrink the NPU copy:
GGML_RKNPU2_MEM_MB=1024 llama-npu chat MODEL. See Models and memory.
- Short prompts mostly measure answer-writing, which runs on the CPU either way.
- Only prompt chunks of 128 tokens or more go to the NPU.
- Check the NPU clock: see Speed and settings.
Open an issue with the output of
llama-npu check and llama-npu version.