Repository navigation
Home
ggml-rknpu2 lets llama.cpp, a popular program for running AI chat models on your own computer, use the NPU in Rockchip chips. The NPU is a small part of the chip built for the kind of maths these models do. With ggml-rknpu2 it reads your prompts while the CPU writes the answers.
You don't need to know anything about NPUs to use it. If you can copy a command into a terminal, you can follow these guides.
- Install: one command on your board.
- Your first chat: talk to a model in the terminal.
- Chat in your browser: a chat page, and an API for other apps.
- Models and memory: which model to pick for your board
- Speed and settings: measuring speed, changing the defaults
- Troubleshooting: when something doesn't work
- Updating and uninstalling
- Building from source
RK3588 and RK3588S, RK3576, RK3566, RK3568 and RK3562, on 64-bit Linux with Rockchip's vendor kernel (the kernel most board makers ship).