Skip to content

Kiln v0.1.0 — RK3576 NPU on a mainline kernel

Choose a tag to compare

View all tags
@gahingwoo gahingwoo released this 08 Jul 12:01
· 58 commits to main since this release
v0.1.0
52d736b

Kiln runs the Rockchip RK3576 NPU — LLMs (RKLLM) and vision (RKNN) — on a
mainline Linux kernel (linux-7.1.3), by building the vendor GPL rknpu
v0.9.8 driver out-of-tree and driving it with the closed librkllmrt /
librknnrt runtimes. Verified on real hardware (Radxa ROCK 4D).

What works

  • LLM chat on the NPU — kiln-chat holds a multi-turn conversation:
    Qwen2.5-1.5B (~9 tok/s) or Llama-3.2-1B (~13 tok/s), switchable live with
    /model. Model-aware chat template + stop tokens (Qwen ChatML / Llama-3),
    readline input (cursor editing + history), and slash commands (/model,
    /system, /history, /clear, /new, /context, /compact) that persist to
    the config.
  • Vision on the NPU — kiln-vision runs MobileNet (RKNN) at ~6 ms / ~169 fps.
  • OpenAI-compatible API — kiln-serve (POST /v1/chat/completions SSE streaming,
    GET /v1/models, optional POST /v1/vision/classify), header-only, no Python.

Why it takes kernel patches

Several RK3576 NPU fixes are kernel code, not something the module or a DT
overlay can supply:

  • The NPU is one device with two IOMMUs; mainline rockchip-iommu drives only
    the primary one, so a naive port only gets task_counter=0 timeouts. Kiln enables
    all four MMU banks from the driver and flushes their TLB per job.
  • The power domain needs a settle delay, a BIU reset, the full clock set, and a
    cold-start "arm"; and the ROCK 4D board DTS drops the NPU rail (vdd_npu_s0) ~30 s
    into boot — the second inference then reads a dead rail and hangs the board.
    One line, regulator-always-on (kernel-patches/0010), fixes it.

It is mainline-based, not stock mainline: kernel-patches/ 0001–0010 are
required. CI publishes the kernel .deb.

Install

  • Armbian — one command (installs the Kiln mainline kernel, then driver +
    runtimes + tools; two phases, reboot between). See ARMBIAN.md: 
    curl -fsSL https://raw.githubusercontent.com/gahingwoo/kiln/main/scripts/kiln-install.sh | bash
  • Flashable image — buildroot br2-external: buildroot/build-image.sh.
  • Kernel — MAINLINE-KERNEL.md.

Demos

Honest caveats

  • LLM quality is bounded by the small quantized models (1–1.5B w4a16); Llama-3.2 is
    the more coherent of the two. /compact is a best-effort application-level summary.
  • RK3568 (ROCK 3B) support is implemented but not yet tested on hardware (RK3568.md).
  • The GPL rknpu source and the closed librkllmrt/librknnrt runtimes are
    fetched, not redistributed; model weights are separate.

GPL-2.0. Companion open-driver project:
linux-rk3576-npu.

Assets 2
Loading