Building VKNN

Building VKNN

./build.sh is the single build entry point. It drives CMake + Ninja for two targets: a host build (CPU backend, IR, ONNX import, tools, and tests — no Vulkan) and an Android build (the full engine including the Vulkan backend and NEON kernels).

Prerequisites

Target	Needs
Host build	A C++17 compiler, CMake, and Ninja. No Vulkan SDK required — the host build compiles the Vulkan path out and the CPU kernels run.
Android build	The Android NDK (r27 arm64-v8a toolchain). Override its location with ANDROID_NDK=... and the API level with ANDROID_API=... (default android-33).
Shaders	The vendored glslang (third_party/glslang) is built once, on the host, as a build-time GLSL→SPIR-V compiler. If the submodule is absent, CMake falls back to a system glslc. Only the Android (Vulkan) build compiles shaders.
Docs site	Python 3 (scripts/gen_site.py). Doxygen is optional — if installed, the C++ API reference is also emitted and linked from the site.

No third-party runtime dependencies are used: ONNX import is hand-rolled, and the engine links only Vulkan and the C++ standard library.

Commands

./build.sh                 # host build: CPU backend + IR + ONNX import + tools + tests (no Vulkan)
./build.sh --android       # Android arm64-v8a build (Vulkan backend + NEON, NDK toolchain)
./build.sh --clean         # wipe the build dir(s) first; combine with a build flag for a clean build
./build.sh --convert       # build only the model compiler (vknn_compile) for the chosen target
./build.sh --docs          # build the static documentation site -> docs/site/index.html

• --clean alone removes build-host/ and build-android/ and exits. Combined with a build flag it cleans that target's directory first, then builds — e.g. ./build.sh --android --clean.
• --convert builds only vknn_compile. Combine with --android for the on-device compiler (./build.sh --android --convert); host --convert (./build.sh --convert) produces the desktop compiler used for offline conversion and the support gate.

Host artifacts land in build-host/, Android artifacts in build-android/.

./build-host/vknn_tests    # run the host unit / integration tests (GoogleTest)

Host vs Android

The host build has no Vulkan backend — it exists for import, conversion, graph passes, and tests, and it runs the CPU oracle. GPU (0-fallback) validation is Android-only: build with --android, push the binaries to a connected arm64-v8a device over adb, and run there.

A new operator or shader is picked up automatically. The CMake globs use CONFIGURE_DEPENDS, so a new src/backend/**/ops/<op>.cpp or a new shaders/<op>.comp is re-globbed on the next configure, which ./build.sh runs. A new .comp changes embeddedShadersHash() and so invalidates device caches on the next load.

Compiling a model to .vxm

vknn_compile runs the import passes ahead of time and writes a pre-optimized .vxm that skips ONNX parsing and graph passes at load:

# model.onnx -> model.vxm.  --fp16 halves the file + host upload; --shape resolves a dynamic input.
vknn_compile model.onnx model.vxm --fp16 --shape input=1x3x224x224

Key flags (full list in the repo's docs/config.md):

Flag	Meaning
--fp16	Store weights as fp16 in the .vxm (default fp32). Halves the file and the runtime repack; compute precision is the separate Config::precision knob.
-O0 / -O1 / -O2 / -O3	Optimization level (default -O1 = the bit-exact production fusion set).
--shape NAME=D0xD1x... (repeatable)	Declare one graph input's full concrete shape, resolving its dynamic axes.
--bucket "NAME=...;NAME2=..." (repeatable)	Declare one plan bucket per occurrence (e.g. a prefill shape and a decode shape in one multi-bucket .vxm).
--no-dequantize	Keep the quantized ops instead of folding them to float (the default folds a quantized checkpoint to a plain float graph).
--support-report <out.json>	Write the per-node backend assignment from the engine's own capability gate — the 0-CPU-fallback oracle (see Architecture).

Running on a device

Push the binaries and model, then run on the device (the scratch dir convention is /data/local/tmp/vxrt/):

adb push build-android/vknn_classify model.vxm input.bin /data/local/tmp/vxrt/
adb shell /data/local/tmp/vxrt/vknn_classify --model model.vxm --input input.bin \
    --backend vulkan --precision low --bench 20

--precision is the quality tier: low (fp16 storage + fp32 accumulation), normal (fp16 with a precision-critical geometry tail kept fp32), or high (full fp32).

Re-push the vknn_* binaries after every Android rebuild. A stale binary on the device silently invalidates a change. For benchmarking, cool the GPU (adb shell sleep 12-14) before each run — the device throttles quickly under sustained load.

Before you push

scripts/ci_host.sh is the host-only gate: host build + vknn_tests + --android + --docs + op-support self-consistency + clang-format drift + CPU determinism. It needs no device. The on-device byte and perf gates run separately.

See also: Building VKNN → Architecture for what the compiled graph looks like, and Running an LLM on VKNN for the LLM-specific convert/run flow.