VKNN 1.2.5

1.2.5 — Llama 3.1 8B + a remote model catalog

Llama 3.1 8B Instruct (int4) — the largest on-device model yet

Adds Llama-3.1-8B-Instruct at int4 to the Chat picker. It decodes entirely on the GPU (Vulkan) with 0 CPU fallback, verified on-device: peak GPU working set ~6.4 GB, so a dense 8B fits on an ~11–12 GB phone-class GPU. Weights: katolikov/Llama-3.1-8B-vknn (~5.3 GB), reusing the Llama-3 chat dialect + tokenizer. Compiling a dense 8B int4 became possible thanks to a compiler fix that stores int8/uint8 initializers natively instead of widening them to fp32 (behaviour-preserving — every existing model's .vxm is byte-identical).

Remote model catalog — ship a model without an app update

The model list is now the built-in catalog merged with a remote catalog.json (GitHub raw). Adding a model becomes a one-line manifest edit instead of an APK rebuild: the app fetches the manifest on launch, merges it over the built-in list (remote wins by id), caches it, and falls back to the built-in list offline. Entries reference a built-in chat dialect by id; unknown-dialect entries are skipped.

APK: vknn-demo-1.2.5.apk (arm64, debug-signed).