VKNN 1.2.0
On-device GPU LLM decode gets faster and the demo app gains a model picker.
Runtime optimization (LLM decode, Vulkan)
- Engine-side greedy argmax (8-byte readback per token instead of the full logits row).
- RoPE chain fusion (one dispatch per q/k site, ~288 fewer dispatches per token).
- Fused decode attention (one flash-lite kernel per layer over the GQA KV cache).
- Decode GPU span ~21.6 → ~19.7 ms/token; per-token host loop ~9 → ~0.5 ms.
- All fusions are load-time, hint-gated, and never change a compiled
.vxm— existing models speed up on load.
Robustness
- Out-of-range Gather/RoPE indices and undersized weight uploads hard-error instead of crashing.
- Multi-bucket weight-lifetime crash fixed (shared mmap blob across prefill/decode plans).
vknn_compilerequantizes every bucket of a multi-bucket.vxm;vknn_run_iogains--bucket N.
Demo app
- Per-tab fp16 / int4 model picker (persistent), whole-window prompt prefill, engine-side argmax.
- 3D-splat capture gains a parallax motion gate and a capture-quality warning.
Gates
10-CNN + yonosplat8 byte-identical vs 1.1.0; chat/VLM token streams unchanged; 0 CPU fallbacks; 311 host tests; both target devices.
vknn-demo-1.2.0.apk is the Android demo (Chat / VLM / 3D Splat / Library).