Supported Models

Supported Models

VKNN runs a broad range of ONNX models on the GPU. The op table (see Operator Coverage) covers CNNs, detection, transformer/attention models, and an autoregressive LLM. The families below are validated on-device against onnxruntime goldens.

Family	Models	Notes
Image classifiers (CNN)	ResNet-50, MobileNetV2 / V3, EfficientNet-B0, Inception-v3, DenseNet-121, SqueezeNet 1.1, ShuffleNet, MnasNet	Run on the GPU (NC4HW4), fp16 storage / fp32 accumulate. cosine 0.9995–1.0 vs onnxruntime; a whole model runs with 0 CPU fallbacks.
Object detection	YOLOv8n (640×640)	Runs 100% on the GPU (1 segment, no CPU fallback); the flat row-major op path keeps the DFL / box-decode head on the GPU.
Transformer encoder (3DGS)	YoNoSplat feed-forward 3D Gaussian Splatting encoder (965M-param, DINOv2 ViT-L/14 backbone + RoPE decoders + Gaussian / camera heads)	Runs end-to-end on the GPU in one segment over ~8700 nodes; paired with a from-scratch Vulkan 3DGS rasterizer that consumes the Gaussian outputs.
Autoregressive LLM	Qwen2.5-Coder-0.5B (qwen2 architecture, ~494M params, apache-2.0)	An autoregressive decoder run entirely on the GPU with zero CPU compute fallbacks, via a host-driven KV-cache decode loop. See Running an LLM on VKNN.

Performance context

Against MNN (Alibaba's production inference engine) on the same device, same model, fp16, thermal-controlled medians, VKNN is faster than MNN's absolute best backend (the min over OpenCL-HEAVY, CPU-4-thread, and Vulkan) on 8 of 9 benchmarked CNN/detection models and at parity on ResNet-50. The YoNoSplat encoder is a model MNN's converter cannot handle, so VKNN is the only engine that runs it correctly on-device. Methodology and per-model numbers live in the repo's docs/benchmark.md.

Bringing a new model

1. Scan for gaps — tools/scan_unsupported_ops.py model.onnx lists unrecognized ops; the authoritative check is vknn_compile model.onnx out.vxm --support-report r.json + tools/check_model_support.py --engine-report r.json (see Operator Coverage).
2. Close any gap — usually an export-side or import-lowering fix, or a new op via the Adding an Operator recipe.
3. Compile and run — vknn_compile to a .vxm (declaring dynamic shapes with --shape / --bucket), then run with vknn_run_io (any multi-input/multi-output model) or vknn_classify (image classifiers). See Building VKNN.

Not supported

RNN / LSTM / GRU, data-dependent control flow (Loop / If / Scan / NonMaxSuppression), training ops, sparse tensors, and the long tail of the ONNX opset. Adding an op is mechanical; until it is in the op table the model will not import.