Repository navigation
Compatibility
What runs, in which format, on which hardware, and how well it is tested.
| Family | Status | Notes |
|---|---|---|
Llama 2 / 3, Mistral without sliding window, SmolLM2 (llama) |
runs | rope scaling: none, linear, llama3, yarn |
Qwen2, Qwen2.5 (qwen2) |
runs | bias on the q, k, v projections |
Qwen3 (qwen3) |
runs | per head q and k norm |
| Gemma, Phi, Falcon, Mamba, mixture of experts, vision and audio models | not supported | the loader refuses them with a message naming the architecture |
The engine is a dense decoder with grouped query attention, RMS norm and a gated MLP (SiLU). Head width up to 256. Logits are checked against Hugging Face Transformers (cosine similarity 1.000000 on SmolLM2 f16 and Qwen3 BF16) and perplexity against llama.cpp.
26 storage formats are read. Every one of them has a decoder, a CPU dot product kernel and a batch kernel; the GPU decodes the formats marked.
| Format | CPU | GPU (Vulkan) |
|---|---|---|
| f32, f16, bf16 | yes | yes |
| Q4_0, Q4_1, Q5_0, Q5_1, Q8_0 | yes | yes |
| Q2_K, Q3_K, Q4_K, Q5_K, Q6_K | yes | yes |
| IQ4_NL, IQ4_XS | yes | yes |
| IQ1_S, IQ1_M, IQ2_XXS, IQ2_XS, IQ2_S, IQ3_XXS, IQ3_S | yes | not yet |
| TQ1_0, TQ2_0, MXFP4, NVFP4 | yes | not yet |
A model that uses a format the GPU cannot decode runs on the CPU, and hk says so. Quantized weights are decoded exactly: each decoder is tested against the reference gguf package on blocks of random data (tests/test_quant.zig).
Byte level BPE and SentencePiece vocabularies, checked against llama.cpp on 301,948 tokens of real text. Chat templates run on a built-in Jinja subset, checked against Python's jinja2. A template that uses something outside the subset is reported, and the chat falls back to a plain transcript.
| Target | Status |
|---|---|
| x86-64 with AVX-512 and VNNI | tested (Ryzen 7 7445HS) |
| x86-64 with AVX2 | tested by forcing the level on the same machine (HK_KERNELS=avx2) |
| x86-64 without AVX2 | portable level, tested by forcing it; slow |
| x86-64 with AVX-VNNI and AVX2 only (Alder Lake and later) | built; has not run on such a processor |
| AArch64 (Apple Silicon, Graviton, Raspberry Pi 4/5) | built and cross compiled for Linux and macOS; not run on ARM hardware |
| Windows, macOS | cross compiled; not run |
| NVIDIA GPU through Vulkan | tested on one card (RTX 3050 Laptop) |
| AMD, Intel and Apple GPUs through Vulkan | not tested |
| Metal, ROCm, CUDA, TPU | no backend |
See Hardware and Kernels for how the kernels are chosen and what has and has not been verified.
HK Neural Tensor Framework | Licensed under Apache 2.0 | GitHub Repository