Home

hk

hk is a local LLM runtime and a model container, written in Zig. One binary downloads a model from Hugging Face, converts it to the .hk format while it downloads, and runs it in a terminal or behind an OpenAI compatible server.

This wiki says what each part does, how to use it, and how well it works, including where it does not work yet. If a page makes a claim, the claim has a number or a test behind it, or it says that it does not.

Start here

If you want to	Read
install it and run a model	Getting Started
get a model from Hugging Face	Downloading Models
know every command and flag	CLI Reference
run it as an API server	Inference and Serving
see what runs and what does not	Compatibility
understand the speed and memory numbers	Benchmarks and Performance
know how it uses your CPU or GPU	Hardware and Kernels
replace another tool with it	Transition Guide
fix a problem	FAQ and Troubleshooting

What it is good at

• Memory. Weights are memory mapped from the file and never copied, the KV cache grows in small segments as the context fills, and the converter and downloader use bounded buffers. A running model costs a few tens of MiB of private memory on top of the file. llama.cpp, measured the same way, costs several times more.
• Prompt processing. On the CPU, hk reads long prompts faster than llama.cpp on eight of ten test models, ties on f16, and is 9 percent behind on IQ4_XS. See Benchmarks and Performance.
• Decode speed is the open gap: 92 to 99 percent of llama.cpp on the same models (tied on f16).
• One binary for every instruction set level. The compute kernels are built once for each level of the processor family (AVX-512 with VNNI, AVX2, plain x86-64; NEON with and without dot product on ARM) and the right one is picked when the program starts. Nothing is tuned to the machine it was built on.
• Downloading. hk pull converts as it downloads, resumes after a dropped connection, and checks the Hub's SHA-256 before the file is put in place.

What it does not do yet

• It runs Llama, Qwen2 and Qwen3 style dense models. Gemma, Phi, sliding window attention, mixture of experts and vision models are not supported.
• The GPU backend uses Vulkan. It runs a whole model on one device, and it has been tested on one NVIDIA card. There is no Metal, ROCm or TPU backend. See Hardware and Kernels.
• The ARM kernels compile and pass the portable tests, but nobody has run them on ARM hardware yet.

The research half

The .hk container and the Python package carry features that predate the engine: growing a model's width, depth and vocabulary, a hash chained version history stored inside the file, 2:4 structured sparsity, in-place metadata edits and a self-training loop. They work in their unit tests. They have not been validated on large models or compared with other training stacks, so each page about them says what is tested. See Dynamic Architecture Growth, In-Container Version Lineage, Storage and Sparsity, Training and Fine-Tuning and Autonomous Self-Training.