Home

1bit engine

A model-agnostic C++ inference engine for AMD Ryzen AI (Strix Halo). It runs inside Lemonade: Lemonade's onebit recipe runs 1bit serve, which serves each model behind an OpenAI-compatible API on the NPU, on the iGPU (HRX and Vulkan), through ZINC (including NVIDIA) or MLX. Repository docs live in docs/. This wiki records measured results, hardware facts and working practice.

Status (2026-09-23, evening)

Step	What	State
1	The engine runs inside Lemonade: 1bit serve (#21), vendored Lemonade removed (#22); the onebit recipe passes Lemonade's LLM tests on Vulkan and HRX, being prepared for upstream	merged (#6, #21, #22)
2	HRX + Vulkan in one llama.cpp build	merged (#7); follows AMD's live ggml-hrx, bumped daily (#11, #13)
3	NPU fast lane on full ELFs, served through Lemonade's onebit recipe	merged (#8, #9): Qwen3-0.6B at 91 tok/s; XDNA driver + XRT pinned upstream and built privately (#12). Layer kernel still captured: our own dx, from scratch (branch npu-dx, IRON + Peano pinned; M0 full ELF builds)
4	Laya router	source + 3 checkpoints pinned, hash-verified fetch (#18); C++ scorer port next
5	Every HF model, kept current (registry + daily census)	planned
+	Apple: Lemonade mlx recipe (lemon-mlx-engine)	merged (#10), verified on an M4
+	Hugging Face tokenizers v0.23.2 behind our C ABI: any tokenizer.json, byte-exact on 18 models (#19)
+	Linux kernel pinned to upstream v7.3-rc4 with amdxdna in-tree; packages built, not installed (#17)
+	ZINC pinned upstream; zinc Lemonade recipe; NVIDIA through ZINC's CUDA backend	merged (#14, with #15): Vulkan on Strix Halo 295 tok/s; RTX 5090 167-173 tok/s (Qwen3.5-9B, " Paris.")

Pages

• NPU: what XDNA 2 multiplies natively (int8 x int4, no ternary), the fast lane, the open dx kernel's speed-up series, and how to read an ERT timeout.
• Quantization: why 4-bit on the NPU, ternary to 4-bit, Unsloth Dynamic GGUFs on the GPU, and the lean option (ROCmFP4, ROCmI4).
• GPU HRX and Vulkan: the two iGPU devices, measured speeds, and building.
• NVIDIA and ZINC: ZINC's CUDA backend on a rented RTX 5090, its model coverage per backend, and how to rent and reach the box.
• Apple Metal and MLX: the Metal backend and the MLX engine on an Apple M4.
• Working on the shared NPU: rules for agents sharing the one NPU.