Two things change what a colibrì release is: a third GPU backend, and — for the first time — archives that actually contain the models the front page promises.
162 commits, 29 pull requests, 25 of them from contributors.
🔴 If you downloaded v1.3.0 to run Kimi K3 or Inkling, it could not work
Every archive until now contained c/colibri alone. Point it at Kimi K3 and there was no engine — while the README front page promised four model families. Nothing you could have typed would have helped (#720).
The cause was worse than an oversight in packaging: inkling and kimi_k3 did not compile on Windows at all. Their serve loops polled stdin with fd_set/select, which msys2/UCRT64 does not provide in that form. Measured across the release matrix: Linux ✅, macOS ✅, Windows ❌.
Three fixes, in order:
- #736 — the portable version now lives once in
compat.h, carrying the two bug fixescolibri.chad already absorbed: #139 (select()on a pipe handle routes to winsock and always returnsSOCKET_ERROR) and #195 (anonymous pipes are not waitable objects;PeekNamedPipefails on file/console handles). Copying that a third and fourth time would have reintroduced both in engines where nobody would look. - CI now builds every engine on Linux, macOS and Windows. Before this it built
colibri+inklingon Linux only andkimi_k3on nothing — which is exactly why a whole engine could stop compiling on two platforms and stay invisible for months. - #737 — archives ship all four engines plus
tools/k3_tokenizer.py(Kimi shipstiktoken.model, nottokenizer.json; without that script the engine is present and still undrivable). 863 KB for all four. The release job now asserts each engine is present, executable, and where the launcher's resolver looks for it.
From this release you can download and run any of the four families directly.
🎮 Vulkan: a third GPU backend
#418 (@steve-m) — the expert tier, dense projections and the MLA attention core on any GPU with a Vulkan 1.2 driver. The only backend for cards the vendor stacks no longer support — an RX 580 works — and competitive with ROCm on RDNA4 (#523 measured 19–24% faster on an RX 9070 XT, after the reporter corrected their own confounded first comparison).
Includes fmt=4 grouped-int4 and fmt=5 int3-g64 decode, 256 MB allocation arenas that remove the tier-size submit tax, priority-classed VRAM allocations with budget-capped fill, a second-device expert tier (COLI_VK_DEV2 — e.g. an RX 580 beside the primary card), single-submit fused prologues, and an explicit warning when ReBAR leaves VRAM host-invisible rather than failing obscurely.
#705 (@steve-m) — Vulkan fmt=7 MXFP4 decode and the Kimi K3 GPU tier. e2m1 nibbles with per-32 group scales decoded on the GPU; shared experts resident, fill-once routed-expert tier. Kimi K3 gets GPU acceleration without leaving its native QAT MXFP4 format — no requantisation, no conversion step.
🍎 Metal
- #457 (@monotophic) —
fmt=4grouped-int4 decode. - #732 (@michael-denyer) —
fmt=6E8/IQ3 expert decode with the Walsh–Hadamard rotation on the GPU, closing #637. - #386 (@monotophic) — platform-aware expert-cache defaults from an honest storage probe.
⚡ Performance and correctness
- #738 — the OpenMP team is sized by physical cores on
kimi_k3andolmoe, which had no tuning at all. #718 measured 2.3× on Zen3 from the thread count alone. They get the sizing half deliberately, not the spin-wait half: #707 measured −2.2× from spin-wait on a low-residency host, and Kimi is the most disk-bound engine here. - #739 —
COLI_CUDA=0no longer disables the OpenMP tuning. The guard tested for presence, so the tuning was skipped for exactly the users with no GPU, who most need their CPU team configured. Reachable in practice:coliitself writesCOLI_CUDA="0"when the plan does not use CUDA. Found by @fredchu in #707. - #741 — the CUDA VRAM tier is staged in rounds, so host RSS is bounded. It used to load the entire VRAM-ranked prefix into host RAM before releasing anything, making
CUDA_RELEASE_HOST=1unable to help: peak RSS was the wholeCUDA_EXPERT_GB. @mcollinswisc found this on a machine with more VRAM than system RAM (96 GB against 64 GB) — a regime that is arriving, and that the engine had assumed away. - #651 / #726 (@ZacharyZcR) — CUDA lossless compressed expert tier and native grouped E8 kernels. #673 — measured machine autotuning.
- #536 (@bherald) — learned partial mirror planner. Plus multi-SSD N-way read replicas with
O_DIRECTstriping, and multi-workerPILOTprefetch.
🧠 One routing history for every engine
#716 / #719 / #742 (@terrizoaguimor) — .coli_usage had two mutually unreadable writers under the same filename and two engines that produced nothing at all. route_trace.h is now the single format; kimi_k3 has routing telemetry for the first time, and inkling adopts it while still reading every IKU1 history it ever wrote. (olmoe remains — #700 stays open and says so.)
📦 Formats
#528 / #529 (@monotophic) — fmt=8 fp8-e4m3-b128 passthrough, and the self-describing container stamp (TRUST-VERIFY-REFUSE) with the FORMATS.md registry.
🐛 Fixes and docs
#728 (@terrizoaguimor) — /home/vincenzo/glm52_i4 removed from the shipped code. Every error message named a maintainer's home directory, and tools/download_glm52.py defaulted a 400 GB download there.
#542 OLMoE chat mode (@opxyc) · #621 per-expert causal-ablation harness (@jeswr) · #620 OpenAI tool-calling docs (@jeswr) · #725 Claude Code system-role fallback in serve (@t83714) · #511 the tiny-oracle expectation now says what it does (@CooperSheroy) · #723 converter selftests without a model source (@ZacharyZcR) · #709 / #740 README and website describe four families instead of one.
#731 → #733 — TEST_BINS derives from the build rules. c/Makefile had one hand-written line listing every gate, so any two PRs that each added a test conflicted by construction — that file appeared in 26 of 40 open PRs.
Thanks
Twenty-five contributor PRs. @steve-m built the Vulkan backend and the Kimi GPU tier; @monotophic the Metal path, the fp8 format and the container stamp; @ZacharyZcR the CUDA tier, autotuning and E8 kernels; @terrizoaguimor the shared telemetry and a run of CUDA fixes; and @michael-denyer, @bherald, @jeswr, @opxyc, @t83714, @CooperSheroy.
Equal thanks to the people who filed measured bug reports — @MichaelFomenko, @mcollinswisc, @fredchu, @dlewis7444, @noobdev-ph and others. The Windows and packaging gaps were found by users, not by us, and several fixes here exist only because someone took the time to reproduce a problem properly.
One thing worth saying plainly: #386 was rebased eleven times in thirteen days, the last one caused by us merging a fix ahead of it instead of after. Merge order is oldest-clean-first from now on — a green, unconflicted PR merges ahead of anything opened after it.
Full changelog: v1.3.0...v1.4.0
