Kapitsa 0.2.9

What's Changed

• Benchmark: MacBook Pro (14-inch, 2026) · M5 Max · 64 GB · Qwen3.8 27B 4-bit by @greencat667 in #331
• Benchmark: Mac mini (M1, 2020) · M1 · 16 GB · Qwen3.5 4B 4-bit by @a-chris in #325
• bench: the Dory cached follow-up pays up to about 40 ms for the tensor attention by @alisher-amantay in #329
• Logit floors: Flint on the M5 Pro (own floor, NI_QMT on/off, reorder gate; per-chip floor files) by @alisher-amantay in #330
• Flint GGUF imports run their IQ experts as stored: unsloth's 2-bit Flint in 11.73 GB, 61.3 tok/s decode on the M1 Pro 16 GB by @fearampgreed in #324
• Poster: M5 Pro 27B after #318, Kapitsa 618 / 79 tok/s (was 459 / 77; 4K prefill 1.35x), headline stays 24 of 26 by @alisher-amantay in #332
• bench: the Dory cached follow-up cost, measured interleaved by @alisher-amantay in #336
• Bench corpus: 650,000 characters for 128K prompts (first 144,500 unchanged); sizes past it fail clearly by @alisher-amantay in #334
• Engine: take cached suffixes up to 128 rows, not 64 by @alisher-amantay in #335
• Poster: show the M5 Pro 27B decode as the tie it already counts as (≈78 on mlx-serve, orange bar); numbers unchanged by @alisher-amantay in #339
• vision: avoid the extra pixel-buffer copy in QwenViT by @chainhelen in #337
• bench/results: Flint on the M3 Pro on AC power supersedes #306's battery run (Kapitsa 73.6 / llama.cpp 50.9 tok/s decode, decode within 1%) by @alisher-amantay in #341
• Flint: the routed experts' prefill GEMMs on the M5's tensor units (prefill 1.63x at 4K, 1.45x at 16K) by @alisher-amantay in #340
• Kapitsa 0.2.9 by @alisher-amantay in #343

New Contributors

• @greencat667 made their first contribution in #331
• @a-chris made their first contribution in #325
• @chainhelen made their first contribution in #337

Full Changelog: v0.2.8...v0.2.9

Kapitsa 0.2.8

Cached follow-ups read their new prompt as verify passes (#315, #328): on a dense target that verifies 16 rows wide (the 27B), a cached turn's short prompt suffix now runs as verify passes instead of a prefill. The first token comes 60-300 ms sooner (16K follow-up TTFT 402 to 325 ms and 401 to 341 ms). The small 4B stays on the prefill path, where the pass would cost 30-40 ms. NI_VPF=1 forces it on, NI_VPF=0 off.

27B Neural Engine split on the M5 (#318): the GPU's share of the FFN split runs on the M5's matrix units. On the M5 Pro, cold prefill goes from 494 to 619 tok/s at 4K (1.25x) and 441 to 516 at 16K (1.17x).

Dory's prefill attention on the M5's matrix units (#314): 1.24x / 1.22x in the kernel's A/B; the 4B's 4K / 16K prefill reads 1,656 / 1,311 tok/s on the M5 Pro.

Flint on macOS 14 (#303) and IQ3_XXS / IQ1_S expert kernels (#311, no default change).

Docs, charts and benchmark results (#313, #316, #317, #319-#321, #323).

Kapitsa 0.2.7

Recursive Flint 40B-A2.5B (#286): kapitsa serve --model flint (or kapitsa opencode --model flint) runs Recursive's coding MoE (2.5B active) from its 4-bit MLX export: a 22.6 GB download, for Macs with 32 GB or more. On an M4 Pro 48 GB:

• It reads prompts 1.1-1.2× as fast as rec 0.1.0, Recursive's own engine.
• It decodes level with rec on short prompts and up to 1.30× faster at 128K context.
• It decodes 1.6-1.9× as fast as llama.cpp b11804.

The numbers are in docs/benchmarks.md (#290).

Dory's prefill on the M5's matrix units (#287): Nemotron Dory1.0 4B's prefill GEMMs run on the Metal 4 tensor units on M5-class GPUs with macOS 26. On the M5 Max, prefill at 4K is 2.2× faster: 2,368 against 1,077 tok/s (#291).

The 35B's prefill attention on the tensor units by default (#293): Qwen3.6 35B-A3B reads 16K-token prompts 11% faster on the M5 Pro. NI_QATTN_G8=0 turns it off.

One GPU lock for the whole Mac (#292, #294): the benchmark harness and the tools share /tmp/kapitsa-gpu.lock, so two GPU jobs no longer overlap.

Docs and benchmark results (#280, #283, #284, #288, #289, #290, #291).

What's Changed

• Kapitsa 0.2.6 by @alisher-amantay in #282
• Poster: both M5 Pro 27B cells are the same-day interleaved pair by @alisher-amantay in #280
• Poster: the M5 Pro 35B-A3B cell moves to the tensor-unit prefill's run by @alisher-amantay in #283
• bench: store four same-morning 35B-A3B runs on the build before #279 (M5 Pro), with the 16-row block's null result by @alisher-amantay in #284
• attn: the 35B's prefill attention on the tensor units (G = 8 qattn_t, opt-in) by @alisher-amantay in #285
• Recursive Flint 40B-A2.5B: prefill 968 vs 796 tok/s against rec at 4K, decode on par, 1.6x llama.cpp's (M4 Pro) by @fearampgreed in #286
• dory: the prefill GEMMs on the M5's matrix units by @alisher-amantay in #287
• Poster: the M5 Max panel back, re-measured on main fd85910 (Kapitsa KFlash; Splash 1.4.0 on the 27B and 35B-A3B) by @itomatik in #288
• README: the M5 Max Nemotron table back, re-measured on main fd85910 (Dory 119, Lightning 267, Super-VL 94 tok/s) by @itomatik in #289
• Poster: Flint 40B on the M4 Pro (968 / 128 tok/s against rec's 796 / 127), and its 1K-128K context sweep (decode 1.30x rec, 1.92x llama.cpp at 128K) by @fearampgreed in #290
• Bench: the M5 Max Dory cell re-run after #287 (decode 119.1 tok/s unchanged, no loop flag; prefill 2,368 vs 1,077 at 4K) by @itomatik in #291
• bench: one GPU lock file for the whole Mac, in /tmp by @alisher-amantay in #292
• Follow-up to #292: the bench submit test takes a temporary GPU lock by @alisher-amantay in #294
• attn: the 35B's prefill attention on the tensor units by default by @alisher-amantay in #293
• Poster: the M5 Pro's Dory cells after #287 (prefill 2.5x at 4K, 2.1x at 16K) by @alisher-amantay in #297
• Benchmark: MacBook Air (13-inch, 2026) · M5 · 24 GB · Qwen3.5 9B 4-bit by @Ragul29 in #298
• attn: drop the kernel instance the GPU cannot create (attn_prefill9_q8_k32) by @alisher-amantay in #299
• poster: the 35B cell after #293 (prefill +3.6% / +10.5%, decode cell unchanged) by @alisher-amantay in #301
• Kapitsa 0.2.7 by @fearampgreed in #302

New Contributors

• @Ragul29 made their first contribution in #298

Full Changelog: v0.2.6...v0.2.7

Kapitsa 0.2.6

kapitsa bench compare works from an install (#275): the release tarball now ships bench/harness, so comparing engines on this Mac needs no git clone and no KAPITSA_HARNESS. install.sh and the Homebrew formula put the harness beside the licence notices, kapitsa update installs it too (and keeps a user's local.env in it), an unpacked release finds it beside the binary, and a shipped copy names its release in reports (harness 0.2.6) instead of "unknown".

35B routed experts on the tensor units (#279): NI_MOE_QTM=1 is now the default on M5-class Macs running macOS 26 (Qwen3.6 35B-A3B prefill +35.6% at 4K and +26.1% at 16K); NI_MOE_QTM=0 restores the CPU-path experts.

Prefill (+1-2% TTFT at 4K, #274): MoE chunk sizes are even, so a prompt a few rows past a multiple no longer pays an extra pass.

Docs and benchmark results (#276, #277, #278).

What's Changed

What's Changed

• Poster: drop the M5 Max panel until it is re-measured by @alisher-amantay in #276
• prefill: even chunk sizes for MoE, so a prompt a few rows past a multiple pays no extra pass (+1-2% TTFT at 4K) by @alisher-amantay in #274
• qmt: drop the dead tile instantiations (only m64_n64 is used) by @alisher-amantay in #277
• Poster: the M5 Pro 27B cell moves to the 16-row block's run; the pair with mlx-serve does not move by @alisher-amantay in #278
• bench: ship bench/harness with releases, so bench compare works from an install by @alisher-amantay in #275
• prefill: the 35B's routed experts on the tensor units by default (M5, macOS 26) by @alisher-amantay in #279
• Kapitsa 0.2.6 by @alisher-amantay in #281

Full Changelog: v0.2.5...v0.2.6

Kapitsa 0.2.5

DFlash2 16-row drafting block by default (#266): on a one-slot engine with a dense target, the drafter now proposes 16 rows per block (NI_DFLASH_BLOCK=8 restores the old block). Qwen3.8 27B on the M5 Pro: 87.6 tok/s greedy and 79.8 at T=0.6, against 76.1 and 71.7 with the 8-row block. Replies match plain decoding up to near-ties; the 16-row path is not gated by logit_compare. Needs an M5-class GPU on macOS 26; other Macs keep the 8-row block.

35B prefill on tensor_ops (#270): NI_MOE_QTM=1 runs the routed experts' GEMMs on the matrix units: +35.6% at 4K and +26.1% at 16K prefill on the M5 Pro (Qwen3.6 35B-A3B). Off by default, because its logits are not bit-identical (verify KL mean 1.07e-3).

Docs and benchmark results (#262, #263, #265, #267, #268, #269, #271).

Kapitsa 0.2.4

Server dynamic model selection (#191, @gnanasreekar): kapitsa serve can load and switch models on demand when a client names one in the request's model field.

M1 Max prefill (#195, #196, @gnanasreekar): larger automatic prefill chunks when a single slot fits, and expert-prefill kernels that stage two input tiles per iteration. Output is bitwise identical to 0.2.3 on the M5 Pro and prefill is neutral there; the M1 Max speedup is as claimed by the author and not re-measured by us.

DFlash2 16-row drafting block (#257): opt-in with NI_DFLASH_BLOCK=16. On the 27B it measured +5% greedy and +10% at T=0.6; the default 8-row drafter is unchanged.

Docs and benchmark updates (#252, #255, #256, #258, #260) and a 16-row check in dflash_attn_test (#259).

Kapitsa 0.2.3

Neural Engine keepalive (#243): a short prompt after a pause no longer pays a 70–120 ms Neural Engine wake.

KFlash wide tree verify (#245, #246): 16- and 32-row verify kernels for the dense Qwen targets and a budget that can pick them per cycle. Opt-in via NI_KFLASH_ROWS=8,16,32; the default (8 rows) is bitwise identical to 0.2.2 on the 4B and 35B. No throughput gain is claimed yet.

M1 Max (#193, #194, contributed by @gnanasreekar): newer 4-bit DFlash2 core and eight-row drafter projections for the 35B. Output is bitwise identical on an M3 Pro with no speed regression; the M1 Max speed claim is not verified by us.

README and benchmark updates (#241–#251).

Kapitsa 0.2.2

Fixes a crash in 0.2.1.

• kapitsa serve no longer segfaults with a DFlash drafter (the default 4B setup). After a prompt with a long system block, saving the SSD prefix cache (at stop, or when a conversation left a slot) dereferenced a missing drafter ring and killed the server (#237). With a drafter the cache now skips that one restore point; a restart restores from the turn checkpoint or live state instead. Models without a drafter are unchanged.
• README and poster: Kapitsa's speed is shown as prefill / decode on four Macs, with compact per-Mac tables (#233, #238); bench/README.md lists every run in bench/results (#235).
• Tools: tools/readme_images/reddit_grid.py (#236).

What's Changed

• community-bench: the table job warns instead of failing when Actions may not open pull requests by @alisher-amantay in #234
• bench/README.md: generated table of every run in bench/results, with CI check by @alisher-amantay in #235
• README: four-Mac poster with prefill, compact per-Mac tables, models in details by @alisher-amantay in #233
• Fix: segfault saving the SSD cache with a DFlash drafter (prefix checkpoint has no drafter ring) by @alisher-amantay in #237
• tools/readme_images: add reddit_grid.py, a 16:9 benchmark grid by @alisher-amantay in #236
• README: Kapitsa speed as prefill / decode in the poster and compact tables by @alisher-amantay in #238
• Kapitsa 0.2.2 by @alisher-amantay in #239

Full Changelog: v0.2.1...v0.2.2

Kapitsa 0.2.1

Highlights since 0.2.0:

• kapitsa bench compare: Kapitsa against the other engines installed on this Mac (oMLX, mlx-serve, mlx-lm, llama.cpp, Ollama, LM Studio) on one model, one table. It refuses to run while another model server is running (--ignore-running overrides) and ranks each measure with a verdict line. kapitsa bench results are submitted only as pull requests from a fork. compare needs the repository's bench/harness (set KAPITSA_HARNESS).
• Prefill and time to first token: a checkpoint at the end of the shared system prompt, copied into new conversations' slots, so a new question or subagent starts from it (#198, #202); dense 4-bit GEMMs and attention on Metal 4 matrix units on the M5, NI_QMT=0 / NI_QATTN=0 disable them (#205, #215).
• Server: SSD prefix cache (--disk-cache, #212), JSON Schema structured output (#211), Flash-Next on 48 GB Macs (#213).
• Tools: tools/logit_dump and logit_compare output checks (#208, #210); the community result checker reads multi-line catalog variants (#222).

What's Changed

• README: "no spec" for engines without speculative decoding by @fearampgreed in #190
• Residency set off the command queue, kept wired by a heartbeat unless memory is under pressure: a request after a pause no longer waits for the model to be wired again (35B server, 10 s pause: first token 432-535 -> 49-118 ms; decode unchanged) by @fearampgreed in #206
• KFlash batch budget: batched decode's verify rows chosen every cycle from the members' calibrated drafts and the measured cost (M4 Pro, 3 at once: 35B +16-20%, 27B +15-23%; 2 and 4 at once -1 to +4%; a request alone unchanged) by @fearampgreed in #207
• SSD prefix cache in the server (kapitsa serve --disk-cache): a follow-up after a restart starts from the SSD (16K tokens: 0.16-0.18 s to the first token on the 35B vs 17-18 s cold, 0.65-0.70 s on the 27B vs 47-48 s; the RAM path's greedy text) by @fearampgreed in #212
• tools/logit_compare.py: two logit dumps compared in fp64 and gated at a noise floor (192 full-vocabulary rows in 0.7 s with numpy, 19 s without); replaces tools/nh_vcheck.py by @fearampgreed in #208
• bench: opt-in --agentic metrics (cold start, context sweep, cache, sustained decode) by @alisher-amantay in #192
• bench: kapitsa bench compare, Kapitsa against the other engines installed on this Mac by @alisher-amantay in #197
• Keep a checkpoint at the end of the shared system prompt (prefix reuse for new questions and subagents) by @alisher-amantay in #198
• Copy the shared-prefix checkpoint into a new conversation's slot (first question starts warm) by @alisher-amantay in #202
• Structured output: JSON Schema response formats, forced and strict tool calls under grammar masks (4B: 54 of 54 replies schema-valid; a constrained reply decodes as fast as the same tokens unconstrained, 168 vs 167 tok/s; unconstrained decode and prefill unchanged) by @fearampgreed in #211
• Output check: tools/logit_dump and tools/logit_floor.py: full-vocabulary logits teacher-forced through the production decode paths, and a model's noise floor measured from them (the same build twice bitwise on every path of the 35B, 27B, 4B and Dory) by @fearampgreed in #210
• Prefill: dense 4-bit GEMMs on Metal 4 tensor_ops (3x kernel throughput, 1.3-1.5x prefill on Qwen3.5-4B) by @alisher-amantay in #205
• Flash-Next Q2_0 on 48 GB Macs with routed experts streamed from the SSD (M4 Pro, 12 GB store: 16-18 tok/s decode, 237-407 tok/s prefill; sparse n-gram table; DFlash2 plumbing only) by @fearampgreed in #213
• Prefill: attention on Metal 4 tensor_ops for head dim 256 / 4 q heads per KV head (+13% prefill at 16K on Qwen3.5-4B) by @alisher-amantay in #215
• kapitsa bench: community results only as pull requests, no issue path by @alisher-amantay in #217
• bench compare: a table you would screenshot (verdict per measure, green winners, finer bars) by @alisher-amantay in #219
• Benchmark: MacBook Pro (16-inch, Nov 2023) · M3 Max · 48 GB · Qwen3.8 27B 4-bit by @VladimirShleyev in #201
• Benchmark: MacBook Pro (16-inch, 2024) · M4 Max · 48 GB · Qwen3.8 27B 4-bit by @pavelxdd in #214
• Benchmark: MacBook Pro (14-inch, 2024) · M4 · 16 GB · Qwen3.5 9B 4-bit by @fearampgreed in #223
• Benchmark: MacBook Pro (14-inch, 2024) · M4 · 16 GB · Qwen3.5 4B 4-bit by @fearampgreed in #224
• Benchmark: Mac mini (2024) · M4 · 32 GB · Qwen3.8 27B 4-bit by @fearampgreed in #225
• community_check: read catalog variants whose .quant spans several lines by @alisher-amantay in #222
• Benchmark: MacBook Pro (13-inch, M2, 2022) · M2 · 16 GB · Qwen3.5 9B 4-bit by @fearampgreed in #226
• Benchmark: MacBook Pro (13-inch, M2, 2022) · M2 · 16 GB · Nemotron Dory1.0 4B NVFP4 by @fearampgreed in #227
• Benchmark: MacBook Pro (13-inch, M2, 2022) · M2 · 16 GB · Nemotron 3 Nano 4B FP8 by @fearampgreed in #228
• Benchmark: MacBook Pro (14-inch, 2024) · M4 Max · 128 GB · Qwen3.8 Flash-Next 180B-A7B 4-bit by @quentinlampin in #203
• Community results: RESULTS.md with the 24 runs by @fearampgreed in #229
• test_bench: skip bench/results files that are not a run's record (#188's kflash-merge.json is a list) by @fearampgreed in #230
• Kapitsa 0.2.1 by @alisher-amantay in #231

New Contributors

• @VladimirShleyev made their first contribution in #201
• @pavelxdd made their first contribution in #214
• @quentinlampin made their first contribution in #203

Full Changelog: v0.2.0...v0.2.1

Kapitsa 0.2.0

What's Changed

• Nemotron 3 Nano 4B (nano-4b, FP8 by default) and Nemotron Nano 9B v2 (nano-9b) in the catalog by @fearampgreed in #127
• README: the CLI's other commands under Install by @fearampgreed in #128
• Nemotron-H: an FP8 dense checkpoint's BF16 LM head loads as 8-bit rows (Nano 4B FP8: +4-7% decode) by @fearampgreed in #129
• README: Quickstart and CLI as two sections by @fearampgreed in #130
• Nemotron Nano 4B FP8: K % 128 != 0 tail variants of the 2-32-row FP8 kernels (prompt lookup 34.6 -> 37.1 tok/s on an M1 Pro) by @fearampgreed in #131
• M1: prefill GEMMs of 3/5/6/8-bit and 4-bit group-32 weights on M-fast grids (Qwen3.5-4B 6-bit prefill +10-12%, bit-identical) by @fearampgreed in #133
• Qwen3.6-35B-A3B on a 16 GB Mac: GGUF routed experts kept in IQ2_XXS / IQ2_S / IQ3_S (M1 Pro: 52.6 vs llama.cpp 38 tok/s) by @fearampgreed in #134
• Benchmarks: MacBook Pro M1 Pro 16 GB, every model that fits (Qwen3.5-9B, Qwen3.6-35B-A3B 2-bit, Nemotron Nano) by @fearampgreed in #135
• Benchmark: MacBook Pro (16-inch, 2021) · M1 Max · 64 GB · Qwen3.8 27B 6-bit by @fearampgreed in #138
• nh_eval: --vcheck dumps Nemotron-H 8-row verify logits (or the one-row decode's) for comparison; NI_SEED for --gen by @itomatik in #140
• 35b:16gb: Qwen3.6-35B-A3B on 16 GB Macs from unsloth's 2-bit GGUF, downloaded and imported in one step (M1 Pro 16 GB: 53.2 tok/s) by @fearampgreed in #141
• tools/nh_moe_bench: a verify's NVFP4 routed experts on Lightning's shapes (per-pair up + down + sum), outputs compared to the bit across variants by @itomatik in #144
• Lightning verify: the routed experts' per-pair products on 4-simdgroup threadgroups on the M5 (bitwise the same) by @itomatik in #145
• Lightning on the M5: NVFP4 routed experts in layout L2 (no decode table), every expert kernel's output the same to the bit by @itomatik in #146
• Lightning's verify router: logits with a step's inputs loaded up front (6.9 -> 4.9 us per layer), routing with 4 experts per lane when E <= 128 (8.1 -> 4.8), bitwise the same by @itomatik in #147
• Lightning verify on the M5: the 8-row NVFP4 GEMMs on the GPU's matrix units (nh_x4t_v8_l0: LM head 379 vs 467 us, shared expert 14.5 / 15.6 vs 18.6 / 19.0) by @itomatik in #148
• Lightning verify: RMSNorm straight into half rows, the attention heads' sums straight to half (59 fewer passes per verify, bit for bit) by @itomatik in #149
• Lightning verify on the M5: the Mamba scan at 4 state rows per simdgroup (nh_ssm1r4: 22.7-24.5 vs 35.3 us per layer, bit for bit) by @itomatik in #150
• NI_PROFILE=2: the per-dispatch GPU profile per kernel and grid by @itomatik in #151
• Lightning verify: the conv ring's writes with the grouped norm, q / k / v in one pass (35 fewer passes, bit for bit) by @itomatik in #152
• Lightning on the M5: attention in chunks of 64 keys (partial + reduce 27.0 vs 36.8 us per layer at ~600 positions) by @itomatik in #153
• Lightning verify: the rows' samples in two passes instead of two per row (the same tokens) by @itomatik in #154
• Nemotron-H prefill: the tile GEMMs on the M5's GPU matrix units (nh_gemm_t: Mamba in/out, attention, shared and routed experts) by @itomatik in #155
• Nemotron-H prefill: attention on the M5's GPU matrix units (nh_attn_t) by @itomatik in #156
• Nemotron-H prefill: the Mamba-2 scan in chunks of 32 rows as matrix products on the M5 (nh_ssd) by @itomatik in #157
• Nemotron-H on the M5: 4096-row prefill chunks for the MoE models when they fit by @itomatik in #158
• NI_PROFILE: Encoder::profile_tag prefixes the names of a phase's dispatches; nh_eval --spec tags the drafting half (d:) by @itomatik in #159
• DSpark: double-buffered decode, the next cycle's drafting half chained behind each verify on the GPU (Lightning: wall time per cycle down to the GPU time, ~0.3 ms less) by @itomatik in #160
• nh_cand2: one simdgroup per row merges the chunk lists (DSpark pass: ~90 us in the pipeline -> ~0, the same candidates) by @itomatik in #161
• DSpark: fused passes in the drafting pass and the commit (19 -> 14 dispatches per layer, 4 -> 2 per commit layer; drafting 2.36 -> 2.20 ms, the same drafts) by @itomatik in #162
• DSpark: the block attention over context pieces, q/k norm + RoPE inside it, q|k|v in one product (drafting 2.30-2.38 -> 1.98-2.01 ms) by @itomatik in #163
• nh_eval --spec: NH_SPEC_LOG writes each cycle's DSpark pass (candidates, log-probs, corrected log-probs, picks), verified rows and acceptance; NI_DSPARK_DEPTH=d verifies d drafts every cycle by @itomatik in #164
• DSpark on the M5: the attention projections on nh_xq8_v8, the x8 design for Q8 (drafting 1.90-1.91 -> 1.83 ms) by @itomatik in #165
• nh_cand2: the chunk lists loaded 8 per lane before the stores, the log-sum-exp's terms on the chunks' lanes (15.6 vs 24 us isolated, the same candidates) by @itomatik in #166
• DSpark: the Markov walk in two passes, every candidate pair's dot at once, then the walk on one simdgroup (30 -> 13 + 10 us isolated, the same drafts) by @itomatik in #167
• nh_eval NH_SPEC_LOG: at T > 0 also the target's distribution of rows 0..6 as its sampler draws from it (top-k, top-p, temperature) and the drafts by @itomatik in #168
• DSpark on the M5: dynamic depth ratio 0.3 (was 0.2; greedy harness verify 11.70 -> 11.35 ms per cycle at the same 150 cycles) by @itomatik in #169
• Benchmark: MacBook Pro (16-inch, Nov 2023) · M3 Max · 48 GB · Qwen3.8 27B 4-bit by @yargeu in #143
• Lightning verify on the M5: the 8-row FP8, BF16 and Q8 products at 8 weight rows per threadgroup (twice the threadgroups, bit for bit) by @itomatik in #171
• Nemotron-H decode on the M5: one-row FP8 and BF16 products at 2 rows per simdgroup, attention's q, k and v in one pass (bit for bit) by @itomatik in #173
• Lightning on the M5: the Mamba short scan's row sums side by side, its gate staged (verify 33.3 -> 24.8 us per layer at depth 7, bit for bit) by @itomatik in #174
• Lightning verify on the M5: the router beside the shared expert's up product, the routing beside its down product (2 fewer passes per MoE layer, bit for bit) by @itomatik in #175
• Benchmark: MacBook Pro (16-inch, 2024) · M4 Max · 64 GB · Qwen3.8 27B 4-bit by @geiman in #172
• Lightning verify on the M5: the rows' samples on simdgroup reductions (T = 0.6: 65.6 -> 47.9 us isolated per cycle, the same tokens) by @itomatik in #176
• Lightning verify: the routed experts' grids of a dynamic verify sized on the GPU to its live rows (no threadgroups for the dropped rows' pairs, the same outputs) by @itomatik in #177
• nh_ssd_scan: a barrier after each chunk's outputs (a simdgroup ahead staged the next chunk's x under a slower one's reads: an M5 prefill's Mamba outputs differed from run to run, Bolt's greedy decode with them) by @itomatik in #178
• Bolt verify on the M5: the 8-row IQ2_XXS shared expert on the GPU's matrix units (nh_x2t_v8: up 24.8 vs 35.4 us, down 26.6 vs 41.6; greedy harness decode 29.5 -> 27.8 ms per cycle) by @itomatik in #180
• Bolt verify on the M5: the BF16 router on nh_mv_bf16_rt, 2 weight rows x 2 token rows per simdgroup (12.2 vs 46 us per layer, the same logits to the bit; harness decode 27.2 -> 26.3 ms per cycle) by @itomatik in #181
• Bolt verify on the M5: routing and the IQ2 experts' bucketing in one dispatch, ranks without atomics (nh_x2_route_bucket; the same logits to the bit; harness decode 26.4 -> 25.8 ms per cycle) by @itomatik in #182
• Bolt verify on the M5: the IQ2 experts' down projection in 4-simdgroup threadgroups (52.2 vs 59.2 us per layer, the same outputs to the bit; harness decode 26.1 -> 25.8 ms per cycle); tools/nh_x2moe_bench by @itomatik in #183
• 8-row verify: the dense MLP's GEMMs on the M5 matrix units too (27B +5% per cycle, greedy identical) by @alisher-amantay in #139
• KFlash trees on every target: the drafter's best-first tree, verified exactly in the 8-row pass (M4 Pro, greedy: 27B +6% code, +16% prose) by @fearampgreed in #187
• PR CI (build, CPU tests, lint, size), AGENTS.md, Bugbot r...