omarchy-mlx v0.7.33

omarchy-mlx v0.7.33 fixes a wrong result in compiled functions on the Vulkan backend. It adds 2-bit and 3-bit gather_qmm. It adds a Muse Spark Open adapter for mlx-lm, with a quick start. The tag is main at 33630f9c.

Fix: a compiled rsqrt returned sqrt (#191)

mx.rsqrt is the Sqrt primitive with a recip flag. The fused elementwise chain mapped every Sqrt to the sqrt operation and ignored the flag. A compiled function with rsqrt in an elementwise chain returned sqrt(x) where it should return 1 / sqrt(x). The chain gate is on by default. Compiled RMS norms were affected.

• The fix is one line in chain_op_for. The eager path already read the flag.
• A new test in omarchy_compiled_tape_tests compiles the pieces of an RMS norm at [1, 24, 256] float32 and compares each with the eager result. I ran it on a CPU-only host with llvmpipe as the Vulkan device. Without the fix, 18447 of 46224 assertions fail. With the fix, 14 of 14 test cases and 46224 of 46224 assertions pass.
• On the 14-inch M2 Max, the pull request reports a compiled RMS norm with a maximum error of 0.57 on the first call and 11.8 on the second. Uncompiled, the errors were 9.4e-07 and 1.0e-06. That run is in the pull request text. It is not in a committed receipt.
• In that run the compiled function matched with MLX_DISABLE_COMPILE=1, which was the way around the bug.

2-bit and 3-bit gather_qmm (#189, #190, #193, #197)

mx.gather_qmm on the Vulkan backend refused every bit width except 4 and 8. It now runs 3-bit and 2-bit affine weights with biases in the transposed layout, with K a multiple of 32. The scale-only routes (GatherQQMM, QQMatmul) keep the named refusal, with a test.

• Scalar kernel for both widths. 3-bit codes are a bit stream along K, so a code can straddle two words.
• Subgroup kernels at the routed-MoE decode shape: 3-bit group 64 with f16, 2-bit groups 64 and 128. Everything else stays on the scalar kernel.
• Device gates. The same check script ran on two chips, one boot each. They are a 14-inch M2 Max and a 16-inch M1 Max, both on Honeykrisp with the same Mesa and driver packages. All configs print RESULT PASS. The 13-inch M1 has not run these checks.
• Per-call cost at the real Muse Spark Open expert shape. The shape is K 4096, N 4096, top-8, f16, 24 experts and one row. Each figure is the median of five calls on random weights:

Config	M2 Max subgroup	M2 Max scalar	M1 Max subgroup	M1 Max scalar
3-bit group 64	2.05 ms	14.69 ms	2.36 ms	69.25 ms
2-bit group 64	1.76 ms	14.25 ms	1.91 ms	70.73 ms
2-bit group 128	1.60 ms	14.21 ms	1.83 ms	71.04 ms

• Error against an fp64 reference equals the CPU stream's error to the printed digits at the real shape. The subgroup route is not bit-identical to the scalar route. At the real shape it differs by at most one f16 step.
• 2-bit with bf16 output at one row also takes the subgroup kernel. On the M2 Max the bf16 check passed for groups 64 and 128. At the real shape it took 1.73 ms and 1.61 ms, against 14.27 ms and 14.20 ms on the scalar kernel. The dispatch trace shows the bf16 subgroup kernel for the default route. 3-bit with bf16 output stays on the scalar kernel.
• Receipts: receipts/2026-10-16-gather-qmm-3bit/ and receipts/2026-10-16-gather-qmm-subgroup-3bit-2bit/.

Muse Spark Open adapter and quick start (#187, #192)

The mlx-community Muse Spark Open 3bit and 4bit checkpoints declare model_type muse_spark. mlx-lm main has no loader for it.

• serve/mlx_omarchy_muse adds the text-only class from the closed mlx-lm pull request 1975. It uses only mx operations.
• The file is byte-equal to head 117b84ca. Its sha256 is in docs/dependency-licenses.md.
• Nine adapter tests pass with mlx 0.32.3, on mlx-lm 0.32.0 and on 0.32.2.
• mlx-lm 0.32.2 ships a muse_spark module of its own. It reads the native checkpoint layout. register() now replaces it with the vendored class (#195).
• Set MLX_OMARCHY_MUSE_UPSTREAM=1 to keep the mlx-lm module.
• docs/muse-spark.md has the quick start.
• The adapter lives in a repository checkout. It is not in the release wheel.
• The routed experts use the 3-bit and 2-bit gather_qmm above.
• No Muse Spark Open decode speed is claimed here. The device checks above use random weights and no checkpoint was loaded in them.

Also in this release

• Chunked SDPA prefill drains its pinned chunk buffers. That fixes a 16384-token out-of-memory on a 16 GB M1.
• The gated-delta backward sums the gate gradient over all 32 lanes.
• The rope gate reads frequencies on the host by default.
• The queue-family priority query now reaches device creation.
• Ring sends and CPU comm primitives hold their outputs until the queued work ran.
• The backend runs on non-Apple Vulkan GPUs (NVIDIA, Intel). There is a one-command hardware run script, and collector reports record GPU capabilities.
• gather_qmm gains a tiled sorted-expert prefill route. The decode kernel computes several outputs per workgroup. Some 4-bit and 8-bit layouts get coopmat prefill.
• The Parakeet ANE worker is pinned to the performance cores.
• tools/vkbw is a Vulkan memory bandwidth microbenchmark.
• The README is split into a front door and nine docs, with refreshed tables.

git log v0.7.32..v0.7.33 has the full list.

Known limits

• The 2-bit and 3-bit paths have two chips and one boot each behind them. The 13-inch M1 is not covered.
• The number for a whole Muse Spark Open model is not in this release.
• The pinned 4-bit decode run on the uploaded aarch64 wheel (docs/release.md step 6) is not in these notes. The asset gate checks hashes, platforms, versions and the build commit. It does not check speed.

Assets

File	Bytes	sha256
mlx_omarchy-0.32.4.dev202610160507+33630f9-cp311-cp311-linux_x86_64.whl	10410078	dd9e7e352b201c073eeb5bcd9c357609df40062101ba3d07047dc9136912e74f
mlx_omarchy-0.32.4.dev202610160529+33630f9c-cp314-cp314-linux_aarch64.whl	422318222	6068ab8370a5e0ffd80778d103114cfb3f42b3153bf4c754465adab036c33d2c
omarchy-mlx-vendor-wheels-v0.7.33-cp314-aarch64.tar	467404800	bc9f8c7527e1324d25212922ca0afd2acc2411ec7cdd4b26e2773bffdef32f0e

SHA256SUMS covers every file in this release, including the .sha256 sidecar of the tar. Both wheels are stamped with the tag commit 33630f9c.

The Muse Spark Open quick start was extended after this tag (#198). That change is in the next release.

v0.7.32: int8 matmul exactness and speed, large-shape correctness fixes, GDN and SDPA serve routes, translator fixes

2026-10-10: mlx-omarchy v0.7.32 (int8 matmul exactness and speed, large-shape correctness fixes, GDN and SDPA serve routes, translator fixes)

Release: v0.7.32, tag v0.7.32 = ae5d0950bc4f2d47843c65112a212e21eb5e0060 (main at the freeze). aarch64 only.
Range v0.7.31..v0.7.32: 265 commits, 587 files, +47,151 / -972 lines. Gates ran on the packaged Honeykrisp driver
omarchy-mlx-vulkan 0.7.28-2 with Mesa 26.2.4-1 (see "Gates on the frozen wheel" and "Known issues").

Read the known issues before you install: training backward (item 2), a long-dispatch fault on the 14-inch M2 Max
(item 3), long attention that runs out of memory on the 16 GB M1 (item 7), and four failing test suites (item 1).

Shipped (user impact)

Serve/GPU: SDPA, GDN, allocator, gather_qmm

• SDPA prefill composed-first route for non-causal bf16 on coopmat
  devices (25052f0, #43); chunked composed route writes GQA output
  through the regrouped view (268a332, #46); causal flash launches the
  heaviest causal q tiles first, bit-identical, 1 to 5% faster
  (a8c7132, #49); opt-in causal flash (PR 41 family).
• GDN prefill masked rows take the single-pass recur32 route on G13
  parts: padded batched prefill 1.64x to 0.98x of sequential on G13C
  (cc9b891, #45; receipt in the PR).
• Allocator: submitted batches + quarantine drained before the OOM
  retry (0651686, #44); bin sizes above 1 MiB geometrically (8 per
  octave), which fixes the 16 GB offload out-of-memory (ad831f0, #47);
  receipt 830e87b: oMLX upstream pin, M1 16 GB A/B on the allocator-fix
  wheel: prefill 1.97x, decode 1.10x, token ids identical.
• gather_qmm fp16 decode takes the subgroup kernel: M1 64.1 to 4.13 ms
  per layer (ce4e77a, #42).
• 4 upstream backports (dbb15d6: compile scalar output, Python dtype
  narrowing, .npy shape validation, pickle bfloat16 strides).

Parakeet worker pin (#50)

• The built ANE worker is now PINNED: stamp step at wheel build, runtime
  checker, tests (adbdc64). Caveat: wheels built before the stamp
  step (pre-stamp) fail verify_runtime_assets. That is the checker
  working as designed; rebuild, do not bypass.

int8 matmul (chunked coop + swiglu coop, extends the headline below)

• Chunked int32 reduction on the f32 cooperative-matrix route
  (a7ce66b), swiglu through two f32 coopmat matmuls (9f6e1e1),
  rhs_offset honored in the coopmat value-half W loads (17734ee).

Stock driver warning

• One-shot warning when the selected driver is a stock Mesa older than
  26.2.4 (95e7b6f). Hosts are on Mesa 26.2.4 now (pacman -Q mesa);
  the warning is for stock-Mesa installs that never got the packaged
  Honeykrisp ICD.

Docs

• TensorFold H3 video: replication guide + patches + render script
  (325d849); fresh-clone replication verified byte-identical mp4
  sha256 36bd5df0a7b4b060304944beeeebb28e89de75ede22b1eaf3e31ce165fafee07,
  58 min 44 s, M2 Max (2b007ac, docs/tensorfold-video.md); the same
  seed-1 render on wheels 53bc1e3 and 95e7b6f produced the same hash
  (the int8 kernels are exact through the whole pipeline).
• README "How close to macOS" table: Linux as a percentage of macOS, same
  MacBook Pro, 5 MLX models + 2 llama.cpp GGUF, with receipts
  (c264198, 5444406).

HEADLINE: int8 matmul speed + exactness (the TensorFold H3 DiT path)

• fast.int8_matmul f32 routes rebuilt (8185470..9a991ff, 19 commits
  in the family since v0.7.31): bitwise A/B test matrix + naive kill
  switch (8185470); tiled shared-memory kernel, bit-identical to naive
  (fa41e66, default since 83a490a which also fixes an out-of-bounds
  shared write in X staging); cooperative-matrix f16 by f16 with f32 accumulation default
  route, bit-identical (a97dfc7); f32 cooperative-matrix route for
  the linear shapes (8f1a2ff): the scalar f32-FMA kernel is flat at
  ~330 GMAC/s on G13C, so the coop route streams to exact f32 temps via
  the converting copy with the same exactness argument as the scalar
  route (commit message, 8f1a2ff); coopmat flush/quarters/64-row-pad
  fixes (59c3a8f, 88f53e4, 2e5f28e); chunked int32 reduction
  lifts the scalar f32 route to group <= 131072 (9a991ff);
  registers-resident FMA accumulators (79cd72e); poison test + extended
  A/B matrix (e6504ef, dedd0fb); f32-FMA default + flags doc
  (a720464, docs/kernel-flags.md).

• Measured on the frozen wheel (release gate, same run, default route against the scalar route; GMAC/s; every
  shape bit-identical to the naive kernel, ab-summary ALL MATCH on each chip). Clip shapes: qkv = 6417x5376x16128,
  fc1-swiglu = 6417x5376x14336 with swiglu, fc2 = 6417x14336x5376 (group size 1024 for fc2).

  Shape	13-inch M1 (G13G)	16-inch M1 Max (G13C)	14-inch M2 Max (G14C)
  qkv	343.8 vs 79.9 (4.3x)	1367.1 vs 324.5 (4.2x)	1829.2 vs 412.1 (4.4x)
  fc1-swiglu	337.8 vs 80.1 (4.2x)	1364.0 vs 324.0 (4.2x)	1818.4 vs 411.9 (4.4x)
  fc2	280.2 vs 79.1 (3.5x)	740.8 vs 319.6 (2.3x)	1846.2 vs 410.4 (4.5x)

  The scalar route stays the fallback below the coop-capable shapes. Single run per chip; the rows come from the
  release gate logs, not from a repeated benchmark.

• This sits ON TOP of the dispatch-clamp fix (2df1aed, below): rows*n

  16,776,960 writes every output row.

HEADLINE: correctness fixes for large shapes (defects present in v0.7.31 and earlier)

One defect class: the one-dispatch thread/matrix clamps silently dropped
every output past the first dispatch. All documented in
docs/known-defects.md (80c3907, section "Int8Matmul, CastBool, and fused
rope_rms_norm went unwritten past one dispatch (16,776,960 threads)" +
"Batched linalg factorizations went unwritten past 65,535 matrices").

• fast.int8_matmul wrong for rows*n > 16,776,960 (2df1aed): tail
  never written past the 16,776,960-thread clamp (first bad element at
  row 2047 col 7936 = floor(16776960/8192) exactly); the TensorFold H3
  DiT hit this. Fix: dispatch_logical_chunked host loop (LogicalOrBool
  precedent). Regression: omarchy_int8_matmul_tests "writes every output
  row past the clamp" (21b9204).
• CastBool > 67,107,840 bools unwritten (ba6f6c4): stride-loop bool
  cast; regression omarchy_copy_offset_tests / test_copy_offsets.cpp
  (2^26 bools, first bad element 67,107,840; Honeykrisp M1 zero-return
  hazard).
• fused rope_rms_norm >= 65,535 rows unwritten (ba6f6c4): stride
  loop + word-tail guard placed after all barriers; regression
  omarchy_fast_ops_tests "fused" case (c57d5ea).
• Batched linalg (Cholesky/Inverse/LU/SVD/Eig/Eigh) unwritten past
  65,535 matrices (2dd7dc0): chunk batched factorizations past the
  workgroup clamp; regressions omarchy_linalg_ops_tests + omarchy_eig_ops_tests
  (91582d5 "must write every matrix past the one-dispatch clamp").
• Bonsai/GDN-VJP refuse by name above the clamp instead of clamping
  silently (ba6f6c4, 2dd7dc0).
• Stuck-submit diagnostic (c8104dd, 58e9881, 36703f5,
  73a2088): names the started-but-never-retired park.

Upstream correctness fixes (Vulkan backend primitives/shaders, landed after v0.7.31)

• bool MaskedScatter native (85c4a0b, merge f3dc90c, upstream
  #4635): dedicated masked_scatter_bool.comp shader + dispatch; verified on
  G13G hardware, matrix row updated (receipts/2026-10-08-bool-masked-scatter/README.md,
  landed via 1e87dc1).
• ArgReduce NaN propagation (b42203b): argreduce_suffix.comp now
  propagates NaN like the reference: argmax/argmin over rows containing
  NaN previously returned wrong indices deterministically (upstream suite
  test arg reduce NaN failed 6/6 before). Documented as
  OMARCHY-ARGREDUCE-NAN in docs/known-defects.md (:1057, FIXED same day);
  test coverage in overlay/tests/omarchy/test_reduce_ops.cpp; audit
  receipt receipts/2026-10-08-audit-followups/README.md (b0fe81f).
• empty Sort / ArgSort short-circuit (e7f56fc): empty arrays
  short-circuit before the dtype refusal instead of erroring; defect entry
  in docs/known-defects.md (same receipt b0fe81f).
• Scan::eval_gpu parent-aliased-output guard (a3ccf3a): shared-buffer
  view allocation guard in Scan eval_gpu prevents aliasing a parent
  buffer's output; failing-before test 7c03171 (tests: CumSum with a
  pre-allocated shared-buffer output); receipt b0fe81f.

Serve performance

• mlx-lm last-logits for the dense qwen3 family (0bdba12, both patch
  lines): cached prefill computes the quantized head only for the final
  prompt position on qwen3 dense models. Measured: 4B pf512 +9.0%,
  digests equal, fp64-reference error equal
  (receipts/2026-10-08-qwen3-last-logits). Proven on pristine wheels:
  g17-patch-series dev-box run 2026-10-08, both lines rc=0 + idempotent,
  "applied: mlx-lm-last-logits-qwen3.patch" in both
  (receipts/2026-10-08-release-0.7.32/g17-0732/). Self-contained patch
  fix (8fed8d8: local os import, no NameError on a stock 0.31.3 tree).
• Assistant wake word (338f639): opt-in openWakeWord ONNX listener,
  --wake-word; bytes coercion + per-detector peak counter (3ccab70);
  ALSA 'pulse' device with default fallback (573037b).
• Batched gated-delta prefill runs per-row on the fused kernel for B>1
  bf16 Dk=Dv=128 (95a879d, #40, receipt + device tests).
• ssm-maskless: left_padding setter clears the host mirror (7156b50,
  review fix).
• matmul direct-kernel route rows for G14C + lower m floors on G13G
  (edac8b4, #38); G13G f32 a @ b route row REMOVED (9b8ff10, #39: it
  lost 8 to 42% at k 2560 / n 9728).

llama.cpp demo (new user-facing path, 13 commits)

• omarchy llama.cpp demo end to end: pacman dep check + omarchy update as
  step 0 (9677365, cc5feab; refuses to build with a partial
  upgrade: llvm 23 vs llvm-libs 22 breaks llvm-spirv), path-independent
  wrapper, every Step 2/3 command works from the repo folder
  (81b22f2), --reasoning-budget 0 so Qwen3.5 does not print a 400-token
  thinking block first (e7...

Wheel for the H3 video guide (dev build 95e7b6f)

Pre-release wheel for the MiniMax-H3 video guide (docs/tensorfold-video.md).

This is a development wheel built from commit 95e7b6f of omarchy-mlx. It is not a supported release. It ships the native mx.fast.int8_matmul route (scalar f32-FMA, cooperative-matrix, chunked large groups, cooperative swiglu) that the guide needs. The newest published release, v0.7.31, has only the slower tiled int8 kernel.

• File: mlx_omarchy-0.32.4.dev202610090724+95e7b6f-cp314-cp314-linux_aarch64.whl
• SHA-256: 4dcae17fc998256018623dc73e65e8e82492865525f542905e3f93f2ddd3d291
• Platform: Linux aarch64 (Apple Silicon), Python 3.14
• Verified on an M2 Max (96 GB) with kernel 7.1.12-2-12.6 and Mesa 26.2.4: the int8 suite passes (190 of 190 assertions) and the 20-step render takes 58.7 minutes.

Install it last, with pip install --force-reinstall --no-deps <wheel>. Other pip packages that depend on mlx replace the files of this wheel with the PyPI mlx package, which has no int8_matmul.

v0.7.31 — batched bf16 coopmat QMM fix, kv-maskless series gate, batched GDN decode, routing default ON

2026-10-07 — mlx-omarchy v0.7.31 (batched bf16 coopmat QMM fix, kv-maskless series gate, batched GDN decode, routing default ON)

Release: v0.7.31, tag v0.7.31 = 9b5c938fe (origin/main tip at the FREEZE;
asserted ancestors: 599068f kv-maskless series gate, 3db5cc1 g17, 72a62ea/
892ea7e upstream backports, 1cc5603 g16b route probe). aarch64-only.
v0.7.29 and v0.7.30 were tagged but never published (superseded by this one;
one line each below). Build: lsdc3build VM, /opt/m2-tc glslc 2026.3 /
shaderc 1.4.357. ICD pin proposal for omarchy-pkgs: honeykrisp-omarchy-v3
bbbfa36dce7 (w7J green; the bf16 direct route additionally needs
VK_KHR_shader_bfloat16 = v3 6543eeb7df7 or newer — without it bf16 keeps the
staged kernel).

Fixes

• Batched bf16 coopmat QMM: rows >= 1 were wrong (the X_F32 batch-stride
  defect that shipped in the unreleased v0.7.29/v0.7.30): fixed in
  qmm_coopmat (6c4b9c4) with per-row correctness pins (d3ede9f) and the
  M16/32-row build pins at the review shape (cf41e9b, 20bd644).
• g1 install blocker fixed: the 0.32-only BatchKVCache patchers
  (kv-maskless, kv-host-offset, batch-greedy-head) are gated per mlx-lm
  series (599068f) — pristine mlx-lm 0.31.3 installs exit 0 again
  (g17-patch-series: both lines apply rc=0 + idempotent; the pre-fix script
  reproduced the g1 failure as the negative control, 9b5c938).
• SDPA-VJP dK/dV zeros at rep=1: the dkt/dvt tiles take kL rows
  (61f1de1); post-fix battery 8 suites green on hardware (7f9e0ff,
  b3ad332); residual last-key-head-1 legs stay may_fail.
• Shared-buffer-view allocation guards restored (a092c24).
• ssm-maskless: exact extend mirror + all(p <= 0) guard (c1adc33,
  bc3f3c1).
• mlx-lm backports #1910, #1935 (4f98f45).
• build-wheel: lapacke.h guidance (9e16363); LAPACK_INCLUDE_DIRS for
  Arch ARM openblas layout (aeef13c).

Defaults ON (batched decode stack)

Batched GDN decode: native SDPA decode over the batch + maskless unpadded
BatchKVCache decode (2dcd24b); greedy head pruned per row (6e1e8e1);
host-built offsets + fused RoPE vector offsets (1a80bc7); QMV batch
(d762cb5); prefill full attention via SDPA — the block-causal stopgap
retired (fe452b4; MLX_OMARCHY_SDPA_CAUSAL_BLOCK=1 opts back in).
Kill switches: MLX_OMARCHY_QMV_BATCH=0, MLX_OMARCHY_BATCH_GREEDY=1 (opt-in),
MLX_OMARCHY_KV_HOST_OFFSET=0, MLX_OMARCHY_SDPA_DECODE_BATCH per call,
MLX_OMARCHY_SSM_MASKLESS=0, MLX_OMARCHY_KV_MASKLESS=0.

Matmul direct-route table (G13C + G13G; needs VK_KHR_shader_bfloat16)

H8 rows nn ws8 / nt k4s8 / tn ws8 (3ab07fb); rows that won on both chips
kept (6b9485c); chip- and shape-keyed table + selection test (6fef262);
n >= tile_n/2 (a1a02c7); shipped rows apply only on the measured chips
(a89eaa9). Requires honeykrisp v3 6543eeb7df7+; without it bf16 keeps the
staged kernel. Direct GEMM rows neutral on G14C (7effedf).

Routing

Automatic decision routing is ON by default (4cd3ef7 in the cut;
docs/serve.md). Kill switch MLX_OMARCHY_ROUTING=0.

Batched-prefill known issue from v0.7.29's investigation — CLOSED

Fixed by the batched GDN decode/BatchKVCache work in this release;
confirmed on the fixed wheel (BatchGenerator 4 identical prompts: rows 0-3
identical token ids; the v0.7.29 control reproduces) per the
batched-prefill wrong-rows investigation receipt (private lane).

Assets (draft)

• mlx_omarchy-0.32.4.dev202610071347+9b5c938-cp314-cp314-linux_aarch64.whl
  sha256 a2f8c83e5c5f635d00702565a8557d87c40a9eccac885329dcec4692d85d300d
  (416,666,133 B; built on lsdc3build, /opt/m2-tc glslc 2026.3/shaderc
  1.4.357, whole-encoder bundle pin-verified at build time)
• omarchy-mlx-vendor-wheels-v0.7.31-cp314-aarch64.tar
  sha256 1cc390b1e25c55a80c8c21327a39d7e498cafb5144a75b0ff1fc9e6f8b54b074
  (36 vendored wheels, verify-vendor twice) + .sha256
• SHA256SUMS (3-asset coverage)

Known pending at publish time

• jw16 (M1 Max) g-g13c leg B on the v0.7.31 wheel is pending (supplemental;
  runs after the phone block frees the host). Leg P and the negative control
  are in the receipts (receipts/2026-10-07-release-0.7.31/gates-run/).

mlx-omarchy v0.7.28

mlx-omarchy v0.7.28

Hotfix: batched B>1 requests crashed with the default qwen3 rope-norm fold

v0.7.25, v0.7.26 and v0.7.27 carried a defect in the default-ON qwen3 dense q/k-norm+rope fold (mx.fast.rope_rms_norm). Every request batch with more than one sequence (B>1) crashed:

ValueError: [broadcast_shapes] (B,8,L,64) and (B,1,L,128)

The shared fallback composition of the C++ op read the norm weight (D values) as rotary frequencies (D/2 expected), so every B>1 leg built wrong-size trig tables and failed to broadcast. The B=1 path was not affected: it used the fused GPU kernel and stayed bit-exact.

Who was affected: the oMLX backend (jundot/omlx) with default settings. omlx serves with completion_batch_size = 8 by default, so two or more concurrent HTTP requests to a default python -m omlx.server hit the crash. A bare python -m mlx_lm.server was exposed the same way (batch defaults 32/8). The managed mlx-omarchy-serve mlx-lm backend and the assistant app pin decode and prompt concurrency to 1, so they were not affected. Any batched B>1 loop over qwen3 dense models (evaluation, training-side inference) crashed the same way.

The fix (48ce2b25f): the fallback composition now uses real frequencies when a norm weight is parked in the inputs vector; the fused kernel already read per-request offsets. Full root cause and evidence: receipts/2026-10-05-rope-norm-batch-fix/README.md. The interim v0.7.27 patcher fence that refused every non-integer cache offset is removed; the fold stays default ON; the kill switches stay (MLX_OMARCHY_ROPE_NORM_FUSE=0, and the q/k-norm patcher keeps its own).

Correction to the v0.7.25, v0.7.26 and v0.7.27 notes: the qwen3 dense rope-norm fold in those releases did not serve batched B>1 requests — it crashed them. The v0.7.27 patcher fence silenced the crash by disabling the fold for batched caches, which also gave up the fold's decode win for any serving shape that used array offsets. This release restores the intended behavior. The same correction text is now on the v0.7.25-v0.7.27 release pages.

• Op-level matrix B{1,2,4} x L{1,17} x offset{int 0, int 5, per-request array}, Qwen3-4B shapes, bf16: fused output bit-exact against the eager rms_norm -> rope chain in 18/18 rows, 0 errors (the shipped v0.7.27-era wheel errored the 4 B>1 array-offset rows).
• Behavior tests through the real patched mlx_lm qwen3.py with left-padded batched caches: patched logits bit-equal to the eager chain for B in {1,2,4}; the fold dispatches by default; 5/5 passed (tests/test_rope_norm_batch_behavior.py).
• B=1 decode digest pins unchanged on the release wheel: 2B d64 76ec87cb, 4B d64 e2c919be, 9B d64 26d569c8. Cross-host confirmation in the jwm1 battery (jundot/omlx-style harness on the T8103 host): d64 greedy digests on the release wheel identical to the v0.7.27 baseline wheel in the same window for 2B (eee1cf96…), 4B (42d27a8c…) and 9B (80274aa7…).

jwm1 host coverage (T8103, G13G; jundot/omlx-style harness, decode-only windows): d64 digest A/B 3/3 PASS, g7c packaged ANE worker verify PASS, g7d fresh-image parakeet transcribe PASS on the live ANE (the cache stage ran before g7c per the gate order), capsim 5/5, and the standing M1 battery: 25/26 PASS. The one red is omarchy_fast_ops (two rope-refusal-message CHECKs); the same two CHECKs are red against the v0.7.27 baseline wheel in the same window, so this is not a regression and stays on the published v0.7.28 surface. omarchy_ane_runtime_tests does not link without MLX_OMARCHY_ANE_DEVICE; that gate is omitted here. jw16 is mid-reinstall and is skipped this release. The jw16-only suites ran as M2 substitutes against the release wheel tree on the T6021: omarchy_gdn_maskless_correctness_tests (g13), omarchy_eq_math_tests (g15a, 7/7) and omarchy_trig_reduction_tests (g15b, 4/4) all exit 0. The M2 release battery: gates g1-g14 all RC=0 (14/14).

Changes since v0.7.27

• Rope-norm B>1 fix (48ce2b25f + fence removal): the hotfix above, with a behavior regression test, an op-level check script, and M2/dev-box verification harnesses.
• mlx-lm tool-call argument parsing fix (cf4ef86be, 46584f42e): backport of mlx-lm #1904, shipped on the applied 0.31.3 and 0.32 patcher lines; idempotent patch-series rerun; self-contained regression test (793546208).
• Nine upstream MLX backports (d0cc18bf1, 8aa962f55, 1019a9edf, 2e2643191, 03c607f93, 7a4853c82, a5740771f, 4c2100f1b, b96ba8890, wired backend-generic by 3f144a1b1): view-offset, eval-cleanup deadlock, shared-buffer reshape contiguity, view last-axis stride, view contiguity flags, multioptimizer empty group, as-strided contiguity, aligned array pointer, vmap scatter axis. Triage and build-fix receipt: 96d9f7cce.
• TensorFold port (efced3a29): translator extensions for the Metal-kernel source dialect (casts, subgroup builtins, pointer guard, pragma/device strip), the mx.fast.int8_matmul W8A8 Vulkan op with a composed fallback, lavapipe-verified tests. The custom-kernel translation cache now keys on the translator's own source hash (33ff979ed), so stale cached GLSL from an older binary cannot load.
• qmm prefill: padded shared B-tile stride default ON (d078129f0, twins ab2af3d62, routing fix 4907c5157): measured +2-3% prefill on the M2 Max (G14C) and +0.9% on the G13G host, bit-exact digests; kill switch MLX_OMARCHY_QMM_LDSPAD=0.
• Opt-in per-SoC GPU base-pstate overlays: ship inert; opt in through /etc/omarchy-mac-boot/dtb-overlays.opt-in (paths follow omarchy-mac-pkgs#3). Docs under docs/gpu-base-pstate-*.md.
• oMLX-on-Linux compat layer (017ca25ca, a0fcaff59, 4b7e2ddfc, bc81c2842, 69801282a): oMLX v0.7.0 and the TensorFold mx.metal gates route to the omarchy backend on Linux; the installer runs the platform gate before the compat series. Live M2 smoke receipts: 19952daef, 910a1c975.
• Hardware probe (bce97ef2a, ed3b40b79): mx.device_info() gains memory_size, max_recommended_working_set_size, gpu_cores, marketing_name; set_wired_limit is an honest no-op off-Metal; interior NULs in chip_compatible become spaces.
• Kernel-battery translator fixes (ca195e620, 2f57a1a26, 585ffba29, 732c333a7, 17f2fd605, 0fa118e50, ba88ad971, e2dc22ead, 986f77bd8, 83b45e82d, dcc9f0641, ebe05fc50, 58cd779b3): bf16 handling, const-int wrapping, device-pointer aliasing, rint/roundEven mapping and more, proven on the M2 minimax_m3 battery.
• Receipts and docs: numerics-gate closure for the batched==single digest equality on Qwen3-4B (ad4f5c898), golden-build recipe (9f4f4f747), rope-norm batch-fix evidence, DFlash/oMLX matrix rows, minimax_m3 family plan.

Known issues (unchanged, open)

• SDPA fused-VJP value defects: qL=1 maskless dk/dv are zero; B=1, kL=5 also fails.
• compile() of the sin*cos tape segfaults on the G13 build host; that leg stays out of the test suite.
• Kokoro voice output stays unqualified.

Verification and assets

Assets: mlx_omarchy-0.32.4.dev202610050725+5c15fba-cp314-cp314-linux_aarch64.whl (sha256 68bb536f…, fresh-dir build on the T6021 aarch64 host from a clean checkout of the tagged commit 5c15fbaea), plus omarchy-mlx-vendor-wheels-v0.7.28-cp314-aarch64.tar (sha256 defbc78d…, + .sha256 sidecar) with 36 vendored wheels verified against the lock. v0.7.28 ships NO x86_64 wheel. v0.7.27 carried a cp311 x86_64 wheel; it was built on the x86 dev box, which was off-limits this release, and the emulated x86 build on the M2 was too slow and held the M2 build slot for hours, so it was killed at the publish decision. aarch64 Apple Silicon is the supported platform; the x86_64 wheel was a dev-box artifact. scripts/verify-release-assets.py v0.7.28 --platforms linux_aarch64 printed VERIFIED before publication: the --platforms flag is a one-flag addition to the script recording that this release intentionally ships the aarch64 wheel only. SHA256SUMS covers every uploaded asset. The release battery ran draft-first; the decode number below was measured on the installed release wheel from the release assets: 4-bit Qwen3-4B d64 greedy decode 64.47 tok/s on the T6021 host (fleet bench, 5 prompts, 2 passes, digest 42d27a8c matching the release wheel's A/B digest).

omarchy-mlx v0.7.27

Correction — 2026-10-05: batched B>1 requests crashed the qwen3 rope-norm fold

The default-ON qwen3 dense q/k-norm+rope fold introduced in this release crashed every request batch with more than one sequence (B>1): [broadcast_shapes] (B,8,L,64) and (B,1,L,128). The oMLX backend's default completion_batch_size = 8 exposed it to two or more concurrent HTTP requests; a bare python -m mlx_lm.server (batch defaults 32/8) was exposed the same way. The managed mlx-omarchy-serve backend and the assistant app pin decode and prompt concurrency to 1 and were not affected. The B=1 decode path was unaffected and stayed bit-exact. v0.7.27's patcher fence silenced the crash by disabling the fold for non-integer (batched) cache offsets. Fixed in v0.7.28 (48ce2b25f); root cause and evidence in receipts/2026-10-05-rope-norm-batch-fix/README.md.

---

mlx-omarchy v0.7.27

9B fused GDN decode is bit-exact and default-on

The fused raw decode route is ON by default on every chip. Set MLX_OMARCHY_GDN_RAW_REPEAT=0 to select the composed route. Commit ee83df76c corrected the F32 A_log expression so the candidate fused kernel matches the composed arithmetic.

G13G H270 passed the formal candidate gate. The 2B d64/d128/d256 and pf512 digests matched their pins. For 9B, route ON and RAW_REPEAT=0 digests matched at d64, d256, and pf512 (80274aa790426468, faa02eddee8be9e7, d9892bdf108e46ed). Decode was +34.6% at d64 and +34.3% at d256; pf512 prefill differed by -0.3%. Teacher-forced top-1 matched at 5,120/5,120 positions, free-run identity was 100% over 10×512 tokens, and both routes had S=1 mean NLL 3.4350581017751542 (126 scored tokens).

On M2 G14C, the exact draft wheel matched route ON/OFF digests at d64 (26d569c8), d128 (2bbf33d4), and d512 (0279994b). The composed-order GDU profile dispatched 1,776/3,312/6,384 times at those depths when ON; both fused and composed GDU dispatch counts were zero with the route OFF. At d64, decode measured 39.25 tok/s ON vs 29.32 OFF (+33.9%). The 4B d64 digest stayed e2c919be in both fold modes; one decode step called rope_rms_norm 72 times by default and zero times with MLX_OMARCHY_ROPE_NORM_FUSE=0.

The tiled G14 fresh-install check on the exact draft assets matched composed at 10×512 tokens, with zero free-run divergences, teacher-forced top-1 at 5,120/5,120, and S=1 PPL delta 0.0%. The published v0.7.26 bytes did not reproduce their release note's “bit-exact/order-matched 100% identity” claim: prefix identity was 29.43% over 10×512 tokens, and six first divergences had a 0.125 top-2 gap (one bf16 ULP). The v0.7.26 notes now carry a dated correction; the corrected v0.7.27 result is bit-exact after ee83df76c. See docs/numerics-gate.md and receipts/2026-10-04-v0727-release.md.

Changes since v0.7.26

Commit	Classification
82407d0dc	Added the T6021 v0.7.26 decode/prefill performance ledger (no same-machine macOS comparison): receipts/2026-10-04-m2-perf-ledger/README.md.
38df659	GduBar audited the published v0.7.26 tiled route: its 29.43% prefix identity and 6/10 0.125 gaps are measured facts.
1c6c2a8	DispatchFuse addendum 8: three-arm tiled/perrow/composed experiment and revised free-run interpretation.
c02e86d	H257 forced-legacy owner-bar result: 99.53% teacher-forced agreement, all first-divergence gaps within one ULP, and PPL -0.079%; free-run identity was about 39%.
9d336e325	Release-lane measurement tools: perrow bisect, divergence analysis, per-chip verification, and deploy loop.
aa84d1781	DispatchFuse addendum 6: perrow_pf measurements and divergence narrowed to a 1-ULP state store.
7b6120824	Added the BS_PASSES control to the A/B runner.
f09a0d5e5	Fixed the g7c trace-ABI heredoc to invoke Python rather than tee.
24ccde18c	Added one ctypes trace-ABI mirror and size probe; fixed the GC double-track crash.
003a86c05	Re-landed the wave scheduler with its regression test; default remains OFF.
ece1a1a97	Added the perrow_pf decode path with composed-order walks behind shader flag bit 9.
f601aa7fb	Fixed the G13 legacy-stream device lookup for the raw decode route.
d36d16822	Added the GDN dispatch policy and related measurement tools; its use_fallback chip term is NOT in this release: the GDN fallback contract is the v0.7.26 one on every chip (only the decode tile selection keeps its existing G13 legacy-stream check).
192e3d265	DispatchFuse addendum 5: documented the H257 perrow/NoContraction root cause and partial fix.
fe2b1d883	Added precise decorations to the perrow composed-order walks.
9d3054a43	Cut v0.7.26, including the now-corrected default-ON and bit-exact claims.
159f93f35	Temporarily set the 9B fused route OFF while preserving the =1 opt-in; updated installer, patcher, docs, and receipts. Superseded after the candidate's exactness fix and final gates.
67404dfcd	Reframed 0.125 first-divergence gaps against an absolute 0.05 cutoff; the later scale-aware analysis and exact candidate result supersede that cutoff.
679ef2a08	Preserved the v0.7.26 C++ GDN dispatch policy outside the raw-repeat route.
20201de1e	Silenced transformers' missing-torch advisory in each launcher and added a regression test.
ee83df76c	Matched the candidate F32 A_log expression to the composed arithmetic order; kept default kernels byte-identical outside raw-repeat.
fb42e15e1	Added reference-vs-candidate digest pins for 2B GDN decode arms and 4B.
6edd258f8	Applied the final scale-aware bf16-ULP gate and the candidate release decision for the now-bit-exact 9B route.

Known issues

• SDPA fused-VJP value defects remain open: qL=1 maskless dk/dv are zero; B=1, kL=5 also fails.
• The sin*cos compile() tape segfaults on the G13 build host; that leg remains excluded from the test suite.
• Kokoro voice output remains unqualified.

Release verification

The fresh aarch64 and x86 wheel builds and all draft-release gates passed. Draft assets were verified against their uploaded checksums before publication. The release receipt records the exact asset hashes, gate evidence, and post-publication verification.

mlx-omarchy v0.7.26

Correction — 2026-10-05: batched B>1 requests crashed the qwen3 rope-norm fold

The default-ON qwen3 dense q/k-norm+rope fold introduced in this release crashed every request batch with more than one sequence (B>1): [broadcast_shapes] (B,8,L,64) and (B,1,L,128). The oMLX backend's default completion_batch_size = 8 exposed it to two or more concurrent HTTP requests; a bare python -m mlx_lm.server (batch defaults 32/8) was exposed the same way. The managed mlx-omarchy-serve backend and the assistant app pin decode and prompt concurrency to 1 and were not affected. The B=1 decode path was unaffected and stayed bit-exact. v0.7.27's patcher fence silenced the crash by disabling the fold for non-integer (batched) cache offsets. Fixed in v0.7.28 (48ce2b25f); root cause and evidence in receipts/2026-10-05-rope-norm-batch-fix/README.md.

---

Correction — 2026-10-04: ULP-based audit passes; v0.7.27 default is ON

The published v0.7.26 wheel did not reproduce the earlier “bit-exact/order-matched 100% identity” claim: prefix identity was 29.43% over 10 prompts × 512 tokens. Under the revised scale-aware bar, the measured route passes: per-op fp64 errors were identical; teacher-forced agreement was 99.51% (5,095/5,120); each first-divergence gap was no more than one bf16 ULP (six at 0.125, four at 0); and S=1 PPL changed -0.0721% on 49 scored tokens. At logits in [16,32), one bf16 ULP is 0.125, so the former absolute 0.05 cutoff incorrectly rejected a one-ULP gap.

v0.7.27 defaults the fused 9B route ON on all chips. On v0.7.26, set MLX_OMARCHY_GDN_RAW_REPEAT=0 to select the composed route. This correction supersedes the interim 2026-10-04 correction below, which used the retired 0.05 cutoff.

Correction — 2026-10-04: 9B fused-route free-run claim did not reproduce

The default-ON and “bit-exact/order-matched 100% identity” claim below did not reproduce on the published v0.7.26 wheel. GduBar measured 29.43% prefix identity over 10 prompts × 512 tokens; 6/10 first divergences had a composed top-2 gap of 0.125, above the 0.05 near-tie cutoff. The route fails the free-run acceptance bar despite passing the per-op fp64 and S=1 perplexity bars (-0.0721% on 49 scored tokens). v0.7.27 turns the 9B fused route off by default on every chip. v0.7.26 users can set MLX_OMARCHY_GDN_RAW_REPEAT=0 now to restore the composed path.

mlx-omarchy v0.7.26

Cut commit: 56488ba (main at the V26 GO). Tag v0.7.26, annotated.

Qwen3.5-9B fused GDN raw decode route, DEFAULT ON — decode +31..35% on the 9B (Hk<Hv models)

Qwen3.5-9B stores its GDN A_log in f32 and has fewer key heads than value
heads (16 vs 32), so it previously fell through the fused decode kernel's
Hk == Hv gate into the composed per-token fallback: ~700 extra small
dispatches per decoded token. This release ships two pieces
(receipts 2026-10-04-jw16-decode-dispatchfuse.md, addenda 1-4):

• d86ea8815 — the tiled GDU decode kernel's walks now reproduce the
  composed C++ fallback's arithmetic exactly for A_log-f32 models (shader
  flag bit 9): state gating rounded per element, two-operand products
  serially summed ascending from 0.0, no fma contraction.
• cd4a8c6f9 — scripts/patch-mlx-lm-gdn-raw-repeat.py expands q/k to the
  value-head count so Hk<Hv models take the fused route; now wired into
  scripts/apply-mlx-lm-patches.sh and applied to every fresh install by
  the vendored mlx-lm patch list. DEFAULT ON.

Kill switch: MLX_OMARCHY_GDN_RAW_REPEAT=0 restores the exact pre-land
dispatch. Measured on the same window, jw16:

arm	9B decode d64	9B decode d512	greedy d64 digest
default (route ON)	37.50 tok/s	36.47 tok/s	26d569c8…
kill switch (=0, composed)	28.49 tok/s	—	26d569c8… (bit-exact)

That is +31.6% at d64 (+31..35% across cells in the A/B battery). The
order-matched kernel keeps ~91% of the pre-order-match +34% gain.

Numerics evidence at the land, on captured real operands and the
production harness: teacher-forced top-1 agreement 99.39% over 5120
positions (bar >=99%), all 31 disagreements near-ties <= 1 bf16 ULP
(max gap 1.0 ULP, mean 0.32), PPL delta -0.049% (bar 0.1%), per-op fp64
error bit-identical to the composed path, free-run greedy identity 100%
(5 prompts x 512 tokens). The d64/d128/d256 production digest pins are
unchanged (26d569c8 / 2bbf33d4 / 0279994b); d512 re-pins to
f36dab24 (free-run path).

2B/4B/27B are untouched by construction: bf16-A_log models keep bit 9
unset (byte-identical kernels) and the 27B is full attention (no GDN).
The fresh-install check on the shipped bytes (clean venv from the draft
assets, patchers applied from this tag's installer path) re-proved the
route on the 9B and the v0.7.25 4B rope-norm fold (digests and dispatch
counts in the release receipt).

Opt-in T6021 GPU base-pstate overlay

8c42330cf: gpu-pstate-t6021 joins the per-SoC overlay set (base
pstate 6, not the top state 8), built by packaging/build-dtbo.sh with
docs under docs/gpu-base-pstate-t6021.md. Ships inert unless opted in
via /etc/omarchy-mac-boot/dtb-overlays.opt-in.

Release-gate and receipts (no runtime change)

56488ba21 — release receipt addendum (fresh-install verification and
dispatch counts for this cut).

Kokoro

No streamer change vs v0.7.25. Voice output remains UNQUALIFIED.

Release gates (draft-first, all PASS on the DRAFT bytes)

M2 (T6021, stock boot 96e89c73, throwaway gate HOMEs, GATE_ROOT on
tmpfs): g1 (INSTALL_EXIT 0, SERVE_ENTRY_EXIT 0), g2
(OUTBOUND_DENIED False), g3 (CARD_CHECK PASS), g4
(OUTBOUND_DENIED True), g5 (both Laya manifests pin 55cf4c4e…),
g6 (codec OK, zero skips), g7a (icd_source = packaged), g7b
(SYSINSTALL_EXIT 0), g8 (DEFAULT_ENGINE kokoro-82m-bf16 DEFAULT_VOICE af_heart, rtf 0.910), g9 (SPEAK_QUEUE_SMOKE PASS,
0 overlaps, 0 gaps, 0 busy), g10 (PRIMED true 16.3 s, TTFA 0.879 s),
g11 (CARD_CHECK PASS on the 9B), g12 (GATE12 PASS), g14 (default ON
routing enabled + head ready; MLX_OMARCHY_ROUTING=0 → disabled).
14/14 local gates RC=0.

jw16 (T6001, one gpuwin window per leg, restore health_ok every
window): g7c (status=match, VERIFY_EXIT 0), g7d (golden db501a8c…
match, ane_mode=true, cpu_tensor_events=0, 7/7 pin checks), g13
GDN doctest on the NEW shader bytes (2/2 cases, 152/152 assertions),
g15 trig contract (eq_math + trig_reduction EXIT 0), and the 9B
fresh-install check on the shipped bytes (below).

scripts/verify-release-assets.py v0.7.26 VERIFIED the uploaded draft
before publish.

9B fresh-install check (shipped bytes, jw16 gpuwin window)

Fresh venv from the DRAFT aarch64 wheel; the tag's installer layout
(scripts/ + patches/) byte-compared against raw.githubusercontent at
v0.7.26, then applied: qwen3.py AND gated_delta.py both patched,
installed gated_delta.py carries the default-ON gate. On the 9B:
GDN-FALLBACK native probe 0 lines with the default route vs >0 under
MLX_OMARCHY_GDN_RAW_REPEAT=0; greedy d64 digest 26d569c8… identical
on both arms (the order-matched kernel makes the routes bit-exact;
identical digests ARE the pass signal); same-window decode rates and
the 4B fold re-check (e2c919be… both arms, rope_rms_norm 72 vs 0)
in the release receipt.

Known issues (open, honest)

• SDPA fused-VJP value defects remain OPEN: qL=1 maskless dk/dv zero;
  B=1 kL=5.
• compile() of the sin*cos tape SEGFAULTS on jwm1 — open item; that leg
  is excluded from the test suite.
• Voice output UNQUALIFIED (listening A/B pending; no streamer change vs
  v0.7.25).
• The wave scheduler stays REVERTED/default OFF and is not in this
  release.
• Fully offline fresh installs have no voice output until the first
  online setup (voice extras via install.sh --voice, not the vendor
  tar).

mlx-omarchy v0.7.25

Correction — 2026-10-05: batched B>1 requests crashed the qwen3 rope-norm fold

The default-ON qwen3 dense q/k-norm+rope fold introduced in this release crashed every request batch with more than one sequence (B>1): [broadcast_shapes] (B,8,L,64) and (B,1,L,128). The oMLX backend's default completion_batch_size = 8 exposed it to two or more concurrent HTTP requests; a bare python -m mlx_lm.server (batch defaults 32/8) was exposed the same way. The managed mlx-omarchy-serve backend and the assistant app pin decode and prompt concurrency to 1 and were not affected. The B=1 decode path was unaffected and stayed bit-exact. v0.7.27's patcher fence silenced the crash by disabling the fold for non-integer (batched) cache offsets. Fixed in v0.7.28 (48ce2b25f); root cause and evidence in receipts/2026-10-05-rope-norm-batch-fix/README.md.

---

Correction — 2026-10-04: revised 9B numerics bar

The absolute 0.05 free-run cutoff is replaced by a one-bf16-ULP limit at the composed top-logit magnitude, plus at least 99% teacher-forced top-1 agreement. The published v0.7.26 measurements pass the revised bar: 99.51% teacher-forced agreement, every first divergence within one ULP, per-op fp64 errors identical, and S=1 PPL -0.0721% on 49 scored tokens. The v0.7.25 raw route remains opt-in and default OFF. v0.7.27 defaults the fused route ON; set MLX_OMARCHY_GDN_RAW_REPEAT=0 to select composed behavior.

Earlier correction — 2026-10-04

The GDN raw route in this v0.7.25 release was opt-in and default OFF, as stated below. The later v0.7.26 claim that the 9B route was default ON with 100% free-run identity did not reproduce on the published v0.7.26 wheel; v0.7.27 turns it off by default again. The v0.7.25 default did not change.

mlx-omarchy v0.7.25

Cut commit: 1e15e05 (origin/main at the V25 GO after the wave-sched
encoder revert; supersedes the pre-publish 7b569af and c7ef91b drafts
whose tag was never published). Tag v0.7.25, annotated.

Qwen3 dense rope-norm fold, default ON — decode +4.18..+4.86% on the 4B

scripts/patch-mlx-lm-qwen3-rope-norm.py (receipts
2026-10-04-jw16-decode-dispatchfuse.md): folds the per-head q/k RMSNorm
into mx.fast.rope_rms_norm at the qwen3 dense attention site — the F3
fold ported to the dense file shape. Removes 2 norm dispatches per
attention layer per token (72/tok on the 4B). Kill switch
MLX_OMARCHY_ROPE_NORM_FUSE=0. Bit-exact: 24/24 iso bitcheck legs, and
production greedy digests identical to the serving pins on every A/B arm
at every depth (e2c919be / 1bb7ff66 / c9480d81 / fff6d03b). Applied to
fresh installs by the vendored mlx-lm serve patch list (wired into
scripts/apply-mlx-lm-patches.sh in this cut; the fresh-install
digest check ran on the shipped bytes — see the release receipt).
Paired alternating A/B on the 4B (jw16):

depth	ctl median tok/s	on median tok/s	paired delta
d64	64.22	67.29	+4.71%
d128	63.59	66.69	+4.86%
d256	62.19	65.05	+4.60%
d512	56.47	58.85	+4.18%

All arms disjoint; neutral arm (patch applied, gate off) inert.

qmm prefill rasterization default: group-of-GM=4

4d717225b + b9a759a75 (receipts 2026-10-03-prefill-axes/): the
32-row QmmPrefillCoopmatBF16X32[FullN] prefill route defaults to
group-of-4 rasterization for multi-row-tile grids, so concurrently
resident workgroups share one dequantized weight slab across 4 adjacent
row tiles. MLX_OMARCHY_QMM_NO_RASTER=1 opts back to the shipped
x-major order. Bit-identical to the shipped kernel by construction
(ascending-k f32 accumulation chain untouched). Measured prefill
tok/s vs the shipped default: jwm1 (G13G): 2B pf512 +1.0% / pf1024
+1.3%, 4B pf512 +2.1% / pf1024 +0.8%, 9B pf512 +3.4% / pf1024 +3.2%;
jw16 (G13C): 2B pf512 +1.0% / pf1024 +2.4%, 9B pf512 +4.4% / pf1024
+3.6%. The default never regressed a cell on either chip.

GDN raw-route enablement (gated, default behavior unchanged)

27b6904ed + d547f112b + bc51a7bd4: the raw-gates decode kernel
accepts an F32 A_log (the Qwen3.5-9B conversion stores it F32) via
shader flag bit 9, with in-kernel f32 softplus matching the C++ fallback
reference; scripts/patch-mlx-lm-gdn-raw-repeat.py (env
MLX_OMARCHY_GDN_RAW_REPEAT, default 0) expands q/k to the value-head
count so Hk<Hv models can pass the fused route's Hk==Hv gate. bf16
A_log models take byte-identical original paths (bit 9 unset), so 2B
numerics are untouched by construction. Off by default; the candidate
still owes the numerics gate before any default flip.

Overlay paths move to the omarchy-mac-boot contract

7b569afab (omarchy-mac-pkgs#3): overlay install path is now
/usr/lib/omarchy-mac-boot/dtb-overlays and the opt-in file
/etc/omarchy-mac-boot/dtb-overlays.opt-in. scripts/collect_deep.py
reads the new opt-in file and prints the new steps;
docs/contribute-data.md tester steps updated (including the base-M2
T8112 section).

Changes users must act on: if you had opted overlays in under the
old path, move your file:
/etc/omarchy-platform/dtb-overlays.opt-in →
/etc/omarchy-mac-boot/dtb-overlays.opt-in. The old paths are no longer
read.

Opt-in per-SoC GPU base-pstate overlays

63b0a70e5 (t8103) + 247087be4 (t6001) + packaging/build-dtbo.sh:
optional device-tree overlays (gpu-pstate-t8103, gpu-pstate-t6001)
shipping inert unless the owner opts them in through the opt-in file
above. Measured on the pinned chips: t8103 −9 ms post-pause TTFT with no
measurable energy cost (63b0a70e5 receipts); T6001 decode/prefill NULL
across arms, base pstate 6 removes the 4–5 ms post-pause TTFT penalty
(a59a143ca).

Release-gate fixes and receipts (no runtime change)

g8 installs mlx-audio with --no-deps so it can never replace the
vendored mlx (4964dc9d4); Kokoro objective A/B scripts + receipts
(b8adbc966); zero-CPU trace receipt closing the published v0.7.24
stack (all arms 0 hits, positive control 2, 8e49b4a4e).

Kokoro

No streamer change vs v0.7.24. Voice output remains UNQUALIFIED.

Release gates (draft-first, all PASS on the DRAFT bytes)

M2 (T6021, stock boot 96e89c73, throwaway gate HOMEs): g1 (INSTALL_EXIT 0,
SERVE_ENTRY_EXIT 0, parakeet launcher staged), g2 (OUTBOUND_DENIED False),
g3 (CARD_CHECK PASS), g4 (OUTBOUND_DENIED True), g5 (both Laya
manifests pin 55cf4c4e…), g6 (codec OK, zero skips), g7a
(icd_source = packaged), g7b (SYSINSTALL_EXIT 0), g8
(DEFAULT_ENGINE kokoro-82m-bf16 DEFAULT_VOICE af_heart, rtf 0.895), g9
(SPEAK_QUEUE_SMOKE PASS, 0 overlaps, 0 gaps), g10 (PRIMED true 4.1 s,
TTFA 0.889 s), g11 (CARD_CHECK PASS on the 9B), g12 (GATE12 PASS),
g14 (default ON routing enabled, kill switch disabled; both legs).

jw16 (T6001, one gpuwin window per leg, restore health_ok every window):
g7c (status=match, VERIFY_EXIT 0), g7d (entry + python3 -S exit 0,
golden db501a8c… match, ane_mode=true, cpu_tensor_events=0, 7/7 pin
checks), g13 (2/2 cases, 152/152 assertions), g15 (7/7 + 128; 4/4 + 49),
FOLDCHECK (fresh install shows the fold active: digest pin e2c919be…
on both arms, 72 vs 0 fused calls).

scripts/verify-release-assets.py v0.7.25 VERIFIED the uploaded draft
before publish.

Known issues (open, honest)

• SDPA fused-VJP value defects remain OPEN: qL=1 maskless dk/dv zero;
  B=1 kL=5.
• compile() of the sin*cos tape SEGFAULTS on jwm1 — open item; that leg
  is excluded from the test suite.
• Voice output UNQUALIFIED (listening A/B pending).
• Fully offline fresh installs have no voice output until the first
  online setup (voice extras via install.sh --voice, not the vendor
  tar).

mlx-omarchy v0.7.24

mlx-omarchy v0.7.24

Cut commit: eaa6972 (= origin/main at V24 GO). Tag v0.7.24, annotated.
Supersedes the never-published v0.7.15/16/18/20; v0.7.23 is the previous
published release.

CORRECTION — sin/cos on GPU (affects v0.7.17 through v0.7.23)

v0.7.17–v0.7.23 in-shader sin/cos had wrong reduction constants: silent
wrong values for 1e4<=|x|<1e7 (max abs err 5e-3 at 1e4–1e5, 4.5e-2 at
1e5–1e6, 0.6 at 1e6–1e7). Fixed in v0.7.24: accurate to 5e5, NaN above.
Large-argument trig on GPU was affected. Kokoro voice output on the tested sentences was unchanged by the fix (measured whole-call delta 0.0004 dB MCD, F0 corr 1.000); on Apple M2 (T8112) the affected build refused to render Kokoro. The same
correction line is appended to the v0.7.17–v0.7.23 release notes.

Trig contract (the fix)

de34407c1: one honest sin/cos contract on every path — shared
Cody-Waite with exact k*C1, NaN above 5e5; measured per-device floors
pinned (1e270805b; jwm1's older-Mesa built-in adds up to 6.9e-4 at
reduced arguments), per-device correction documented (441b37aa0),
receipts 3385c14d4 + 76a36ee3f. NEW release gate g15 runs
omarchy_eq_math_tests + omarchy_trig_reduction_tests on the aarch64
build host.

Routing ON by default

4cd3ef727: automatic decision-routing ships ON by default with the
MLX_OMARCHY_ROUTING kill switch (=0 restores the always-main-pair
behavior); receipt eaa697236 (jw16 product smoke PASS). NEW release
gate g14: default-ON leg (a structured-decision turn is answered by the
routed head, head ms recorded, status routing captured) + kill-switch
leg (the main pair answers).

NormApple — Apple BF16 row reductions

ed83d941d (+ shape gating 69ab4cfc3, GDN qk epilogue reduce_size 128
95f611e85, rope overload fix 31a800e8f, receipts a37ecbd0c/
eaf0fb5a4, docs 1de52e70f/6f0bd5bfc).

SealFast

Dual-path sha vectors + tamper tests 21933520d, pipelined seal +
root-gated stamp 379611e09, docs 40b1eb448, jwm1 A/B (performance
neutral) 6cbcf70ad, root-chain stamp A/B -332 ms/process b2d04ef36,
pk-warm instrumentation c59bfc9cd, L3 daemon default-on DESIGN ONLY
d645d9caa (no default change in this cut).

SDPA fused-VJP OOB fix

6ebdb4d53 (tile shapes derived from the score plane rank) + fd sweep
495b0323d + receipt be9c6c592: the composed-VJP SIGABRT documented
in v0.7.21 is fixed at the fused tile shapes.

Collector worker-side (service, not the wheel)

d62d980f4 PII strip instead of refuse; f751646ac incident-hardened
deploys.

Kokoro streamed decoding (shipped as v0.7.23, unchanged here)

No streamer change in this cut. An objective A/B found the streamer
shortens speech ~8% (prosody re-prediction per utterance); WER is
unaffected; qualification is pending a corrected-instrument run. Voice
output stays UNQUALIFIED. Kill switch: MLX_OMARCHY_KOKORO_STREAM=0.

Release gates (draft-first, all PASS on the DRAFT bytes)

g1 (incl. the serve-entry startup check), g2, g3, g4, g5, g6, g7a, g7b,
g8 (Kokoro default smoke — renders with the trig fix), g9, g10 (primer +
TTFA <= 2.0 s), g11, g12 (streamed default, first audio <= 1.5 s), g14
(routing default ON + kill switch), g7c, g7d, g13 (GDN doctest), g15
(trig contract), SDPA vjp fast_ops coverage on the build host.
scripts/verify-release-assets.py v0.7.24 VERIFIED the uploaded draft
before publish.

Known issues (open, honest)

• SDPA fused-VJP value defects remain OPEN: qL=1 maskless dk/dv zero;
  B=1 kL=5. (The OOB/SIGABRT class IS fixed above.)
• compile() of the sin*cos tape SEGFAULTS on jwm1 — open item; that leg
  is excluded from the test suite.
• Voice output UNQUALIFIED (streamed listening A/B pending; corr 0.988
  vs 0.990 floor accepted with exact-input fixtures; streamer shortens
  speech ~8% as above).
• Routing: automatic decision-routing is now default ON; the gate is
  per-pair evidence-gated and the MLX_OMARCHY_ROUTING=0 kill switch
  restores main-pair-only answers.
• Fully offline fresh installs have no voice output until the first
  online setup (voice extras via install.sh --voice, not the vendor
  tar).

mlx-omarchy v0.7.23 (collector hotfix)

Collector hotfix: bare --submit stages a temp archive (regression fix); serial-family dtc properties removed from the redacted archive. Wheel/engine identical to v0.7.22 (no engine commits); only the collector scripts and the build stamp differ. Exact tester command: from a checkout of this tag, python3 scripts/collect_deep.py --ane-smoke --submit

---

Correction (added with v0.7.24): sin/cos on GPU for 1e4<=|x|<1e7 returned silently wrong values in this release (max abs err 5e-3 at 1e4-1e5, 4.5e-2 at 1e5-1e6, 0.6 at 1e6-1e7); fixed in v0.7.24 (accurate to 5e5, NaN above). Large-argument trig on GPU was affected. Kokoro voice output on the tested sentences was unchanged by the fix (measured whole-call delta 0.0004 dB MCD, F0 corr 1.000); on Apple M2 (T8112) the affected build refused to render Kokoro.