aikit v1.6.0 — the GEMM release: public matmul 7% → 69% of peak
A measurement-driven performance release. An external review suggested the
f32 matmul formulation had register-level headroom; the house rule is
measure first, so this cycle started with a peak-fraction gate — and the
gate found more than the review predicted. Three changes later, the public
linalg.MatmulBT went from ~7% to 46–69% of the machine's measured f32
peak (~6.3× at prefill shapes), with every consumer inheriting it. Plus
one small Hard-tier addition from the goinfer cross-repo review.
The GEMM campaign (arm64; AVX2 ports are hardware-gated follow-ups)
- Measured the ceiling before touching anything. A register-saturating
FMA micro-benchmark clocked the M1 Pro P-core at 95.4 GFLOPS f32
(≈15 FMA/cycle — correcting the 8/cycle back-of-envelope). Against that
measured ceiling, the encoder's blocked GEMM read 40–49% — below the
≤50% gate, so the work was justified by numbers, not vibes.
(BenchmarkGEMMPeakFractionis now in the tree.) Dot2x8— a 2×8 register micro-kernel replaces the 1×8Dot8x4as
the blocked GEMM's inner loop: 2 a-rows × 8 b-rows, 16 accumulators held
across the K loop, every b-load feeding two FMLAs. Same accumulation
order per dot, so it's bit-identical — golden parity unchanged, no
tolerance changes. Encoder FC matmuls 1.52–1.58×; end-to-end
EncodeBatch1.36×, single-doc encode 1.27×. M=1 decode/gemv
paths untouched (they're bandwidth-bound; nothing to win).- The blocked GEMM is now
linalg.MatmulBTitself, not an encoder
private. A cross-repo gate (goinfer prefill) caught the public
MatmulBTrunning a naive re-streaming span at ~7% of peak — every
consumer outside the encoder was paying for the missing cache blocking.
The encoder's blocked+register-tiled implementation is hoisted into
linalgas the single shared home. Prefill shapes: 7% → 46%
(~6.3×); transformer tiles 68–75%. Small matmuls keep the naive span
via a threshold, so nothing regresses. - B-panel packing at large K fixes L1 set-associativity conflicts
(8 b-rows K·4 bytes apart collide; packing makes them contiguous).
Bit-identical — same values, same order. Prefill M=512×4096×4096:
46% → 69% of peak; the encoder's own K=3072 fc2 +15%. Gated to
K≥2048; the K=768 tiles were already conflict-free.
Also added
linalg.MatmulBTInto— the serial blocked GEMM into a caller-owned
dst, for consumers that manage their own parallelism (Experimental).embed.SafetensorsFile.TensorF32/TensorI32— shape-checked typed
tensor reads with BF16/F16 widening (Hard-tier, additive). Lifts the
helper that aikit's own loaders and goinfer's decoder/vision loaders had
hand-written ≥6 times between them; goinfer drops its copies at its next
pin bump. Direct outcome of the 2026-06-12 cross-repo review.
Numerics & compatibility
MatmulBT's results differ from the old naive order by ~1e-5 — f32
reassociation from the blocked accumulation order, the same accepted
class as the v1.2.0 ann change. Its documented width-invariance is
preserved (column shards align to the 8-column kernel group;
TestParallelWidth_bitIdentical). Callers needing exact determinism
haveMatmulBTAcc64, unchanged.Dot2x8and the B-panel packing are bit-identical to their
predecessors; encoder golden parity is unchanged across the entire
campaign.- Hard tier: additive only (
TensorF32/TensorI32). All new linalg
surface is Experimental.-raceclean; cgo-free; Windows cross-build
verified. - The 2×8 kernel and packing are arm64 NEON; amd64 keeps its AVX2 path —
the ports are tracked, gated on x86 test hardware (roadmap §2.4).
Full detail: CHANGELOG.md