Skip to content

Releases: townsendmerino/aikit

v1.11.0 — the embedder-coverage release

Choose a tag to compare

@github-actions github-actions released this 23 Jul 03:32

aikit v1.11.0 — the embedder-coverage release

1.10.x was about hardening what already shipped. 1.11.0 is about reach. It takes aikit's set of embedding models certified against their HuggingFace references from two to eight — and, just as importantly, makes "which models are supported" a fact the code proves rather than a line in a README.

The unlock underneath the model count is a pure-Go SentencePiece/Unigram tokenizer (no cgo, no sentencepiece dependency), which opens the entire XLM-RoBERTa multilingual family, plus a mixture-of-experts FFN for the one MoE embedder. Everything is certified the same way: cosine 1.000000 against the reference, a hidden-state gate, and a break-it-first check that must go red when you perturb the pooling, the position offset, or the expert routing.

This is a minor release — no breaking changes. New API and new capability only; every behavior change widens what loads (things that used to fail now succeed).


Certified embedders (2 → 8)

Each row is gated by a parity test at cosine ≥ 0.9999 (all currently at 1.000000):

Model Architecture Pooling Tokenizer Dims Truncatable
all-MiniLM-L6-v2 BERT mean WordPiece 384
CodeRankEmbed nomic-bert (RoPE, SwiGLU) cls WordPiece 768
bge-small-en-v1.5 BERT cls WordPiece 384
nomic-embed-text-v1.5 nomic-bert (RoPE, SwiGLU) mean WordPiece 768 768→64
xlm-roberta-base XLM-R — (bare LM) Unigram 768
multilingual-e5-base XLM-R mean Unigram 768
bge-m3 XLM-R cls Unigram 1024
nomic-embed-text-v2-moe nomic-bert + MoE (top-2/8) mean Unigram 768 768→256

Bold rows are new in this release. The generated docs/embedder-coverage.md is the source of truth.


Highlights

Pure-Go SentencePiece/Unigram tokenizer. A Viterbi-decoded Unigram model plus its normalizer and pre-tokenizer, all in Go — the piece that makes the multilingual family work end-to-end. No cgo boundary, no external tokenizer library.

Mixture-of-experts. Top-2-of-8 routing on alternating layers, certifying nomic-embed-text-v2-moe — the first MoE encoder in aikit.

encoder.MatryoshkaFloor(model) (min int, ok bool) — the one capability a serve layer must not guess. Truncating a Matryoshka-trained embedding to a shorter width is fine; truncating any other model returns a unit-length, entirely plausible vector that simply retrieves worse — a silent failure. This exports the per-model floor so a dimensions request can be refused instead of silently degraded. Only two of the eight certified models qualify, and the claim is measured, not asserted: at a quarter width, multilingual-e5-base drops paraphrase-pair recall 1.00 → 0.80 while genuine MRL models hold their floor.

Coverage you can't fake. docs/embedder-coverage.md is generated from a registry whose pooling/dimension claims are read back from the real checkpoints, behind a freshness gate — so the published table can't drift from the code, and a model can't be listed without a passing gate behind it.

Declared pooling, explicit loader variants. Pooling (CLS vs mean) is now read from 1_Pooling/config.json rather than assumed — the difference is silent and total when wrong. BERT loaders gained the XLM-R position-id offset (pad+1) and optional token_type embeddings.


Upgrading

go get github.com/townsendmerino/aikit@v1.11.0

Drop-in. The changed behaviors only accept more:

  • LoadBERT no longer hard-fails on a tokenizer it can't parse — the model loads forward-only (best-effort) so you can still run it on pre-tokenized ids.
  • embed.LoadTokenizer now accepts Unigram, not only WordPiece.
  • (*encoder.Config).ValidateAssumptions accepts configs it used to reject (the new loader variants made them legitimate).
  • (*embed.Tokenizer).EncodeWithSpecials reads the tokenizer's post-processor template instead of hardcoding [CLS]/[SEP].

If you serve an OpenAI-style dimensions parameter, this is the release to wire encoder.MatryoshkaFloor into your request validation.


Quality

Cut after a full qualification sweep: gofmt / vet / golangci-lint clean; the complete suite and go test -race ./... green; the chunk/treesitter cgo submodule green; a fuzz smoke pass over every untrusted-input parser; and apidiff confirming the Hard-tier 1.0 compatibility guarantee holds (every exported change is additive). CI covers Linux (amd64 + arm64, with the NEON asm and bit-identity gates) and Windows; the sweep additionally ran the full race suite natively on arm64/macOS.

Full changelog: CHANGELOG.md · Compare: v1.10.1...v1.11.0

v1.10.1

Choose a tag to compare

@github-actions github-actions released this 20 Jul 15:14

A follow-up patch to 1.10.0's security review. A second, fix-completeness pass
found that several 1.10.0 fixes had landed at the exact site the first review
named but not at their structurally identical siblings — reintroducing the
classes they were meant to close — plus one genuine regression the 1.10.0
dependency bump carried in silently.

Pin this over 1.10.0. All fixes; no API change (Hard tier unchanged).

Highlights

  • 🩹 Restored a runaway-parse guard that 1.10.0 silently broke. The
    gotreesitter 0.20→0.40 bump changed the parse-timeout contract — 0.20
    returned an error on timeout, 0.40 returns a truncated tree with a nil
    error
    (tree.ParseStoppedEarly() == true). The chunker only checked the
    error, so on a pathological/slow file it stopped falling back to the line
    chunker and instead chunked a partial AST — with the degradation invisible
    (the parseErr stat never incremented). Now detects the early stop and falls
    back, restoring the pre-bump behavior. Regression-tested.
  • 🔒 Finished the 1.10.0 crash-class fixes at their missed sites. Two matmul
    entries (MatmulBTW8A8Batch, MatmulBTAcc64) still panicked uncatchably
    inside worker goroutines on a bad shape — the M2 hazard 1.10.0 closed
    everywhere else; a grep-checklist now confirms all nine fan-out matmuls are
    guarded. And the id = 100 OOB-token fallback (which panics on any vocab ≤
    100, e.g. the repo's own vocab_size:4 fixtures) was still live in the primary
    EncodeBatch path and two others — all five gathers now share one
    clampTokenID helper.
  • 🖼️ Qwen2.5-VL loader brought to the vision H7/H8 bar. Shape-checked
    patch-embed, plus validate() now rejects a head_dim not divisible by 4
    (rotary ÷0), an unsupported hidden_act, and a zero temporal_patch_size.
  • 🧷 Closed the last mmap use-after-free. The bare per-Tensor safetensors
    accessors (Float32s/…/BFloat16sToF32) could be munmap'd mid-read; a new
    Tensor.owner back-pointer + runtime.KeepAlive fixes the whole class. And a
    crafted HNSW blob with mL = +Inf no longer panics on Add-after-load (Load
    recomputes mL from m).

Upgrade notes

  • Patch release — no API change, straight drop-in from 1.10.0.
  • No behavior change on well-formed input; the fixes affect only mismatched /
    crafted / small-vocab / pathological inputs, plus the treesitter fallback which
    now (correctly) triggers on a timed-out parse instead of silently chunking a
    partial tree.

The full categorized changelog is in
CHANGELOG.md → 1.10.1.

v1.10.0

Choose a tag to compare

@github-actions github-actions released this 18 Jul 15:56

A hardening + maintenance release. The centerpiece is a security-focused code
review swept across the untrusted-input boundary
— aikit is the parser layer
the ecosystem routes every hostile GGUF/safetensors byte and every persisted
index through. Alongside the fixes: two additive API surfaces, real allocation
and throughput wins, arm64 now gated in CI, and a toolchain/dependency refresh.

No Hard-tier API breakage. Everything here is additive or behavioral; the
changed kernels stay bit-identical, pinned by the exact-equality parity gates.
Safe drop-in upgrade from 1.9.0.

Highlights

  • 🔒 Untrusted-input boundary hardened. A crafted or merely-mismatched model
    file / persisted index could previously cause a path-traversal read, an
    uncatchable crash (stack exhaustion, worker-goroutine panic), a ~250 GB
    over-allocation, a SIGBUS/checkptr throw from an unaligned unsafe cast, or
    a use-after-munmap segfault. All closed — see Security below.
  • New API (additive). linalg.MatmulBTW4A8Into — a zero-alloc int4 decode
    matmul (0 allocs/op steady-state, the int4 twin of MatmulBTW8A8Into) — and
    Close() on Model / BERT / CrossEncoder / SPLADE for deterministic
    release of mmapped weights instead of waiting on a GC finalizer.
  • Perf. BERT/SPLADE/CrossEncoder attention now runs through the pooled
    scratch arena: 432 → 3 allocs per forward, and a lone forward parallelizes
    instead of running single-threaded. MatmulBTQ8 widens each weight row once
    (was M times) at prefill.
  • 🧪 arm64 in CI. The NEON asm and the bit-identity gates (M-/width-invariance,
    packed-path identity) now run on a real ubuntu-24.04-arm runner — an
    arm64-only regression can no longer ship green. Plus staticcheck via
    golangci-lint (54 findings → 0).

Upgrade notes

  • Version is a MINOR bump for the additive API above. Existing callers need no
    changes.
  • Behavioral hardening you might notice: several parsers/loaders now return a
    clean ErrFormat (or a recoverable, caller-side panic) where a malformed input
    previously slipped through or crashed deep in a kernel. DotI8 now panics on a
    length mismatch (it read out of bounds before). Vision loaders validate tensor
    shapes and config at load. None of this affects well-formed input.
  • chunk/treesitter bumped gotreesitter 0.20.2 → 0.40.0. Its exact chunk
    boundaries are best-effort (ADR-010), so a few may shift on real code; the
    load-bearing contracts (byte-fidelity, AST-meaningful boundaries) are gated and
    hold. The pure-Go core is unaffected — this is isolated to the quarantined
    submodule.
  • Toolchain moved to Go 1.26.5; golang.org/x/text 0.37→0.40, x/sys 0.45→0.47.

The full categorized changelog (Added / Changed / Fixed / Security, with the
per-finding detail) is in
CHANGELOG.md → 1.10.0.

Security (the untrusted-input boundary)

  • safetensors sharded load — path traversal. Shard names from the untrusted
    index JSON were joined to the model dir; "../../../etc/passwd" mmap'd a file
    outside the bundle. Rejected as ErrFormat.
  • safetensors header — no shape × dtype cross-check. A giant shape with a
    tiny byte range parsed, then panicked at inference. Now requires overflow-safe
    ∏shape · dtypeSize == byteRange.
  • safetensors typed accessors — no alignment check. The unsafe reinterpret
    cast assumed alignment the format doesn't guarantee (a -race/checkptr throw,
    SIGBUS on strict-alignment ports). Zero-copy when aligned, byte-copy otherwise.
  • GGUF metadata — unbounded array-of-array recursion. Millions of frames past
    the goroutine-stack limit → unrecoverable abort. Depth capped at 128.
  • HNSW load — missing product allocation guard. The f32 branch clamped
    ndocs/dim individually but not their product (~250 GB attempt). Guarded.
  • mmap lifetime — use-after-unmap. Three zero-copy accessors could be
    munmap'd mid-read → SIGSEGV. runtime.KeepAlive backstops added.
  • Unrecoverable matmul panics. A shape violation above the parallel threshold
    panicked on a worker goroutine (uncatchable). Validation now runs before the
    fan-out; constructors reject negative dims / overflow.

v1.9.0

Choose a tag to compare

@github-actions github-actions released this 17 Jun 22:51

A weight-memory substrate (mmap + madvise + a span-residency cache) lifted into a
new leaf package, plus the payoff it unlocks: an int8 ANN index that can be queried
larger than RAM. The mechanism is the one goinfer proved demand-paging a 35B-A3B
MoE's experts; only the generic, model-free core moves here. All new surface is
Experimental (outside the 1.0 guarantee).

Added

  • mmap — new leaf package: read-only mapping + residency control. The
    read-only MAP_PRIVATE mapping primitive that ann and embed each kept a
    private byte-identical copy of (to avoid an ann→embed edge) now lives once, in a
    zero-dependency leaf both import:

    • MapReadOnly / Unmap — the mapping itself.
    • Advise(span, willNeed)MADV_WILLNEED / MADV_DONTNEED residency hints.
      Firm cap on Linux; on darwin WILLNEED is an advisory prefetch and eviction is
      an OS-discretion no-op; best-effort no-op on the BSDs/Windows.
    • SpanCache[K] — a demand-signal-agnostic LRU of page-aligned spans bounded by a
      byte budget: Add registers a member's spans, Touch faults it in and releases
      the LRU tail to stay under budget. No model logic — the caller owns the demand
      signal. Gated by an eviction unit test and a model-free property test that a
      MADV_DONTNEED-released read-only mapping re-faults byte-identical.
    • PageAlignedInterior, AvailableRAM, AutoBudget helpers.

    Leaf invariant: stdlib-only, except golang.org/x/sys/unix on darwin only
    (the stdlib has no madvise wrapper there) — so it stays invisible to the core
    dependency invariant on Linux, and cgo-free everywhere. !unix keeps the existing
    heap-read fallback.

  • linalg.WeightMat.MappedSpan(base, end). Returns the page-aligned interior of
    a weight's int8/int4 backing bytes iff they alias the given mapping, else nil
    (f32/heap-backed → skip). The bridge from a WeightMat to mmap.SpanCache.

  • ann.LoadFlatI8MmapPaged(path, budget) — query an int8 index larger than RAM.
    The int8 code block is split into blocks that page in and out of the mapping
    through mmap.SpanCache under budget (≤ 0 auto-selects ~half of available RAM);
    cold blocks re-fault from the read-only mapping. Per-block scoring is
    byte-identical to the resident whole-corpus scan (deterministic query quant +
    independent row dots), so paging changes residency only, never results — gated by
    a paged-equals-resident test that also asserts eviction count > 0. The default
    LoadFlatI8Mmap is unchanged (paging is opt-in). FlatI8.PageStats() exposes the
    cache's hit/miss/eviction counts. A paged index serializes concurrent Query
    calls (the pager is stateful) — the cap traded against cross-query parallelism.

Changed

  • ann and embed now call the mmap leaf instead of their local
    mmapReadOnly/munmap copies, which are deleted. Pure refactor — the existing
    OpenSafetensorsMmap / OpenGGUFMmap / LoadFlatI8Mmap behavior and tests are
    unchanged.

v1.8.1

Choose a tag to compare

@github-actions github-actions released this 15 Jun 04:02

Supersedes 1.8.0, which is retracted — it was tagged before the release gate
passed (a missing CHANGELOG compare link) and before the GGUF parser hardening
below. 1.8.1 carries the same Qwen2.5-VL vision tower + arm64 W4A8 work as 1.8.0
plus the fix; pin this instead of 1.8.0.

Fixed

  • GGUF metadata parser: nested-array allocation blowup (embed). A hostile
    array-of-arrays where every level claims a count near the remaining input drove
    make([]any, 0, n) at each nesting level; since the nesting depth is itself
    ~input/12, total preallocation was O(input²) — a ~1 MB file parsed in ~700 ms,
    surfacing as a FuzzParseGGUF "context deadline exceeded" slow path. The eager
    array capacity is now bounded by a small constant (append still grows to the
    true element count), making allocation linear in the input (same ~1 MB file:
    ~700 ms → ~100 ms). Gated by TestParseGGUF_nestedArrayBomb (asserts bounded
    allocation). Parse output unchanged for valid files.

v1.7.3

Choose a tag to compare

@github-actions github-actions released this 13 Jun 15:07

Fixed

  • amd64 AVX2 Dot8x4/Dot4x4 produced wrong results for odd n4 (= kSpan/4;
    e.g. K=13 → n4=3, K=300 → n4=75). The register-blocked kernels (dotFMA4/
    dotFMA8 in dot_amd64.s) consume two 4-element groups per 256-bit YMM
    iteration; a trailing single group (n4 odd ⇒ n%8==4) was accumulated with an
    XMM / VEX.128 FMA, which zero-extends and so wiped the upper 128 bits of
    each YMM accumulator — discarding the loop's lane-4..7 partials. The tail now
    loads the 4 a/b floats into zero-extended YMMs and FMAs in YMM form, preserving
    the upper lanes (the zero upper operand lanes contribute 0). Latent since
    1.6.0
    (the blocked-GEMM hoist): nothing routed odd-n4 shapes through the
    blocked kernel until 1.7.2 removed MatmulBT's small-shape threshold — so
    this is what makes 1.7.2's MatmulBT M-invariance actually correct on amd64.
    amd64+AVX2 only — arm64 NEON and the non-AVX2 scalar fallback were always
    correct (why every prior release and arm64 CI passed). It surfaced through
    MatmulBT and, transitively, the f32 reference of MatmulBTQ4 / MatmulBTW4A8
    (whose quant kernels were never wrong — one root cause). The 1.7.2 threshold
    removal stands (fixed forward, not reverted). Gated by a new direct kernel
    regression test (TestAVX2_blockedKernels_oddN4, odd + even n4 vs scalar) plus
    TestMatmulBT_MConsistent.

v1.7.2

Choose a tag to compare

@github-actions github-actions released this 13 Jun 01:33

Fixed

  • linalg.MatmulBT is now M-invariant — fixes same-model speculative-decoding
    parity (regressed since 1.6.0).
    Each output dst[i,j] is bit-identical
    regardless of M: a row computed alone (M=1) equals the same row computed inside
    a batch (M>1). 1.6.0's blocked-GEMM hoist left a MAC-count threshold in MatmulBT
    that switched a small matmul to a naive dot-per-output span (a different f32
    reduction order than the blocked kernel). So a per-layer projection computed at
    M=1 (single-token decode) and at M=K (batched verify) differed by the f32
    reassociation (~1e-5) — enough to flip an occasional greedy argmax. Downstream
    (goinfer) this broke same-model speculative decoding: the target's batched-verify
    logits no longer matched the draft's one-at-a-time logits, dropping acceptance
    from ~1.0 to 0.893 and diverging from plain greedy. The threshold is removed
    all M route through the one blocked-kernel reduction order, so the per-output
    result no longer depends on M (nor on the parallel fan-out width, which already
    shards 8-aligned). Measured bonus: the blocked kernel is 2–3.8× faster than the
    naive span it replaces at small-M decode/attention shapes, so M-invariance costs
    nothing. Gated by linalg.TestMatmulBT_MConsistent (also pins blockedFill's
    internal paired-vs-odd-row consistency), and the invariant is documented on
    MatmulBT. The quantized kernels (MatmulBTW4A8/Q8/W8A8*) were already
    M-consistent and are untouched. The encoder is unaffected (it routes through
    MatmulBTInto, which was always blocked) — golden parity unchanged.

v1.7.1

Choose a tag to compare

@github-actions github-actions released this 12 Jun 22:51

Added

  • linalg.WrapInt8 / linalg.WrapInt4 — zero-copy constructors for
    already-quantized WeightMat weights (Experimental).
    The inverse of the
    Int8()/Int4() accessors: they wrap caller-owned, pre-quantized slices (int8
    • per-row scales; or packed int4 nibbles + per-group scales + group size)
      without copying or re-quantizing, aliasing the caller's backing arrays the
      same way WrapF32 does. This fills the gap that blocked the goinfer
      decoder.weightMatWeightMat migration: 1.7.0 shipped only quantize-from-f32
      constructors (QuantizeInt8/QuantizeInt4), but the decoder's .giw
      deserialization reads weights already quantized and zero-copy-aliases the int4
      nibbles straight off an mmap'd blob
      — re-quantizing from f32 would have
      regressed both that fast load and its OS-page-cache residency. The wrap path
      preserves both. Shape-validated (panics on a mismatched length, like
      QuantizeRowsInt8). Additive.

v1.7.0

Choose a tag to compare

@github-actions github-actions released this 12 Jun 22:12

Added

  • vision — a pure-Go SigLIP / ViT image encoder (Experimental). aikit gains
    image embeddings: decode (stdlib image/jpeg+png) → preprocess (resize +
    normalize → pixel_values, with a pre-decode pixel-count guard against
    decompression bombs) → a pure-Go transformer forward (bidirectional MHA +
    gelu-tanh MLP, patch-embed conv as im2col+matmul) → last_hidden_state. The
    attention/FFN projections run f32 or int8 W8A8; parity is cosine vs the HF
    SiglipVisionModel golden (scripts/pin_siglip_vision.py) — 1.0 f32,
    ~0.9999 int8
    . No cgo, no new external dependency (it's embed+linalg; the
    image codecs are stdlib). It exposes an import-free GPU-export seam
    (GPUWeights/GPUMat) and a RegisterResident inversion so goinfer's WebGPU
    backend attaches without the core importing it — the same seam pattern as
    encoder.Backend. This makes aikit the only cgo-free image-embedding
    retrieval library
    (image→image similarity and image-as-document indexing work
    day one). Additive — a new leaf package; nothing existing changes.

    The code moves in from goinfer's vision package (same author, MIT), verbatim
    and parity-preserving; goinfer deletes its copy and imports aikit's on the next
    pin bump. The Gemma-specific connector (the vision→LLM-token projector and the
    image-soft-token sentinels) stays in goinfer — aikit ships the encoder, not the
    multimodal glue. Not yet present: a SigLIP text tower, so true text↔image
    retrieval is a documented follow-up (Gemma drives the text side with its LLM,
    which aikit doesn't have); image→image and image-as-document need only the
    encoder shipped here.

  • linalg.WeightMat — a precision-hiding quantized-weight matrix (Experimental).
    One type that holds an f32 / per-row-int8 / group-int4 weight behind a uniform
    MatmulBT(a, dst, M) (+ a Workspace-scoped variant honoring SetThreshold/
    SetWorkers), a Row(i) dequant for embedding lookup, and raw accessors
    (Int8()/Int4()/F32()) for GPU export, serialization, and a consumer's own
    kernel. It consolidates the weight-matrix wrapper that was open-coded three
    times
    — aikit encoder.LayerWeightsQ8, goinfer decoder.weightMat
    (f32/int8/int4-group/W8A8), goinfer vision.qmat (f32/W8A8). It hides storage
    only
    — precision, scales, dispatch; model policy (which table gets which
    precision, int4 group size, GPU-backend routing) stays with the consumer.
    Dispatch reuses the existing linalg kernels — no new asm; outputs are
    bit-identical to each consumer's prior kernel call. Additive.

Changed

  • encoder int8 (Q8) path migrated onto linalg.WeightMat — bit-identical, zero
    output change.
    LayerWeightsQ8 now stores each of the five big projections
    (Wqkv/OutProj/fc11/fc12/fc2) as a weight-only-Q8 linalg.WeightMat instead of an
    open-coded []int8 + []float32 scales pair. LoadWeightsQ8 quantizes via
    linalg.QuantizeInt8, which is bit-identical to the encoder's quantizeRowsInt8
    (same per-row symmetric max/127 round+clamp), and the forward still drives the
    encoder's own baked-scale matmulBTQ8Into over the codes/scales pulled via
    WeightMat.Int8() — the kernel is unchanged (it is numerically distinct from
    linalg.MatmulBTQ8: large-M dequant-once-into-scratch). TestModelQ8_cosineMatchesF32
    holds at cosine 0.997, full Q8 golden/parity suite green, -race clean. The
    removed LayerWeightsQ8 int8/scales fields are Experimental-tier surface. First
    of the three consumer migrations; goinfer's vision.qmat and decoder.weightMat
    migrate against the released minor.

aikit v1.6.0 — the GEMM release: public matmul 7% → 69% of peak

Choose a tag to compare

@github-actions github-actions released this 12 Jun 18:18

A measurement-driven performance release. An external review suggested the
f32 matmul formulation had register-level headroom; the house rule is
measure first, so this cycle started with a peak-fraction gate — and the
gate found more than the review predicted. Three changes later, the public
linalg.MatmulBT went from ~7% to 46–69% of the machine's measured f32
peak
(~6.3× at prefill shapes), with every consumer inheriting it. Plus
one small Hard-tier addition from the goinfer cross-repo review.

The GEMM campaign (arm64; AVX2 ports are hardware-gated follow-ups)

  • Measured the ceiling before touching anything. A register-saturating
    FMA micro-benchmark clocked the M1 Pro P-core at 95.4 GFLOPS f32
    (≈15 FMA/cycle — correcting the 8/cycle back-of-envelope). Against that
    measured ceiling, the encoder's blocked GEMM read 40–49% — below the
    ≤50% gate, so the work was justified by numbers, not vibes.
    (BenchmarkGEMMPeakFraction is now in the tree.)
  • Dot2x8 — a 2×8 register micro-kernel replaces the 1×8 Dot8x4 as
    the blocked GEMM's inner loop: 2 a-rows × 8 b-rows, 16 accumulators held
    across the K loop, every b-load feeding two FMLAs. Same accumulation
    order per dot, so it's bit-identical — golden parity unchanged, no
    tolerance changes. Encoder FC matmuls 1.52–1.58×; end-to-end
    EncodeBatch 1.36×, single-doc encode 1.27×. M=1 decode/gemv
    paths untouched (they're bandwidth-bound; nothing to win).
  • The blocked GEMM is now linalg.MatmulBT itself, not an encoder
    private.
    A cross-repo gate (goinfer prefill) caught the public
    MatmulBT running a naive re-streaming span at ~7% of peak — every
    consumer outside the encoder was paying for the missing cache blocking.
    The encoder's blocked+register-tiled implementation is hoisted into
    linalg as the single shared home. Prefill shapes: 7% → 46%
    (~6.3×); transformer tiles 68–75%. Small matmuls keep the naive span
    via a threshold, so nothing regresses.
  • B-panel packing at large K fixes L1 set-associativity conflicts
    (8 b-rows K·4 bytes apart collide; packing makes them contiguous).
    Bit-identical — same values, same order. Prefill M=512×4096×4096:
    46% → 69% of peak; the encoder's own K=3072 fc2 +15%. Gated to
    K≥2048; the K=768 tiles were already conflict-free.

Also added

  • linalg.MatmulBTInto — the serial blocked GEMM into a caller-owned
    dst, for consumers that manage their own parallelism (Experimental).
  • embed.SafetensorsFile.TensorF32 / TensorI32 — shape-checked typed
    tensor reads with BF16/F16 widening (Hard-tier, additive). Lifts the
    helper that aikit's own loaders and goinfer's decoder/vision loaders had
    hand-written ≥6 times between them; goinfer drops its copies at its next
    pin bump. Direct outcome of the 2026-06-12 cross-repo review.

Numerics & compatibility

  • MatmulBT's results differ from the old naive order by ~1e-5 — f32
    reassociation from the blocked accumulation order, the same accepted
    class as the v1.2.0 ann change. Its documented width-invariance is
    preserved
    (column shards align to the 8-column kernel group;
    TestParallelWidth_bitIdentical). Callers needing exact determinism
    have MatmulBTAcc64, unchanged.
  • Dot2x8 and the B-panel packing are bit-identical to their
    predecessors; encoder golden parity is unchanged across the entire
    campaign.
  • Hard tier: additive only (TensorF32/TensorI32). All new linalg
    surface is Experimental. -race clean; cgo-free; Windows cross-build
    verified.
  • The 2×8 kernel and packing are arm64 NEON; amd64 keeps its AVX2 path —
    the ports are tracked, gated on x86 test hardware (roadmap §2.4).

Full detail: CHANGELOG.md