v0.12.0

This is a @karume/runtime performance release, with a @karume/hub cache-eviction fix and two @karume/models fixes folded in. The three packages are versioned in lockstep; the karume wheel (tools/exporter/) moves with them and has no code change this release (it is still not published to PyPI). Nothing breaks: every public API change is additive, and the runtime paths that changed by default are bit-identical to the reference path they replace. The one numeric shift is for stateAttentionReduce: "parallel" users — which Gemma4Pipeline is out of the box — whose prefill now runs the reference-identical path (third Performance item). Distribution artifacts are unchanged — no HuggingFace repository is rebuilt and no pin moves.

The headline is long-context Gemma 4. On a 16K-token context, the new tiled prefill path cuts prefill wall time by about 64%, and the parallel QK reduction cuts decode wall time by a further 9–15% (RTX 3080 Ti, Vulkan). Each figure is the A/B of the single change that introduced it, measured on top of the changes before it — not one cumulative 0.11.0 → 0.12.0 measurement. The hub's evictCachedAssets now actually frees a selection whose sibling quantizations share the same weight files, and every fromPretrained finally passes onRetry through, so an app can show HuggingFace rate-limit waits instead of a frozen progress bar.

⚡ Performance

All figures are from one machine (RTX 3080 Ti, Vulkan, Deno 2.9.6) with the karume-gemma4 distribution; they are ratios to compare, not guarantees. Bit-identical means the output buffer matches the reference ("sequential") path word for word. In every measurement below the gemma4 golden outputs and generated token sequences were unchanged.

• @karume/runtime: the prefill of states-form attention (the generation-context path Gemma 4 uses) now runs on the GEMM tile skeleton. For plans with 16 or more chunk rows, the QK and PV stages read each K / V row once per row tile instead of once per query row. The result is bit-identical to the reference path, and it is selected regardless of stateAttentionReduce. Measured on a 14.7K-token prompt (per-op breakdown, 20 prefill chunks): prefill GPU time 35.2 / 40.9 s → 10.3 / 11.0 s; measured at 16K capacity (wall clock): prefill 53.6 / 56.8 s → 19.5 / 20.6 s (−64%). Decode is unchanged, and the generated token sequence is identical. The tile geometry needs 16,384 bytes of workgroup shared memory — the WebGPU default limit, and the same requirement the fused attention path already had.
• @karume/runtime: stateAttentionReduce: "parallel" now also parallelizes the QK stage for decode. The opt-in seat that already split the PV reduction across 16 lanes ("parallel", ADR 0058 / 0067) now does the same for the QK dot product along the head dimension, for plans with exactly one chunk row (decode). One knob still switches both stages. Measured at a 14.7K context: the QK stage 6.31 / 6.30 → 3.66–4.08 ms per token; decode wall time (median, 16K context) 37.2–39.4 → 33.0–35.4 ms per token (−9–15%); at a 256-token context about −4.5%. As before, "parallel" is not bit-identical to "sequential" (the reduction order changes; results stay deterministic — measured worst case 4.2e-7 against an f64 reference). Gemma4Pipeline keeps its default of "parallel"; the low-level runtime default stays "sequential".
• @karume/runtime: stateAttentionReduce: "parallel" is now applied by plan size, not everywhere. The QK variant is used only for one-row (decode) plans — on prefill plans it measured slower, so those keep the sequential kernel. The PV variant is used only for plans under 16 rows; from 16 rows up the tiled path above takes over. Net effect for an app that opts in: decode still gets the parallel kernels, and prefill numerics are now those of the reference path (bit-identical to "sequential"), where 0.11.0 ran the parallel PV reduction there too. This applies to Gemma4Pipeline out of the box, because its default is "parallel": its prefill intermediate values differ from 0.11.0 (they now equal the reference path's), while its measured golden outputs and token sequences did not change.
• @karume/runtime: 8-bit weights now take the single-row GEMV path. The GEMV family that already served i4 weights for M = 1 now also covers i8 storage with a per-output-channel scale (f32 compute, N and K multiples of 4 — the GEMM vec4 condition, kept as a measured-shapes guard — and K a multiple of 16), which is exactly the shape of Gemma 4's lm_head. Bit-identical to the GEMM path it replaces. Measured on lm_head (262,144 × 1,536): 5.32 → 1.06 ms per call in the single-op bench (×5.0); in a full run (one chat turn, prefill + 29 tokens) its per-token cost went 5.5 / 5.3 → 1.10 ms and decode GPU time per token 22.8 / 21.8 → 17.5 / 17.0 ms (−21–24%).

🐛 Fixes

• @karume/hub: evictCachedAssets no longer lets sibling quantizations protect each other into a no-op. In a distribution manifest, several (model, quant) selections can point at exactly the same weight files (for example f16 and f16-c16, which differ only in session settings). 0.11.0 counted every other selection as a protector, so with any such sibling fully cached the target's files were all reported kept: "shared" and nothing was ever evicted, in any order. Now a selection whose file set is identical to the target's is not counted as a protector by default — at cache granularity the two cannot be told apart, so "delete one but keep the other" was never a real option. Any selection whose file set is not exactly the target's — a strict subset (a smaller model sharing the tokenizer and text encoder) or a superset — still protects exactly as before. The result gains alsoEvicted: the "<model>/<quant>" labels of selections that were fully cached before the call and lost at least one file to it (by default, the identical siblings), so a model manager can retract its "downloaded" mark for them.
• @karume/hub: CacheInventoryOptions.protect lets an app name the protectors itself. Pass a list of { model?, quant? } selections and only those are candidates to protect files (still only if fully cached); the identical-set rule above is then not applied, so an app that wants an identical sibling kept can list it and get kept: "shared" as before. The target itself is ignored if listed; an unknown model or quant throws ManifestReferenceError. listCachedAssets does not read this option.
• @karume/models: every fromPretrained now forwards onRetry. The hub added LoadManifestOptions.onRetry in 0.11.0, but none of the eight pipelines passed it through, so a load that went through fromPretrained sat silent for up to ~31 seconds on a HuggingFace 429 / 503. All eight XFromPretrainedOptions accept onRetry now and it reaches both the manifest fetch and every asset fetch (streamed shards, prefetch, whole-file). The hub-facing knobs are also collected into one exported type, FromPretrainedHubOptions (headers / onProgress / onCacheError / onRetry / fetch / caches), so code that loads several families with the same fetch settings can name that object once. Each family's option type keeps its shape — the shared type is intersected in, not substituted. As on the hub, headers / fetch / caches / onRetry apply to HTTP origins only; a localDirectory / denoDirectory source never calls onRetry.
• @karume/models: BirefnetPipeline.fromPretrained points at BIREFNET_SOURCES when ref is missing. Its error message for a missing repository now recommends BIREFNET_SOURCES["birefnet-hr"] like the other families do, and the JSDoc no longer claims the family has no published distribution (it has had one, with pins, since 0.10.0).

📝 Docs

• docs/limitations.md corrects the largest intermediate buffer of the BiRefNet family: it is the decoder's cat_211 at 320 MiB (1024²) / 1,280 MiB (2048²), not the 256 MiB / 878 MiB previously listed. If you derive a pre-download GPU compatibility threshold from that section, use the new numbers — the old ones let a device with a binding limit between them download gigabytes and then fail at session build. The model-card generator carries the same correction; the published cards of karume-birefnet-hr / karume-lucida will pick it up on their next upload.
• ADR 0067 gains the plan-size geometry table for states-form attention (which kernel serves QK / PV for 1, 2–15, and ≥ 16 rows), ADR 0082 records extending the GEMV family to i8 storage, ADR 0058 records that the stateAttentionReduce seat now has two tenants, and ADR 0094 records the eviction rule change with protect / alsoEvicted. docs/known-issues.md notes that the i8 GEMV and tiled attention paths have not been checked for bit identity on Metal, where the i4 GEMV already shows a 1-ULP difference.
• Five research notes under docs/research/2026-09-06-* hold the measurements behind this release — the per-op baseline across four model families, the two elementwise-fusion spikes that were measured, found under the adoption threshold, and reverted, and the three adopted kernels above.

Full Changelog: v0.11.0...v0.12.0