v0.2.0

Full Changelog: v0.1.0...v0.2.0

v0.1.0

What's Changed

• Reference Metal compute backend by @digitaloten in #1
• fix(vulkan): size flash-attention tile to device shared-memory limit by @junjitree in #2
• fix(vulkan): size register-O flash tile to device shared-memory limit by @junjitree in #3
• perf(metal): profiling-driven Metal backend optimization — decode ~40x, prefill 2.2x by @digitaloten in #4
• perf(metal): native K-quant decode, half-fragment GEMM + flash attention by @digitaloten in #5
• perf(metal): flash attention + mul_mm-shape cooperative GEMM for prefill by @digitaloten in #6
• docs(metal): architecture doc + CONTEXT/README refresh by @digitaloten in #7
• perf(metal): flash_attn_ext ports — long-context prefill 2.5x, decode at depth 1.9x by @digitaloten in #10
• feat(metal): device DeltaNet + Conv1dSilu — qwen3-next linear attention on GPU by @digitaloten in #12
• feat(metal): device MoE path — expert-batched GEMVs off a GPU expert table by @digitaloten in #11
• fix(metal): Op::Linear.w_off support — fused-QKV models crashed the Metal backend by @digitaloten in #13
• perf(metal): hd=256 flash/vec kernels (gemma 3x) + honest bench sessions by @digitaloten in #14
• fix(serve): honor the request's max_tokens by @digitaloten in #19
• perf(metal): native Q5_0 kernels + concurrent decode replay + on-device Copy by @digitaloten in #15
• feat(engine): speculative decoding on the Metal seam — draft-verify, token-identical by @digitaloten in #17
• fix(metal): error instead of silent corruption when the f32 weight cache exceeds the working set by @digitaloten in #22
• test(metal): coupled quant-KV attention parity at depth by @digitaloten in #20
• feat(serve): speculative decoding on Metal serve — one selection funnel for run and serve by @digitaloten in #25
• refactor(metal): move MSL kernels into shaders/*.metal files by @digitaloten in #24
• perf(metal): simdgroup-per-column DeltaNet — qwen35 prefill 0.73x -> 0.80x by @digitaloten in #26
• fix(metal): shaders.rs regressed to the embedded monolith — native-read KV silently disabled by @digitaloten in #27
• test(metal): kernel-name tripwire — missing kernels fail red instead of silently degrading by @digitaloten in #28
• docs(metal): absorb the KV-quant, spec-decode, multi-slot, and DeltaNet arcs by @digitaloten in #29
• feat(spec): conversation slots for speculative serve — 4x on conversation return by @digitaloten in #31
• fix(vulkan): enumerate portability drivers + probe features instead of assuming them by @digitaloten in #32
• fix(diffusion): box DiffusionSess variants — clippy large_enum_variant (macOS-only) by @digitaloten in #37
• feat(mtp): Metal + CPU MTP drivers — correct on both, Metal head-numerics gap flagged by @digitaloten in #36
• fix(metal): invalidate the decode-replay tape on compile — stale replay corrupted repeated forwards by @digitaloten in #38
• docs(metal): MTP, the replay-tape fix, spec/multi-slot, and the MTP perf ceiling by @digitaloten in #40
• fix(cpu): arch-gate the x86-only repack test import by @digitaloten in #41
• feat(cli): measure Metal MTP self-speculation in bench by @digitaloten in #42
• perf(metal): route the wide lm_head off mrv onto cmm at m=2..8 (#35) by @digitaloten in #43
• feat(metal): native IQ2/IQ3 codebook kernels (all five formats) by @digitaloten in #44
• feat(metal): native Q5_K decode kernel (+10-14% on Q5_K_M) by @digitaloten in #45
• perf(vulkan): multi-warp dp4a decode GEMV for wave32 GPUs (opt-in) — 3.4x tg128 on Turing by @junjitree in #48
• test(metal): numeric parity for the GPU-resident decode ops (argmax / sample / embed_gather) by @digitaloten in #46
• perf(metal): replay GPU-resident decode by @digitaloten in #49
• perf(metal): skip inactive MoE tile rows by @digitaloten in #51
• perf(metal): route small IQ4_NL batches to RT by @digitaloten in #50
• perf(metal): accelerate quantized prefill by @digitaloten in #52
• perf(metal): label linear kernel profiles by @digitaloten in #53
• fix(metal): propagate request context by @digitaloten in #55
• perf(metal): unroll cooperative matmul loops by @digitaloten in #54
• perf(metal): run f16 linear natively by @digitaloten in #56
• perf(metal): bind f32 linear weights directly by @digitaloten in #57
• perf(metal): read bf16 linear weights natively by @digitaloten in #58
• perf(metal): tile small-row bf16 linear by @digitaloten in #59
• perf(metal): extend bf16 row tiling by @digitaloten in #60
• perf(metal): add bf16 cooperative matmul by @digitaloten in #61
• perf(metal): add f32 cooperative matmul by @digitaloten in #62
• perf(metal): tile small-row f32 linear by @digitaloten in #63
• perf(metal): lower f32 matmul threshold by @digitaloten in #64
• perf(metal): lower bf16 matmul threshold by @digitaloten in #65
• fix(metal): keep f16 matmul activations exact by @digitaloten in #66
• perf(metal): lower f16 matmul threshold by @digitaloten in #67
• perf(metal): vectorize wide rmsnorm by @digitaloten in #68
• perf(metal): fuse gated rmsnorm by @digitaloten in #69
• perf(metal): hoist deltanet gates by @digitaloten in #71
• perf(metal): hoist deltanet qk norms by @digitaloten in #72
• perf(metal): parallelize batched conv1d by @digitaloten in #78
• perf(metal): widen kquant CMM headers by @digitaloten in #79
• fix(cli): translate Metal compare device by @digitaloten in #83
• fix(metal): correct IQ4 split-K dispatch by @digitaloten in #82
• fix(metal): honor benchmark depth and row count by @digitaloten in #85
• perf(metal): route Q5_K four-row prefill through row tile by @digitaloten in #84
• perf(metal): flash four-row deep prefill by @digitaloten in #86
• perf(metal): native kernels for the remaining quant formats by @mxaddict in #87
• docs: reorganize and consolidate into docs/ with lowercase-kebab names by @mxaddict in #88
• perf(metal): widen four-row flash KV tiles by @digitaloten in #89
• fix(runtime): address correctness review findings by @digitaloten in #90
• Add native Windows build and runtime support by @Headmaster218 in #91

New Contributors

• @digitaloten made their first contribution in #1
• @junjitree made their first contribution in #2
• @mxaddict made their first contribution in #87
• @Headmaster218 made their first contribution in #91

Full Changelog: https://github.com/kryptic-sh/infr/commits/v0.1.0