Skip to content

v0.28.1

Latest

Choose a tag to compare

@yairpatch yairpatch released this 05 Nov 08:25
· 25 commits to main since this release
3b5dbd7

What's Changed

  • chore(ci): a change to what ships must bump the version by @yairpatch in #65
  • docs(readme): rewrite around what a user does, checked against the code by @yairpatch in #66
  • bench: flyweight 0.2.0 against llama.cpp on the same files and settings by @yairpatch in #67
  • chore(tests): split two semicolon statements the linter rejects by @yairpatch in #69
  • perf(prefill): dense prefill 510 -> 624 tok/s; three launches fewer per token by @yairpatch in #68
  • bench: DeepSeek-V4 speed probe and the KV-cache retrieval eval by @yairpatch in #71
  • perf(prefill): MMQ profiling switches and the Nsight findings by @yairpatch in #72
  • perf(prefill): dense prefill 624 -> 824 tok/s; the int8 MMA shape was the bound by @yairpatch in #73
  • perf(prefill): dense prefill 781 -> 824 tok/s, and no dense spill at 32K f16 KV by @yairpatch in #74
  • fix(gpu): Turing (sm_75) rejected the kernel corpus at load (#70) by @yairpatch in #75
  • perf(prefill): tensor-core attention at every context, 32-token tail tiles, small-batch routing, a GPU timeline by @yairpatch in #76
  • perf(prefill): the host prompt-cache spill moves off the request's critical path by @yairpatch in #77
  • ci: install numpy for the tests, and stop running the matrix twice per pull request by @yairpatch in #79
  • perf(prefill): the tensor-core attention crossover is 512 visible tokens, not 1024 by @yairpatch in #80
  • bench: pass 9 -- the attention crossover, and the two kernels the last 346 ms sits in by @yairpatch in #81
  • perf(prefill): hoist the DeltaNet scan's per-token scalar work into a prologue by @yairpatch in #83
  • fix(ci): the type gate, and the two Windows contract tests by @yairpatch in #84
  • fix(test): the golden packer bytes were keyed to glibc's sinf by @yairpatch in #85
  • fix(cli,bench): the two POSIX-only calls the Windows type gate reached by @yairpatch in #86
  • ci(release): let a manual run publish to PyPI by @yairpatch in #87
  • feat(images): Qwen-Image-2.1 text-to-image and editing on the diffusion tower by @yairpatch in #88
  • feat(images): Qwen-Image-2.1 editing with reference images by @yairpatch in #89
  • fix(images): invoke tower edit with condition images and avoid shadowing by @yairpatch in #90
  • feat(diffusion): implement prefix KV caching for Qwen-Image-2.1 by @yairpatch in #91
  • feat(diffusion): support GGUF DiT checkpoints for Qwen-Image-2.1 by @yairpatch in #92
  • docs, test: document --image-transformer in README and add GGUF edit test by @yairpatch in #93
  • feat(ui): 1-click 'Edit this' button in Studio, documentation, and GGUF edit test by @yairpatch in #94
  • feat(server): scoped edit resolver and automatic transcript edit repair by @yairpatch in #95
  • feat(native): Sylvester-Hadamard outlier suppression on Q8 KV cache and prefill query invariance by @yairpatch in #96
  • feat(kv): asymmetric K8V4 KV cache with Sylvester-Hadamard outlier suppression by @yairpatch in #97
  • fix(cli): restore missing NativeV2InferenceService import in _serve by @yairpatch in #98
  • feat(kv): physical page table indirection (P=64) for paged KV cache allocation by @yairpatch in #99
  • feat(attention): split-KV decode attention for long-context acceleration by @yairpatch in #100
  • fix(ci): robust flyweight_gpu_available probe and skip guards on non-GPU runners by @yairpatch in #101
  • fix(ci): check AVX-VNNI in q4_gemm contract and fix mtime resolution in hf_loader test by @yairpatch in #102
  • feat(bench): quality harness for DeltaNet/attention kernel variants by @yairpatch in #103
  • feat(cpu): direct-from-packed IQ3_S fold for the int16 rows path by @yairpatch in #104
  • feat(cpu): native host twins for attention gate, sigmoid router, delta rows by @yairpatch in #105
  • feat(cpu): direct-from-packed folds for all IQ2/IQ3 gate/up types by @yairpatch in #106
  • Admit IQ1_M CPU experts and IQ2_S/IQ1_M routed MMQ by @yairpatch in #107
  • Remove the unused Hadamard KV cache paths by @yairpatch in #108
  • feat(prefill): route Q8_0 experts through MMQ by @yairpatch in #109
  • Default the IQ gate/up direct fold on by @yairpatch in #110
  • Scale prefill streaming from PCIe bandwidth by @yairpatch in #111
  • Share the hybrid expert step between decode drivers by @yairpatch in #112
  • Drop the expert-history sidecar by @yairpatch in #113
  • Serve DeepSeek-V4 on the GPU: FP4 experts, cache, and attention by @yairpatch in #114
  • Keep dense IQ4_NL and NVFP4 in place, and run Muse attention in f32 by @yairpatch in #115
  • Support Codex tool flows with Muse Glimmer by @yairpatch in #116
  • Optimize sliding-window and turbo prefill by @yairpatch in #117
  • Keep Flash-Next from decoding each cache row once per query head by @yairpatch in #118
  • Add Q2_0 dense matvec and make dense requant opt-in by @yairpatch in #119
  • Speed Flash-Next hybrid expert decode by @yairpatch in #120
  • Gather IQ1_M embeddings packed, and charge snapshots and KV stage in the spill check by @yairpatch in #121
  • Make MTP verify and K-quant MoE experts faster by @yairpatch in #122
  • Add -ngl / --n-gpu-layers (dense and MoE) by @yairpatch in #123
  • Placement report at startup + flyweight plan by @yairpatch in #124
  • Auto expert paging: staged under plain hybrid decode by @yairpatch in #125
  • Placement report, staged auto paging, and -ngl all on MoE uses the expert cache by @yairpatch in #127
  • Make MTP faster than plain decode, including under serve by @yairpatch in #129
  • Run on AMD RDNA GPUs through ROCm (--backend rocm) by @yairpatch in #130
  • Get CI green again: mypy, stale tests, verbose Windows build by @yairpatch in #131
  • Stage mismatched K/V cache types instead of attending row by row by @yairpatch in #132
  • Keep the prompt cache on disk across restarts (--cache-dir) by @yairpatch in #133
  • Pin the first prefill checkpoint at the end of the system prompt by @yairpatch in #134
  • Tensor-core decode attention for turbo caches and 6-way groups by @yairpatch in #135
  • RDNA3 matrix cores for prompt projections: the _wmma twins by @yairpatch in #136
  • Run LFM2 and LFM2-MoE (LFM2.5-24B-A2B) on the native runtime by @yairpatch in #138
  • Direct expert paging when FLYWEIGHT_GPU_MISS puts uploads on the decode path by @yairpatch in #139
  • Run Nemotron 3.5 Bolt (nemotron_h_moe) on the native runtime by @yairpatch in #140
  • Build the Nemotron prefill experts under MSVC; clear the CI lint by @yairpatch in #141
  • Accelerate Nemotron H MoE across prefill and decode with strict parity by @yairpatch in #142
  • Stop PR #142's Nemotron changes from breaking the shared expert paths by @yairpatch in #143
  • Settle Nemotron's int8 decode, test the released formats, close the audit's loose ends by @yairpatch in #144
  • Add Raon-1: correct output, batched prefill, device decode with an expert cache by @yairpatch in #148
  • Speed up qwen4exp decode: parallel router top-k, long-row bf16 inject, overlapped n-gram reads by @yairpatch in #149
  • qwen4exp: 3.5x prompt processing, MTP fixes, opt-in routing approximations, no auto-pinning by @yairpatch in #150
  • Flyweight Desktop: installers, one-click GPU setup and "Get a model" by @yairpatch in #151
  • Desktop: make "Choose file" open a dialog, or say why it cannot by @yairpatch in #152
  • Flyweight Desktop: native chat and every engine option by @yairpatch in #153
  • Desktop: open file dialogs with the system's libraries, not the bundle's by @yairpatch in #154
  • Add flyweight tune and Desktop "Find best settings" by @yairpatch in #155
  • Load split GGUFs whose tensorless first shard ends unpadded by @yairpatch in #156
  • README: lead with the desktop app by @yairpatch in #157

Full Changelog: v0.2.0...v0.28.1