Repository navigation
What's Changed
- chore(ci): a change to what ships must bump the version by @yairpatch in #65
- docs(readme): rewrite around what a user does, checked against the code by @yairpatch in #66
- bench: flyweight 0.2.0 against llama.cpp on the same files and settings by @yairpatch in #67
- chore(tests): split two semicolon statements the linter rejects by @yairpatch in #69
- perf(prefill): dense prefill 510 -> 624 tok/s; three launches fewer per token by @yairpatch in #68
- bench: DeepSeek-V4 speed probe and the KV-cache retrieval eval by @yairpatch in #71
- perf(prefill): MMQ profiling switches and the Nsight findings by @yairpatch in #72
- perf(prefill): dense prefill 624 -> 824 tok/s; the int8 MMA shape was the bound by @yairpatch in #73
- perf(prefill): dense prefill 781 -> 824 tok/s, and no dense spill at 32K f16 KV by @yairpatch in #74
- fix(gpu): Turing (sm_75) rejected the kernel corpus at load (#70) by @yairpatch in #75
- perf(prefill): tensor-core attention at every context, 32-token tail tiles, small-batch routing, a GPU timeline by @yairpatch in #76
- perf(prefill): the host prompt-cache spill moves off the request's critical path by @yairpatch in #77
- ci: install numpy for the tests, and stop running the matrix twice per pull request by @yairpatch in #79
- perf(prefill): the tensor-core attention crossover is 512 visible tokens, not 1024 by @yairpatch in #80
- bench: pass 9 -- the attention crossover, and the two kernels the last 346 ms sits in by @yairpatch in #81
- perf(prefill): hoist the DeltaNet scan's per-token scalar work into a prologue by @yairpatch in #83
- fix(ci): the type gate, and the two Windows contract tests by @yairpatch in #84
- fix(test): the golden packer bytes were keyed to glibc's sinf by @yairpatch in #85
- fix(cli,bench): the two POSIX-only calls the Windows type gate reached by @yairpatch in #86
- ci(release): let a manual run publish to PyPI by @yairpatch in #87
- feat(images): Qwen-Image-2.1 text-to-image and editing on the diffusion tower by @yairpatch in #88
- feat(images): Qwen-Image-2.1 editing with reference images by @yairpatch in #89
- fix(images): invoke tower edit with condition images and avoid shadowing by @yairpatch in #90
- feat(diffusion): implement prefix KV caching for Qwen-Image-2.1 by @yairpatch in #91
- feat(diffusion): support GGUF DiT checkpoints for Qwen-Image-2.1 by @yairpatch in #92
- docs, test: document --image-transformer in README and add GGUF edit test by @yairpatch in #93
- feat(ui): 1-click 'Edit this' button in Studio, documentation, and GGUF edit test by @yairpatch in #94
- feat(server): scoped edit resolver and automatic transcript edit repair by @yairpatch in #95
- feat(native): Sylvester-Hadamard outlier suppression on Q8 KV cache and prefill query invariance by @yairpatch in #96
- feat(kv): asymmetric K8V4 KV cache with Sylvester-Hadamard outlier suppression by @yairpatch in #97
- fix(cli): restore missing NativeV2InferenceService import in _serve by @yairpatch in #98
- feat(kv): physical page table indirection (P=64) for paged KV cache allocation by @yairpatch in #99
- feat(attention): split-KV decode attention for long-context acceleration by @yairpatch in #100
- fix(ci): robust flyweight_gpu_available probe and skip guards on non-GPU runners by @yairpatch in #101
- fix(ci): check AVX-VNNI in q4_gemm contract and fix mtime resolution in hf_loader test by @yairpatch in #102
- feat(bench): quality harness for DeltaNet/attention kernel variants by @yairpatch in #103
- feat(cpu): direct-from-packed IQ3_S fold for the int16 rows path by @yairpatch in #104
- feat(cpu): native host twins for attention gate, sigmoid router, delta rows by @yairpatch in #105
- feat(cpu): direct-from-packed folds for all IQ2/IQ3 gate/up types by @yairpatch in #106
- Admit IQ1_M CPU experts and IQ2_S/IQ1_M routed MMQ by @yairpatch in #107
- Remove the unused Hadamard KV cache paths by @yairpatch in #108
- feat(prefill): route Q8_0 experts through MMQ by @yairpatch in #109
- Default the IQ gate/up direct fold on by @yairpatch in #110
- Scale prefill streaming from PCIe bandwidth by @yairpatch in #111
- Share the hybrid expert step between decode drivers by @yairpatch in #112
- Drop the expert-history sidecar by @yairpatch in #113
- Serve DeepSeek-V4 on the GPU: FP4 experts, cache, and attention by @yairpatch in #114
- Keep dense IQ4_NL and NVFP4 in place, and run Muse attention in f32 by @yairpatch in #115
- Support Codex tool flows with Muse Glimmer by @yairpatch in #116
- Optimize sliding-window and turbo prefill by @yairpatch in #117
- Keep Flash-Next from decoding each cache row once per query head by @yairpatch in #118
- Add Q2_0 dense matvec and make dense requant opt-in by @yairpatch in #119
- Speed Flash-Next hybrid expert decode by @yairpatch in #120
- Gather IQ1_M embeddings packed, and charge snapshots and KV stage in the spill check by @yairpatch in #121
- Make MTP verify and K-quant MoE experts faster by @yairpatch in #122
- Add -ngl / --n-gpu-layers (dense and MoE) by @yairpatch in #123
- Placement report at startup + flyweight plan by @yairpatch in #124
- Auto expert paging: staged under plain hybrid decode by @yairpatch in #125
- Placement report, staged auto paging, and -ngl all on MoE uses the expert cache by @yairpatch in #127
- Make MTP faster than plain decode, including under serve by @yairpatch in #129
- Run on AMD RDNA GPUs through ROCm (--backend rocm) by @yairpatch in #130
- Get CI green again: mypy, stale tests, verbose Windows build by @yairpatch in #131
- Stage mismatched K/V cache types instead of attending row by row by @yairpatch in #132
- Keep the prompt cache on disk across restarts (--cache-dir) by @yairpatch in #133
- Pin the first prefill checkpoint at the end of the system prompt by @yairpatch in #134
- Tensor-core decode attention for turbo caches and 6-way groups by @yairpatch in #135
- RDNA3 matrix cores for prompt projections: the _wmma twins by @yairpatch in #136
- Run LFM2 and LFM2-MoE (LFM2.5-24B-A2B) on the native runtime by @yairpatch in #138
- Direct expert paging when FLYWEIGHT_GPU_MISS puts uploads on the decode path by @yairpatch in #139
- Run Nemotron 3.5 Bolt (nemotron_h_moe) on the native runtime by @yairpatch in #140
- Build the Nemotron prefill experts under MSVC; clear the CI lint by @yairpatch in #141
- Accelerate Nemotron H MoE across prefill and decode with strict parity by @yairpatch in #142
- Stop PR #142's Nemotron changes from breaking the shared expert paths by @yairpatch in #143
- Settle Nemotron's int8 decode, test the released formats, close the audit's loose ends by @yairpatch in #144
- Add Raon-1: correct output, batched prefill, device decode with an expert cache by @yairpatch in #148
- Speed up qwen4exp decode: parallel router top-k, long-row bf16 inject, overlapped n-gram reads by @yairpatch in #149
- qwen4exp: 3.5x prompt processing, MTP fixes, opt-in routing approximations, no auto-pinning by @yairpatch in #150
- Flyweight Desktop: installers, one-click GPU setup and "Get a model" by @yairpatch in #151
- Desktop: make "Choose file" open a dialog, or say why it cannot by @yairpatch in #152
- Flyweight Desktop: native chat and every engine option by @yairpatch in #153
- Desktop: open file dialogs with the system's libraries, not the bundle's by @yairpatch in #154
- Add flyweight tune and Desktop "Find best settings" by @yairpatch in #155
- Load split GGUFs whose tensorless first shard ends unpadded by @yairpatch in #156
- README: lead with the desktop app by @yairpatch in #157
Full Changelog: v0.2.0...v0.28.1