Repository navigation
Releases
Releases · yairpatch/flyweight
v0.28.1
What's Changed
- chore(ci): a change to what ships must bump the version by @yairpatch in #65
- docs(readme): rewrite around what a user does, checked against the code by @yairpatch in #66
- bench: flyweight 0.2.0 against llama.cpp on the same files and settings by @yairpatch in #67
- chore(tests): split two semicolon statements the linter rejects by @yairpatch in #69
- perf(prefill): dense prefill 510 -> 624 tok/s; three launches fewer per token by @yairpatch in #68
- bench: DeepSeek-V4 speed probe and the KV-cache retrieval eval by @yairpatch in #71
- perf(prefill): MMQ profiling switches and the Nsight findings by @yairpatch in #72
- perf(prefill): dense prefill 624 -> 824 tok/s; the int8 MMA shape was the bound by @yairpatch in #73
- perf(prefill): dense prefill 781 -> 824 tok/s, and no dense spill at 32K f16 KV by @yairpatch in #74
- fix(gpu): Turing (sm_75) rejected the kernel corpus at load (#70) by @yairpatch in #75
- perf(prefill): tensor-core attention at every context, 32-token tail tiles, small-batch routing, a GPU timeline by @yairpatch in #76
- perf(prefill): the host prompt-cache spill moves off the request's critical path by @yairpatch in #77
- ci: install numpy for the tests, and stop running the matrix twice per pull request by @yairpatch in #79
- perf(prefill): the tensor-core attention crossover is 512 visible tokens, not 1024 by @yairpatch in #80
- bench: pass 9 -- the attention crossover, and the two kernels the last 346 ms sits in by @yairpatch in #81
- perf(prefill): hoist the DeltaNet scan's per-token scalar work into a prologue by @yairpatch in #83
- fix(ci): the type gate, and the two Windows contract tests by @yairpatch in #84
- fix(test): the golden packer bytes were keyed to glibc's sinf by @yairpatch in #85
- fix(cli,bench): the two POSIX-only calls the Windows type gate reached by @yairpatch in #86
- ci(release): let a manual run publish to PyPI by @yairpatch in #87
- feat(images): Qwen-Image-2.1 text-to-image and editing on the diffusion tower by @yairpatch in #88
- feat(images): Qwen-Image-2.1 editing with reference images by @yairpatch in #89
- fix(images): invoke tower edit with condition images and avoid shadowing by @yairpatch in #90
- feat(diffusion): implement prefix KV caching for Qwen-Image-2.1 by @yairpatch in #91
- feat(diffusion): support GGUF DiT checkpoints for Qwen-Image-2.1 by @yairpatch in #92
- docs, test: document --image-transformer in README and add GGUF edit test by @yairpatch in #93
- feat(ui): 1-click 'Edit this' button in Studio, documentation, and GGUF edit test by @yairpatch in #94
- feat(server): scoped edit resolver and automatic transcript edit repair by @yairpatch in #95
- feat(native): Sylvester-Hadamard outlier suppression on Q8 KV cache and prefill query invariance by @yairpatch in #96
- feat(kv): asymmetric K8V4 KV cache with Sylvester-Hadamard outlier suppression by @yairpatch in #97
- fix(cli): restore missing NativeV2InferenceService import in _serve by @yairpatch in #98
- feat(kv): physical page table indirection (P=64) for paged KV cache allocation by @yairpatch in #99
- feat(attention): split-KV decode attention for long-context acceleration by @yairpatch in #100
- fix(ci): robust flyweight_gpu_available probe and skip guards on non-GPU runners by @yairpatch in #101
- fix(ci): check AVX-VNNI in q4_gemm contract and fix mtime resolution in hf_loader test by @yairpatch in #102
- feat(bench): quality harness for DeltaNet/attention kernel variants by @yairpatch in #103
- feat(cpu): direct-from-packed IQ3_S fold for the int16 rows path by @yairpatch in #104
- feat(cpu): native host twins for attention gate, sigmoid router, delta rows by @yairpatch in #105
- feat(cpu): direct-from-packed folds for all IQ2/IQ3 gate/up types by @yairpatch in #106
- Admit IQ1_M CPU experts and IQ2_S/IQ1_M routed MMQ by @yairpatch in #107
- Remove the unused Hadamard KV cache paths by @yairpatch in #108
- feat(prefill): route Q8_0 experts through MMQ by @yairpatch in #109
- Default the IQ gate/up direct fold on by @yairpatch in #110
- Scale prefill streaming from PCIe bandwidth by @yairpatch in #111
- Share the hybrid expert step between decode drivers by @yairpatch in #112
- Drop the expert-history sidecar by @yairpatch in #113
- Serve DeepSeek-V4 on the GPU: FP4 experts, cache, and attention by @yairpatch in #114
- Keep dense IQ4_NL and NVFP4 in place, and run Muse attention in f32 by @yairpatch in #115
- Support Codex tool flows with Muse Glimmer by @yairpatch in #116
- Optimize sliding-window and turbo prefill by @yairpatch in #117
- Keep Flash-Next from decoding each cache row once per query head by @yairpatch in #118
- Add Q2_0 dense matvec and make dense requant opt-in by @yairpatch in #119
- Speed Flash-Next hybrid expert decode by @yairpatch in #120
- Gather IQ1_M embeddings packed, and charge snapshots and KV stage in the spill check by @yairpatch in #121
- Make MTP verify and K-quant MoE experts faster by @yairpatch in #122
- Add -ngl / --n-gpu-layers (dense and MoE) by @yairpatch in #123
- Placement report at startup + flyweight plan by @yairpatch in #124
- Auto expert paging: staged under plain hybrid decode by @yairpatch in #125
- Placement report, staged auto paging, and -ngl all on MoE uses the expert cache by @yairpatch in #127
- Make MTP faster than plain decode, including under serve by @yairpatch in #129
- Run on AMD RDNA GPUs through ROCm (--backend rocm) by @yairpatch in #130
- Get CI green again: mypy, stale tests, verbose Windows build by @yairpatch in #131
- Stage mismatched K/V cache types instead of attending row by row by @yairpatch in #132
- Keep the prompt cache on disk across restarts (--cache-dir) by @yairpatch in #133
- Pin the first prefill checkpoint at the end of the system prompt by @yairpatch in #134
- Tensor-core decode attention for turbo caches and 6-way groups by @yairpatch in #135
- RDNA3 matrix cores for prompt projections: the _wmma twins by @yairpatch in #136
- Run LFM2 and LFM2-MoE (LFM2.5-24B-A2B) on the native runtime by @yairpatch in #138
- Direct expert paging when FLYWEIGHT_GPU_MISS puts uploads on the decode path by @yairpatch in #139
- Run Nemotron 3.5 Bolt (nemotron_h_moe) on the native runtime by @yairpatch in #140
- Build the Nemotron prefill experts under MSVC; clear the CI lint by @yairpatch in #141
- Accelerate Nemotron H MoE across prefill and decode with strict parity by @yairpatch in #142
- Stop PR #142's Nemotron changes from breaking the shared expert paths by @yairpatch in #143
- Settle Nemotron's int8 decode, test the released formats, close the audit's loose ends by @yairpatch in #144
- Add Raon-1: correct output, batched prefill, device decode with an expert cache by @yairpatch in https://gi...
flyweight 0.2.0
What's Changed
- V2 native runtime by @yairpatch in #1
- feat(runtime): add K2-Horizon architecture support by @yairpatch in #2
- feat(server,web): support K2-Horizon thinking tags in reasoning strea… by @yairpatch in #3
- chore(repo): untrack the profiler dumps and captured stderr by @yairpatch in #4
- test(native): parity cases for the three K2-Horizon host kernels by @yairpatch in #7
- docs(repo): contribution scaffolding for outside contributors by @yairpatch in #5
- feat(server): support K2-Horizon Q8_0 quant and <|ifm|im_end|> EOS te… by @yairpatch in #6
- feat(runtime): K2-Horizon MoVA 36B-A4B -- MoE blocks, softplus gate, … by @yairpatch in #8
- feat(runtime): value-expert prefetch and direct paging for K2-Horizon… by @yairpatch in #9
- perf(runtime): K2-Horizon attention takes the tensor-core and fused decode paths by @yairpatch in #10
- perf(attention): tensor-core GQA decode kernel for 128-dim heads in 4-way groups by @yairpatch in #11
- Fix/k2 kernel parity cases by @yairpatch in #12
- fix(prefill): bound the tensor-core score block by heads, not by the GQA group by @yairpatch in #13
- ci: release wheel workflow for PyPI publishing by @yairpatch in #14
- ci: move release workflow into place by @yairpatch in #15
- Revert "ci: move release workflow into place" by @yairpatch in #16
- feat(server): K2-Horizon native tool calls by @yairpatch in #17
- ci: move the release workflow into .github/workflows by @yairpatch in #18
- ci(release): create the GitHub release page with the wheels attached by @yairpatch in #19
- build: publish the distribution as flyweight-llm by @yairpatch in #20
- fix(runtime): honor a --context-window the caller typed instead of shrinking it by @yairpatch in #21
- fix(runtime): rewind attention-only KV to a shared prefix on every such architecture by @yairpatch in #22
- fix(server): close a forced thinking block with the tag that opened it by @yairpatch in #23
- test: skip the four slow parity tests by default, and run the suite in parallel by @yairpatch in #24
- feat(server): a request log with structure, colour and an off switch by @yairpatch in #25
- fix(server): keep the column grid when a line is not an ordinary request by @yairpatch in #26
- Feat/server log by @yairpatch in #27
- fix(types): clean mypy under the version CI installs by @yairpatch in #28
- fix(server): honor a request's max_tokens and scope the thinking cap to /v1/messages by @yairpatch in #29
- Fix/ada kernel load by @yairpatch in #30
- fix(server): a client that goes away on Windows is not a 500 by @yairpatch in #31
- fix(sampler): cap temperature inside a tool call, for every client by @yairpatch in #32
- fix(tokenizer): Qwen checkpoints get the pre-tokenizer their GGUF asks for by @yairpatch in #33
- fix(tokenizer): K2-Horizon and Bailing GGUFs get their own splitters too by @yairpatch in #34
- perf(cpu): whole-block VNNI dot for IQ1_S experts by @yairpatch in #35
- fix(dense-spill): one OpenMP team per spilled block, Q3_K default recorded, Gemma 4 excluded by @yairpatch in #36
- perf(qwen4exp): capture the routed delta layer front as one graph; bl… by @yairpatch in #37
- perf(cpu): IQ3_S experts on the whole-block VNNI dot by @yairpatch in #38
- perf(prefill): open the expert stream on qwen4exp -- pinned mirror, shared staging, int8 IQ1_S rows, routed defaults; fix block_reduce_sum race by @yairpatch in #39
- perf(prefill): step the auto stream arena down before disabling it by @yairpatch in #40
- fix: the runtime loads and compiles on machines CI never sees (no-AVX-512 CPUs, older drivers, older toolkits) by @yairpatch in #41
- fix(gpu): compile kernels to a device cubin, not PTX, so a driver older than the toolkit still loads them by @yairpatch in #42
- fix(gpu): target the newest architecture NVRTC knows when it predates the GPU by @yairpatch in #43
- perf(decode): prompt-lookup self-speculation for qwen4exp; fix MTP replay rollback discarding sampler choices by @yairpatch in #44
- fix(leaks): close four small release gaps found by the memory audit by @yairpatch in #45
- Fix/load time avx512 by @yairpatch in #47
- chore(repo): move bench and check scripts out of the repository root by @yairpatch in #48
- Show prefill progress in the chat UI instead of a blinking cursor by @yairpatch in #49
- fix(vision): reserve the tower's workspace at load, not on the first image by @yairpatch in #50
- Reserve the vision workspace at load, and document Flash-Next vision by @yairpatch in #51
- fix(vision): the tower check called a property by @yairpatch in #52
- fix(server): an effort level the checkpoint never named failed the render by @yairpatch in #53
- fix(web): the effort picker offered levels the checkpoint does not have by @yairpatch in #54
- fix(server): preserve_thinking false took the reasoning the model sti… by @yairpatch in #55
- fix(server): the off switch was spelled in a field nobody read by @yairpatch in #56
- perf(qwen): IQ3_S dense projections decoded a weight at a time by @yairpatch in #57
- feat(qwen): Qwen3.8-Flash-Next packed as Q2_K had no kernels to run on by @yairpatch in #58
- feat(diffusion): Z-Image-Turbo images and MiniMax-H3 video on the native engine by @yairpatch in #59
- fix(qwen4exp): QSA selection on turbo KV caches by @yairpatch in #60
- feat(qwen4exp): run the GSQ-RCO checkpoints; exact int16 prefill GEMM by @yairpatch in #61
- perf(cpu): hybrid Intel parts ran IQ1_S experts on the float path and decode on E-cores by @yairpatch in #62
- docs(plans): status line under every plan title, real names for three by @yairpatch in #63
- fix(ci): workspace contract asserts the old delta floor; MSVC has no sincosf by @yairpatch in #64
New Contributors
- @yairpatch made their first contribution in #1
Full Changelog: https://github.com/yairpatch/flyweight/commits/v0.2.0