v0.1.0

What's Changed

• Multi-host disk prefix caching by @original-el8 in #29
• Add requested generation logprob token IDs by @yatesdr in #35
• Add System One scoring through requested logprob IDs by @yatesdr in #36
• Enable ARM SHA-256 acceleration for package hashing by @eugr in #41
• docs: describe the existing compact Qwen GDN recovery path by @Maxrio9 in #15
• Preserve JSON number literals in ChatML tool arguments by @mark-yong in #31
• Repair prefix cache retention by @yatesdr in #38
• Serve and prepare over hosts cabled as a chain by @FujitsuPolycom in #40
• Add Decisions adapter over System One scoring by @yatesdr in #37
• Add Gemma 4 text recipe with Assistant MTP by @SeraphimSerapis in #22
• Include chat-template assets in the Debian build context by @original-el8 in #58
• Support numeric reasoning effort for DeepSeek V4.1 by @voipmonitor in #50
• Raise the rank export quiet-period bound above a payload hash by @original-el8 in #60
• Load Gemma 4 chat templates in Helios by @mark-yong in #56
• Add Messages cache checkpoint hints by @yatesdr in #28
• Parse Gemma 4 reasoning channels and tool calls by @mark-yong in #62
• Prepare NVFP4 draft and Markov heads behind independent flags by @original-el8 in #65
• Use NVFP4 draft vocabulary shards for Qwen3.8 tensor parallelism by @eugr in #66
• Install the cuTile compiler toolchain for model preparation by @eugr in #68
• Keep b12x progress live across tuning jobs by @eugr in #70
• Preserve native CUDA toolkit selection with cuTile dependencies by @eugr in #71
• Keep prose tool-tag mentions in function-tag and GLM output as text by @mark-yong in #67
• Check sectioned unverified-payload inspection output by @voipmonitor in #89
• Pin FlashInfer to the supported tuning-tier API by @voipmonitor in #85
• Move the host prefix-cache tiers into a CUDA-free selene crate by @yatesdr in #80
• Fuse DSpark draft scoring and sparse support restoration by @voipmonitor in #95
• Overlap DeepSeek V4.1 query expansion with cache preparation by @voipmonitor in #97
• Prepare an opt-in MXFP8 DeepSeek V4.1 draft vocabulary head by @voipmonitor in #98
• Return undeclared MiniMax tool calls under automatic tool choice by @mark-yong in #49
• Carry the checkpoint license across package promotion by @mark-yong in #83
• Count single-token reasoning delimiters only when generated as that token (#48) by @mark-yong in #94
• Preserve programmatic CUDA graph dependencies in native package loading by @voipmonitor in #104
• Flatten the selene crate to the top level by @yatesdr in #101
• Broadcast speculative proposals by block and fuse BF16 Markov sampling by @voipmonitor in #107
• Prepare DeepSeek V4.1 MXFP8 attention with public b12x plans by @voipmonitor in #109
• Tune DeepSeek attention projections together in complete layers by @voipmonitor in #110
• Serve directory-form packages through Orbit by @original-el8 in #114
• Apply bundled Inductor patches before preparation by @eugr in #117
• Include FlashInfer CUDA sources in preparation compatibility checks by @voipmonitor in #120
• Allow FP8 weights for the DeepSeek V4.1 draft transition head by @voipmonitor in #121
• Share prefix KV cache between instances and hosts through a selene pool manager by @yatesdr in #92
• Keep vocabulary probabilities local during DeepSeek speculative sampling by @voipmonitor in #135
• Append DeepSeek V4.1 window snapshots during cache writes by @voipmonitor in #108
• Keep self-reading complete copies in program order by @original-el8 in #158
• Drop reasoning close delimiter tokens from visible content by @brandonmmusic-max in #162
• Register the DeepSeek control tokens the frontend tests use by @yatesdr in #166
• Selene: let a slow transfer-rate estimate recover without new samples by @yatesdr in #155
• Keep Selene's shared regions across a manager restart by @yatesdr in #138
• Selene manager: store work proportional to changes, not record count by @yatesdr in #139
• Prefix cache pools for multi-host tensor parallelism; opt-in RoCE on x86_64 by @yatesdr in #140
• Selene host-tier eviction: unified host memory, local spill on pressure, optional max age by @yatesdr in #141
• Add a hard reasoning-token budget and DeepSeek V4.1 sampling ranges by @yatesdr in #167
• Build and test Selene and Helios on Linux and Windows again by @yatesdr in #171
• Selene pool: hash records once, direct-I/O row reads, prompt window credits by @yatesdr in #142
• Selene: store only what a request may resume, once, and use VRAM for snapshots by @yatesdr in #168
• Open, sync, lock, and reclaim prepared-package files on Windows by @original-el8 in #173
• Report reasoning token counts in Responses and Chat usage by @mark-yong in #170
• Windows: prepare, export and serve prepared packages on RTX Spark N1X by @R4yHarris in #159
• Keep request and tool-call IDs unique across server restarts by @yatesdr in #176
• Remove repeated work from scheduler planning by @voipmonitor in #188
• Reuse retained warmup mappings across compiled warmup by @brandonmmusic-max in #154
• Prepare optional clustered PCIe collectives by @voipmonitor in #174
• Ignore unreferenced Inductor kernels in static launcher bundles by @mark-yong in #185
• Share startup race storage and reuse resident weights for L2 eviction by @voipmonitor in #197
• Keep streamed responses alive when clients pause reading by @voipmonitor in #207
• Fix complete-copy ordering, CuTe tensor pickling and GB300 package naming by @original-el8 in #129
• Hash signed payloads once and hard-link unchanged wrappers by @original-el8 in #209
• Fix DeepSeek V4.1 top-p at 1 and require unfiltered vocabulary sampling by @original-el8 in #208
• fix: preserve both private draft pages in shifted prefix checkpoints by @Maxrio9 in #165
• Bound Windows integrated-GPU cache sizing by available commit by @original-el8 in #175
• Fix nondeterministic SM103 mHC prefill projection by @original-el8 in #131
• Consult ignore_eos when classifying a finish reason at the output limit by @mark-yong in #102
• Stop making reviewers excavate your chat history by @original-el8 in #214
• Use KDA-only MXFP8 and reduce Kimi K3.1 TP16 communication by @voipmonitor in #234
• Match native RoCE transport to the pinned FlashInfer proxy ABI 11 by @original-el8 in #251
• Fix RTX Spark revision detection and failed download cleanup by @eugr in #260
• Improve WinUI model metadata and GPU discovery filtering by @eugr in #261
• Add exact counted Kimi owner-candidate transport by @voipmonitor in #257
• Copy the native CLI crate into the Debian build context by @yatesdr in #252
• Allow XTML message closure after structured answers by @voipmonitor in #228
• Retain GLM tool arguments that do not match their declared type by @mark-yong in #179
• Declare DeepSeek V4's layer and DSpark weight containers by their own fields by @kyonjova in #224
• Bind one DFlash2 draft embedding layout on every Qwen Flash rank by @original-el8 in #250
• Reuse dead GLM prefill storage to enlarge the KV cache by @yatesdr in #253
• Use two row blocks for eight-row Kimi paired gathers by @voipmonitor in #265
• Fix TP16 host prefix records and retain deep DRAM prefixes by @voipmonitor in #264

New Contributors

• @original-el8 made their first contribution in #29
• @yatesdr made their first contribution in #35
• @eugr made their first contribution in #41
• @Maxrio9 made their first contribution in #15
• @mark-yong made their first contribution in #31
• @FujitsuPolycom made their first contribution in #40
• @SeraphimSerapis made their first contribution in #22
• @brandonmmusic-max made their first contribution in #162
• @R4yHarris made their first contribution in #159
• @kyonjova made their first contribution in #224

Full Changelog: https://github.com/NVIDIA/hyperflare-inference/commits/v0.1.0