v0.3.1

What's Changed

• Unblock the GPU CI tiers on the Crusoe/Spur fleet by @limou102 in #168
• fix(e2e): an empty rocm-smi process list is not an unreadable node by @limou102 in #169
• Carry a serving anchor across width, score prefill/decode disaggregation, and add the simulation manual by @araina-amd in #171
• ci: pin a node of our own after a JobLaunchFailure by @limou102 in #170
• docs(compat): add ROCm 10.1 to the compatibility matrix by @llying-001 in #148
• Feat/prefill wait for decode by @weilei0120 in #173
• Roll a worker without dropping traffic by @weilei0120 in #176
• Keep interleaved system messages in place on /v1/responses for GLM by @weilei0120 in #178
• Merge ServiceSpec podAnnotations onto rendered Infera pods. by @weilei0120 in #180
• Expose serving metrics on /metrics with engine federation. by @weilei0120 in #183
• feat(sglang): enable opt-in Decode radix and HiCache with MTP by @llying-001 in #184
• Fix the GPU CI tiers and split run_tests.sh into tests/lib by @limou102 in #181
• feat(router): release prefill load independently on completion by @llying-001 in #185
• fix: harden engine startup and Mooncake preflight validation by @llying-001 in #192
• feat(router): add opt-in session affinity with independent PD bindings by @llying-001 in #186
• feat(glm5.2 atom): Update atom nightly image, add glm5.2 atom dcp examples by @ChengYao-amd in #197
• feat(atom): GLM-5.2 ATOM on nightly_202610131505 (LMCache GTT pool, DCP pad-row fix), PP4 prefill 4P4D kit, ATOM router support by @ChengYao-amd in #198
• feat(router): add opt-in Decode input-demand routing; key session affinity on X-Session-ID by @llying-001 in #188
• feat(agent_sys): an agent work system, and the five-stage LLM e2e optimisation flow by @dorado269 in #167
• Refuse an out-of-regime anchor, and price sparse attention as selection plus indexing by @araina-amd in #174
• Price pipeline-parallel hops, the token return, and cross-node stage boundaries by @araina-amd in #179
• Price hybrid-model layers from their own operations by @araina-amd in #190
• Add tiered KV placement below HBM and a Pareto sweep by @araina-amd in #194
• Restyle the Pareto dashboard and draw measured runs over it by @araina-amd in #195
• Speculative drafter in the Pareto sweep, decode context parallelism, dashboard depth pick by @araina-amd in #206
• Model each disaggregated pool as it is served by @araina-amd in #200
• pareto: disaggregated layouts in both modes by @araina-amd in #201
• Projection accuracy: busiest-rank MoE load, measured decode sharding, sparse KV dtype by @araina-amd in #205
• Report the GEMM shapes a DES replay runs by @araina-amd in #193
• EP AllToAll: size dispatch on the sequences a rank owns under attention-DP by @ajassani in #175
• Attention-DP: apply the request split once in the profiler by @ajassani in #177
• feat(build): carry Mooncake RDMA rail and ACK-timeout patches for SGLang by @llying-001 in #187
• Map Anthropic thinking.budget_tokens to GLM-5.3 hard think cap by @weilei0120 in #199
• Report streamed input tokens, bound Mooncake batch registration, enable thinking budget by default by @weilei0120 in #204
• Apply the thinking-budget logit processor in the EAGLE verify step by @weilei0120 in #211
• feat(atom): optimize atom pd performance by @ChengYao-amd in #209
• feat(sglang): v0.5.22 image, GLM-5.2-MXFP4 mixed and PD (TP1xPP4 prefill) examples, router decode request balancing by @llying-001 in #210
• ci(release): publish an atom-nightly image for the GLM-5.2 ATOM kit by @ChengYao-amd in #214

New Contributors

• @ajassani made their first contribution in #175

Full Changelog: v0.2.11...v0.3.1