Repository navigation
v2.2.0
Flash-Next v2.2.0: rebased on vLLM 0.30.0 — requires the new runtime. Same checkpoint, same serving shape
(TP2 × PP2 + EP, MTP K=3) and the same 806,792-token KV pool as v2.0.1 (three full 262,144-token sessions measured resident
at once on v2; re-tested at 3 × 256K on v2.2), on the public vLLM v0.30.0 tag plus 58 commits. A reliability release at parity with v2, not a speed-up:
single-stream decode lands at 0.973–1.040 of the v2 gate medians (v2.2.0: one release-candidate boot; v2: five-boot
gate medians; maintainer-measured).
- Greedy output repeats within one compile cache. Four determinism switches (
MERLIN_FULL_K,MERLIN_QSA_SORT,
MERLIN_TIE_DET,MERLIN_TIE_RECENT) plus in-kernel sorted emission. With prefix caching and thinking off, 8
identical T=0 requests gave 1 distinct output in every sampled cell from 3,960 to 261,802 prompt tokens on release
candidates; sampled, not a batch-invariance guarantee, and only within one compile cache. - A fail-closed fault on any GPU rank now stops the engine instead of hanging it (workers gone in 3.0–3.5 s in the
injection tests); drain-order shutdown fixes (orderly shutdown ~9.0 s to 2.65–3.55 s, measured on release candidates
before the final tree). - The rare illegal-address engine death is contained, not proven absent. Ring rows are tagged and validated
before pooling; with the shippedGUARD=warna mismatched row stops the engine with a named cause instead of an
illegal-address fault. No rate is claimed. - First-serve toolchain. FlashInfer and Triton compile kernels on the first request: the build now installs the
CUDA 13.0 nvcc wheels and the links FlashInfer needs, andserve-v2.2.shchecks compiler, ninja, the Python headers
and nvcc against the driver before starting. The container recipe carries all of them (image build and entrypoint checks tested without GPUs; GPU serving
verified on the bare-metal routes). Thanks to the user who reported the
runtime-JIT failures on the v2.0.1 image. - Opt-in topology selector (
AUTO_TOPO=1). For NVLink-bridged pairs, hosts with more than four GPUs and
multi-socket hosts: it orders the lane's four GPUs so each tensor-parallel pair gets the best link (the NVLink rule is
tested on synthetic topologies, not on bridged hardware), and turns on
vLLM's P2P check when the driver reports P2P on a layout other than the reference one. Off by default, where the launch is exactly as before. Seedocs/topology-selector.md.
Build: scripts/build-v2.2.sh (verifies commit and tree of the source against upstream/PIN-v2.2). Serve:
scripts/serve-v2.2.sh. Use a new compile cache; v2.0.x artefacts do not carry over. Details, the benchmark card and
known behaviours: README ("What changed in v2.2.0") and benchmarks/2026-09-24/BENCH-CARD.md.
Requirements: Linux x86-64 with glibc 2.34 or newer (the two extensions are built on Ubuntu 22.04), Python 3.13 with
headers, an NVIDIA driver with CUDA 13.0 or newer, four 24 GB Ampere cards (qualified with peer-to-peer; also runs
without, with lower prefill), headless with no other CUDA process on them, 96 GB host RAM (qualified; ~69 GiB measured
resident floor).
Assets:
| file | sha256 |
|---|---|
v2.2.0-from-upstream-v0.30.0.bundle.gz (the source; upstream/PIN-v2.2 pins the uncompressed bundle) |
4de4ae5e175f58cdde9bfbfa570e813ff9d089b3e6627f4cf22c933a9c9cb206 |
build-artifacts-sm86-py313-cu130-v2.2.0.tar.gz (the two v2.2 extensions for sm_86 + the stock 0.30.0 build products) |
cdca5ffcb3003d15e00896f0969134e16d7bfbe0a3c6dfe10698bec32573f592 |
build-artifacts.list (the 23 build-product paths; also in the repository) |
2befac68d349aea26e0b35f59f4d04eda6c652ae7037f48fab8920c33ab5e432 |