Skip to content

v2.5.1

Choose a tag to compare

View all tags
@halt95 halt95 released this 06 Oct 15:13
· 13 commits to main since this release
v2.5.1
352e006

Flash-Next v2.5.1: agent follow-up turns resume from cache. Same checkpoint and serving shape (TP2 × PP2 + EP,
MTP K=3) as v2.2.0, on the public vLLM v0.30.0 tag plus 97 commits, with a 924,993-token KV pool (806,792 on v2.2.0).
v2.5.0 was not published: these notes cover everything since v2.2.0.

  • Follow-up turns resume on a 64-token grid (--prefix-match-unit 64) instead of a 4,096-token block, and a
    conversation's cached prefix is kept until its next turn. One conversation: a follow-up starts in 0.96 s (v2.2.0:
    1.56 s). Eight deep conversations at once: 92.5 % of follow-up tokens from cache, median follow-up 1.61 s.
  • A 15 % larger KV pool on the same weights: three full 262K sessions, or six 131K sessions, resident at once.
  • Cold prefill 1.0–2.8 % faster than v2.2.0; single-stream decode about 3 % slower (v2.5.0's 4-boot release
    candidate: −2.81 %, significant; v2.5.1's 2 boots: −2.87 %).
  • Tokenization: transformers 5.18.0 reads tokenizer.json, so combining marks (Hindi, Thai, Arabic) tokenize as
    upstream intends; the Hugging Face repository carries the upstream tokenizer.json.
  • Images: up to 42 per request, each capped at 4 MP.

Build: scripts/build-v2.5.sh with this release's bundle and the v2.2.0 release's build-artifacts tarball (v2.5.1 changes
no compiled source; the tarball is reused, pinned by sha256). Serve: scripts/serve-v2.5.sh. Use a new compile cache.
Patch notes: CHANGELOG.md; measurements: benchmarks/2026-10-06/BENCH-CARD.md; reference: docs/reference.md.

Requirements: as v2.2.0 (Linux x86-64, glibc 2.34+, Python 3.13 with headers, an NVIDIA driver with CUDA 13.0 or newer,
four 24 GB Ampere cards, 96 GB host RAM qualified).

Assets:

file sha256
v2.5.1-from-upstream-v0.30.0.bundle.gz (the source; upstream/PIN-v2.5 pins the uncompressed bundle, d904e4685877e56b57c9a23a621b537239ba6316ad238c3848c5f03437d03d7c) 660b20b07f6f190890e4c1dbfccd4f4282ec336cd9bb0562f8adf01fd81192a3
build-artifacts.list (the 23 build-product paths; also in the repository) 2befac68d349aea26e0b35f59f4d04eda6c652ae7037f48fab8920c33ab5e432
build-artifacts-sm86-py313-cu130-v2.2.0.tar.gz (reused from the v2.2.0 release) cdca5ffcb3003d15e00896f0969134e16d7bfbe0a3c6dfe10698bec32573f592
Assets 4
Loading