ReVolta 0.1.4

ReVolta 0.1.4

A fix and performance release for ReVolta 0.1.3 (Qwen3.8 Flash-Next on 4 × V100 32 GB). It fixes an engine exit with CPU KV offload and DFlash2, makes MTP4 faster at two and three concurrent requests on PCIe-switch systems, and turns DFlash2 state replay into an option that now costs well under a millisecond per step. Each item links to its pull request, which has the details and measurements.

Fixes

• Engine exit with CPU KV offload and DFlash2. Under long-context traffic with CPU KV offload, DFlash2 deployments could exit with an OffloadingConnector.update_state_after_alloc assertion, after which every request failed. Block allocation for requests with both local and CPU-tier prefix hits is now two-phase. MTP4 was not affected. (#36)
• KV offload stores a prefix again after a restart from token 0. A preempted request that restarted from the beginning never re-stored a prefix the CPU tier had evicted, so later requests sharing it kept missing. Requests that are never preempted are unchanged. (#32)
• Anthropic endpoint returns 4xx for client errors. /v1/messages and /v1/messages/count_tokens return 400 for a prompt longer than --max-model-len and 404 for a missing model, instead of 500. Internal errors stay 500. (#32, fixes #26)
• INT8 KV with DFlash2 is rejected at startup. This combination used to fail after several minutes of loading; it now stops at configuration time. Use fp8_e4m3 with DFlash2, or INT8 KV with MTP. (#33, fixes #25)

Performance

• MTP4 at two and three concurrent requests, on SM70 TP4 behind a PCIe switch: the MoE all-reduce no longer uses the slow generic sum2 kernel at these batch sizes. End to end, two requests produce about 3% more tokens per second; one, four and eight requests are unchanged. DFlash2 is not affected. (#34)
• DFlash2 state replay, when turned on, now runs as a CUDA graph and costs about 0.4 ms per step with greedy drafting and about 0.8 ms with probabilistic drafting, instead of about 7 ms in 0.1.3. Outputs are identical with replay on and off. (#24)

New features

• DFlash2 state replay is optional ("spec_state_replay": true in sm70_dflash2). It keeps fewer recurrent-state blocks per request, so the GPU KV pool grows by about 12% (with greedy drafting, 1,379,634 to 1,542,558 tokens at 512K). Useful for long contexts or several long requests at once; see DFlash2. (#24)
• Opt-in stack-dump watchdog (--watchdog-config): writes the Python stacks of the engine and workers to a file when they make no progress for a set time. Off by default; a diagnostic aid, not a recovery mechanism. (#33)
• FP4-Local KV scales now include the MTP draft scales, mirroring the v1.1 pack. (#27)
• Documentation: a README first page for deciding whether ReVolta fits, NVLink community results, the experimental PLE whole-table disk option, and a round of consistency fixes. (#19, #29, #30, #35)

Upgrade notes

• DFlash2 drafting is now recommended as probabilistic ("draft_sample_method": "probabilistic"). It was validated at one to eight concurrent requests at temperature 1 with thinking, including tool calls and structured output, and the reference deployment now uses it. greedy still works with DFlash2; MTP must stay greedy. With probabilistic drafting the GPU KV pool is 1,366,121 tokens at 512K (2.61×), against 1,379,634 with greedy drafting. See Launch options.
• spec_state_replay now defaults to off. 0.1.3 turned it on by default where supported, and the recommended configuration turned it off. If your DFlash2 deployment relied on the 0.1.3 default, the GPU KV pool becomes smaller; set it to true to keep the larger pool. (#24)
• MTP4 output at two and three concurrent requests is not bitwise identical to 0.1.3, because the all-reduce summation order changed. Single-request output and DFlash2 output are unchanged. (#34)
• First start after upgrading: with a kept 0.1.3 compile cache, MTP4 and DFlash2 each compiled four graphs again on the first 0.1.4 start (about 5 s on the reference machine); later starts load them from the cache.

Known issues

• On /v1/messages, a response cut by max_tokens inside a tool call streams the partial tool_use input but returns no tool_use block when not streaming.
• DFlash2 remains workload-dependent: short English and code prompts benefit most; Chinese and long agent sessions often favour MTP4.
• Concurrent output can differ at near-ties as request batches change.
• The tool-calling limits listed for 0.1.3 remain. All limits are in Support and known limits.

Validation

The 0.1.4 release image passed the release gate on the reference 4 × V100 32 GB PCIe setup (24 arms, reference configuration):

• Single-request greedy output is bitwise identical to 0.1.3 for MTP4 and for DFlash2 with greedy drafting. Concurrent output differs only at near-ties, as before.
• The long-context replay that triggered the CPU-offload engine exit (#36) ran clean on 0.1.4: 2,808 of 2,808 requests, no engine exit. Both stress replays, 30 minutes of eight concurrent requests (flat host memory), multimodal, tools, CPU and SSD KV offload, developer-mode endpoints, INT8 KV with MTP4, the Anthropic error paths and the 512K boundary passed.
• Two checks were accepted with a reason: the eight-request top-p check of the DFlash2 arm is a false positive of its separate re-scoring prefill (the probabilities at generation time show no violations), and the 0.5 ms limit of the optional replay session check was set for greedy drafting (0.8 ms with probabilistic drafting; replay is off by default).
• Speed: MTP4 at two concurrent requests is about 3% faster than 0.1.3; other MTP4 levels are unchanged within 1%. A repeated A/B measurement at three to five requests found DFlash2 within about 2% per step of 0.1.3, with the same acceptance length. Details are in Performance.

Assets

File	Content
revolta-0.1.4-cp312-cp312-linux_x86_64.whl	Wheel (Linux x86-64, Python 3.12, CUDA 12.8; debug symbols stripped), 79,257,938 bytes, sha256 c3227094d9e2ca2de4b87c8870a2a945882f0e737573d3d808de9e6bf26e7239
revolta-0.1.4-cp312-cp312-linux_x86_64.debug.tar.xz	Separate debug symbols for the native extensions (unchanged from 0.1.3), 86,018,096 bytes, sha256 f7dcddab74e0e6c61107ae6265a50d97867fb3948b5a81ae6b44738b9a5dbfd4
SHA256SUMS	Checksums of the files above

Docker image: ghcr.io/leonccaa/revolta:0.1.4 (digest sha256:56a8e3aed3f515af575f82025823481fc86d3ff6c16928026f0be534cd4df0c0). PyPI: pip install revolta==0.1.4 (revolta-0.1.4-cp312-cp312-manylinux_2_34_x86_64.whl, sha256 ed23ca544a978486c1f6dc9cf6c2ddc9bae256b4ca890c46b1ba6ef35cf4596b).

Thanks

Thanks to the vLLM contributors behind vllm-project/vllm#52246, vllm-project/vllm#63010, vllm-project/vllm#55700, vllm-project/vllm#44409 and vllm-project/vllm#52807, and to @That80-20 and @vincentyan2023 for their test reports (#38, #37).