ReVolta 0.1.3

ReVolta 0.1.3

A fix and feature release for ReVolta 0.1.2 (Qwen3.8 Flash-Next on 4 × V100 32 GB). It returns client errors for corrupt media, fixes a multimodal scheduling stall and memory growth during sustained traffic, and makes Anthropic errors usable by Anthropic SDKs. DFlash2 gains selector codebook sharding and speculative state replay, both on by default when DFlash2 is selected and the configuration is supported. Together with smaller CUDA graph defaults, they leave more GPU memory for the KV cache. Replay slows decoding when the target samples, so the recommended DFlash2 configuration turns it off; sharding and the graph defaults still raise the KV pool by about 9% over 0.1.2. Unsupported INT8 draft KV is rejected during configuration. The reference image/video limits no longer include audio, which this model does not support. MTP4 stays the default speculative method; DFlash2 remains optional.

Behaviour changes

• DFlash2 CUDA graph defaults. With the ReVolta AWQ TP4 profile, DFlash2 now defaults VLLM_SM70_DFLASH2_TAIL_CUDAGRAPHS to 0 and VLLM_V2_CUDAGRAPH_MEM_MIB to 1200 before sizing the KV pool (#4, #23). This disables the single-request verifier tail graphs; other verifier and drafter graph settings stay as configured. The combined development run used 1.10 GiB of graph memory per GPU, within the 1200 MiB reserve. MTP, non-speculative runs and NVFP4 keep their existing profile defaults, including the 700 MiB reserve. Explicit environment values still win. After capture, a warning reports when graph memory exceeds the reserve by more than the greater of 64 MiB and 5%, and suggests a value.
• DFlash2 memory options. Selector codebook sharding (#9) and speculative state replay (#21) now default on when DFlash2 is selected and their requirements are met. Both can be disabled in sm70_dflash2. The recommended DFlash2 configuration sets "spec_state_replay": false; see New features for scope and Deployment notes for the capacity and speed tradeoff. This does not change the default speculative method from MTP4.
• Anthropic error types. /v1/messages now derives error.type from the HTTP status instead of returning internal vLLM class names (#8; vllm-project/vllm#62294). The mapping is 400 → invalid_request_error, 401 → authentication_error, 403 → permission_error, 404 → not_found_error, 413 → request_too_large, 429 → rate_limit_error, 503 → overloaded_error, other 5xx → api_error, and other 4xx → invalid_request_error. Status codes and messages stay the same. The mapping also covers token-count errors and SSE error events, so clients can handle and retry them by type.
• INT8 draft KV with DFlash2. Configuration now raises a ValueError before loading weights when DFlash uses FLASH_ATTN_V100 (or automatic backend selection), the resolved draft KV dtype is INT8, and the draft uses sliding windows or attention sinks (#22). The released DFlash2 draft uses sliding-window attention and inherits the target KV dtype unless overridden. Use E4M3 or FP16 KV, or MTP4 with INT8 KV. An INT8 target with speculative_config.kv_cache_dtype set to float16 or auto still passes this check, but in 0.1.3 it then fails during KV cache initialization (see Known issues, #25). In 0.1.2, the unsupported inherited-INT8 configuration reached worker warm-up, failed and never became healthy.
• Reference multimodal limits. The 0.1.3 reference launch settings use --limit-mm-per-prompt '{"image": 16, "video": 1}', removing the previous "audio": 16 entry; the README and multimodal guide update is tracked in #19. Qwen3.8 Flash-Next supports images and video, not audio. Generic multimodal defaults are unchanged, and the audio loader fix below does not add audio support to this model.

Fixes

• DFlash2 graph memory is reserved before KV allocation. The AWQ profile now selects the graph reserve by speculative method (#4, refined by #23). In 0.1.2, DFlash2 without an explicit 1500 MiB reserve could let KV allocation consume memory needed by its 1.37–1.39 GiB graphs, causing out-of-memory errors in concurrent long prefill steps that filled the 8192-token budget. The new default pairs a 1200 MiB reserve with the verifier tail graphs disabled; explicit overrides still need enough reserve for the graphs they enable.
• Corrupt media returns HTTP 400. A PNG with a corrupt chunk structure, or an image above Pillow's own pixel safety limit, could return 500 before ReVolta's image resizing ran (#5; vllm-project/vllm#62704, vllm-project/vllm#60308). These are now client errors. Valid images follow the same path as before. The generic soundfile loader also converts a recognised but corrupt audio file to a client error when that loader is installed and used.
• Multimodal requests can progress with EAGLE/MTP lookahead. Two adjacent media items that could not fit together in the encoder cache could leave a request scheduling zero tokens forever, blocking requests behind it (#6). The scheduler now finishes the tokens before the next item so the first can be released. The same fix covers an indivisible media item whose size exactly matches the token batch budget. Normal chat templates put tokens between media items; raw prompts can expose the adjacent-item case. Related report: InternLM/lmdeploy#5061.
• Memory during sustained traffic. Shared-memory message readers now drain their notification socket periodically while busy (#7; vllm-project/vllm#63165). Before, queued notification commands accumulated until the reader went idle. Message contents and delivery semantics do not change.
• Anthropic streaming errors. An error carried inside the chat stream is now returned with its own status and message instead of failing chunk parsing (#8). Other mid-stream failures use api_error.

New features

DFlash2 selector codebook sharding (on by default)

sm70_dflash2.sharded_selector_codebook splits the selector's vocabulary rows across tensor-parallel ranks instead of keeping both complete codebooks on every GPU (#9). At TP4 this reduces the codebooks from about 242 MiB to 61 MiB per GPU, saving about 182 MiB for the KV pool. Each row has one owner; all-reduce combines the lookups. Finite nonzero values stay bitwise identical; negative zero can lose its sign. Checkpoint shapes are checked before slicing. Set the option to false to keep the previous layout. The isolated development comparison measured 3.5% more KV capacity, identical greedy output and decode speed within noise; the combined configuration is recorded below.

DFlash2 speculative state replay (on by default where supported)

sm70_dflash2.spec_state_replay repeats the recurrent-state calculation after sampling and commits the accepted states, instead of retaining a separate state block for every draft position (#21). With Qwen3.8 Flash-Next and 7 drafts, the six GDN cache groups need 12 state blocks per running request instead of 54, leaving more cache space for concurrent requests. Replay uses the verifier's recorded inputs and launch shapes, including prefix-cache block boundaries.

Replay applies to linear DFlash verification on SM70 with Model Runner V2, align cache mode, synchronous scheduling and the mixed-QKV verifier. Other configurations keep the per-draft state blocks. When the packed verifier sm70_dflash2.fused_gdn_verify is enabled, by its automatic policy or explicitly, default replay yields to it; an explicit "spec_state_replay": true overrides an automatic fused_gdn_verify default, and explicitly enabling both is an error. VLLM_SM70_DFLASH2_FUSED_QKV_PACK is a different setting and does not disable replay. MTP and non-speculative runs keep their existing state allocation and arithmetic paths.

Development validation reported 177 of 177 kernel state comparisons bitwise identical to stored states, and matching mixed-QKV and split-verifier outputs and states for contiguous and row-strided inputs. Replay alone raised capacity from 1,268,831 tokens / 2.42× to 1,418,305 tokens / 2.71× at 512K. With replay, sharding and the new graph defaults together (the 0.1.3 code defaults), 24 of 24 single-request greedy streams matched 0.1.2 in token IDs and logprobs; eight concurrent 96K-token prompts used about 26% peak KV against 60% on 0.1.2, without preemption. These development runs precede the final candidate and release-image gate below.

Replay costs decode speed when the target samples. In a same-window replay of 30 real agent sessions (one request at a time, target sampling), a decoding step took 37.9 ms with replay on against 30.6 ms on 0.1.2; with only replay turned off, 0.1.3 took 30.7 ms and its 30 outputs were identical to 0.1.2's. Profiling traced the extra time to the CPU cost of launching replay, not to more GPU work: each step issues 36 Triton replay launches and about 150 small table-building operations, which take about 1 ms of GPU time. With greedy decoding the GPU is still busy with queued target work and hides these launches; with target sampling the GPU has already drained its queue and waits for each launch. The recommended configuration therefore keeps replay off in 0.1.3, until the launch cost is removed and replay is measured again with sampling.

Experimental: whole PLE table on disk (opt-in, off by default)

The AWQ checkpoint's PLE table (about 48 GiB) normally stays in pinned host memory. An existing code path can instead keep the whole table as file-backed mappings of the checkpoint's safetensors files and read the needed rows in a separate PLE process. 0.1.3 does not change this code; this release records its first functional check on the reference machine. It is not PLE cascade support and is not part of the reference configuration.

To try it, set VLLM_PLE_CPU_OFFLOAD=1, VLLM_PLE_DISK_OFFLOAD=1 and VLLM_SM70_QWEN38_HYBRID_PLE=0, add --safetensors-load-strategy lazy, and keep "ple_disk_cascade": false in --kernel-config (the check also set "ple_disk_release_pages": true). Run it without CPU KV offload: the check removed --kv-transfer-config, and the combination was not tested.

On the 0.1.3 candidate (Uncensored AWQ, E4M3 KV, 4 × V100 PCIe), MTP4 started and served 131K-token prompts, 4 concurrent requests and images under both a 64 GiB and a 32 GiB container memory limit; under 64 GiB it also served a 491K-token prompt and eight queued requests (at most four ran at the same time). Peak container memory was about 54 GiB under the 64 GiB limit; under 32 GiB, startup peaked at 16.5 GiB. DFlash2 also started and served under 64 GiB. The GPU KV pool was about 7% smaller (1,073,265 tokens with MTP4). Greedy outputs matched the pinned-host path; one of eight sampled single-request outputs diverged at a near-tie, so the two paths are not bitwise identical. When the PLE process was killed, the client received an error after about 9 s and the engine exited after 11–13 s instead of hanging. Not measured: decode speed and time to first token, a machine with little physical RAM, a local SSD (the check read the checkpoint from network storage), and the net memory saving against the default path in the same window.

Deployment notes

• Default and optional methods. MTP4 remains the reference default. For DFlash2, keep the 0.1.2 drafter setup, "draft_sample_method": "greedy" and the verification-width ladder "dflash_concurrency_drafts": {"4": 3, "5": 4}. Keep these six sm70_dflash2 options enabled: context_kv_graph, context_pipeline, verify_fastpath, fused_gdn_metadata, fused_smallq_metadata and grouped_smallq_metadata. Sharding and replay default on inside sm70_dflash2 in --speculative-config; they need no new environment variables and can be disabled independently. Add "spec_state_replay": false to the same object (see Turn state replay off below). Leave --block-size unset so the drafter cache can use the automatic layout.
• Remove the old graph-reserve override. Remove an explicit VLLM_V2_CUDAGRAPH_MEM_MIB=1500 from a 0.1.2 DFlash2 deployment to use the new AWQ profile defaults: tail graphs off and a 1200 MiB reserve. An explicit 1500 still wins and leaves less memory for KV. To restore the old tail graphs deliberately, set both VLLM_SM70_DFLASH2_TAIL_CUDAGRAPHS=1 and VLLM_V2_CUDAGRAPH_MEM_MIB=1500 (#23).
• Turn state replay off. Set "spec_state_replay": false inside sm70_dflash2. This restores the per-draft state blocks and removes about 7 ms per decoding step when the target samples (30.6 ms per step on 0.1.2, 37.9 ms with replay on, 30.7 ms with replay off, at one request). Sharding and the new graph defaults stay in effect: the GPU KV pool at 512K is 1,379,634 tokens (2.63×), against 1,268,831 (2.42×) on 0.1.2 and 1,542,558 (2.94×) with replay on. At eight greedy requests, two release-image runs with replay off measured 302.1 and 335.8 tokens/s and one run with replay on measured 312.6 (with replay, three or more running requests use the mixed-QKV verifier); development runs showed replay up to about 5% slower there. The difference at high concurrency is within run-to-run variation. Consider turning replay on only if your clients decode greedily (temperature 0) and need the extra KV capacity.
• Compile cache. The first start after upgrading to 0.1.3 compiles cold once, including with MTP or no speculation: the graph cache hashes the full traced GDN source file changed by #21. Sharding and replay also add configuration factors when enabled; disabling them removes their added factors but does not restore the old source hash. Allow extra startup time and retain persistent compile caches for later restarts.
• FP4-Local MTP draft in E4M3. The FP4-Local scale pack v1.1-base-b9ce18a6 adds calibrated K/V scales for the checkpoint's built-in MTP draft, so the draft can use "kv_cache_dtype": "fp8_e4m3" as well. A 512K smoke on the release image passed with E4M3 target and draft KV and reported a GPU KV pool of 1,474,106 tokens (2.81×), against 1,321,612 (2.52×) with an FP16 draft cache; see the checkpoint guide. The v1.0 tag has no draft scales and still needs an FP16 draft cache.
• Images and video. On V100 with FlashAttention-V100, set --limit-mm-per-prompt explicitly and include "video". Without it, the default remains 1 image and no video. For another model that supports audio, install its audio dependencies separately with vllm[audio].

The table below records the 0.1.2 reference measurements, before the 0.1.3 DFlash2 memory changes. The tested setup was our 4 × V100 PCIe deployment, TP4, DCP2, --max-model-len 524288, --max-num-seqs 8, an 8192-token batch budget and 48 GiB CPU KV offload. DFlash2 used 7 drafts and the 1500 MiB graph reserve. Each cell pairs the engine's reported GPU KV cache size with its maximum-concurrency ratio for 512K (524,288-token) requests, rounded to two decimals. This follows the engine's hybrid-cache calculation, including recurrent state and sliding windows.

Weights	KV format	MTP4: KV tokens / maximum concurrency at 512K	DFlash2: KV tokens / maximum concurrency at 512K
Uncensored AWQ	E4M3	1,153,143 / 2.20×	1,268,831 / 2.42×
Uncensored AWQ	FP16	673,749 / 1.29×	781,584 / 1.49×
NVIDIA FP4-Local	E4M3	1,321,612 / 2.52×	1,335,042 / 2.55×
Standard NVFP4	E4M3	1,250,448 / 2.39×	1,337,745 / 2.55×
Uncensored AWQ	INT8	1,140,072 / 2.17×	Unsupported with inherited INT8 draft KV

KV format names refer to the target. FP4-Local's MTP4 configuration uses its own draft with FP16 KV; standard NVFP4 uses the tested FP8 draft. The AWQ FP16 configurations use FP16 KV for both target and draft.

The table compares the 0.1.2 reference with AWQ E4M3 DFlash2 capacity measured on the 0.1.3 release image on the same setup (#9, #21 and #23). Other formats were not remeasured with the 0.1.3 DFlash2 changes. AWQ E4M3 with MTP4 measured the same 1,153,143 tokens as 0.1.2.

DFlash2 configuration	GPU KV tokens / maximum concurrency at 512K
0.1.2 reference, tail graphs on, 1500 MiB reserve	1,268,831 / 2.42×
0.1.3 recommended: sharding on, state replay off, tail graphs off, 1200 MiB reserve	1,379,634 / 2.63× (+8.7% KV tokens)
0.1.3 defaults: sharding and state replay on, tail graphs off, 1200 MiB reserve	1,542,558 / 2.94× (+21.6% KV tokens)

These figures depend on the 512K maximum length and 8-sequence scheduler configuration; they are not a fixed capacity for every request length. Neither 2.63× nor 2.94× fits three full 512K requests at once. Shorter requests can fit at higher concurrency, up to the configured scheduler limit. Requests beyond that limit queue, and cache pressure can cause preemption. Changing the maximum length, sequence limit, cache format or speculative method can change the pool layout and capacity.

Cold-prefix prefill measurements with 0.1.2, one request on the same PCIe setup, are below. Throughput is computed prompt tokens divided by time to first token. These are reference measurements, not 0.1.3 release-gate results.

Prompt length	MTP4	DFlash2
8K	3,382 tokens/s	3,120 tokens/s
131K	3,016 tokens/s	3,093 tokens/s

The release gate records final 0.1.3 prefill and C1/C2/C4/C8 output-throughput comparisons against 0.1.2 in the same window and configuration; the historical prefill figures above are not those results.

Known issues

• DFlash2 remains workload-dependent. In the 0.1.2 measurements, short English and code prompts benefited most; Chinese and long agent sessions often favoured MTP4. The 0.1.3 release-image check with the recommended settings shows the same pattern: at one request 1.15–1.24× MTP4 on English and code and 0.85–0.97× on Chinese, and 302.1 and 335.8 (two runs) against 361.6 tokens/s at eight requests.
• Explicit graph reserves still override the profile. A reserve smaller than the captured graphs can leave too little memory for concurrent long prefills; the warning does not resize the pool after capture.
• DFlash2 state replay adds about 7 ms per decoding step when the target samples, because the CPU launch cost of the replay kernels is no longer hidden behind GPU work. The recommended configuration turns replay off (see Deployment notes). Removing that launch cost, for example by capturing replay in a CUDA graph, and then measuring replay with sampling is follow-up work before replay is recommended.
• The DFlash2 draft stays on "draft_sample_method": "greedy". Probabilistic drafting has not been validated on the SM70 Flash-Next configuration and is not recommended for 0.1.3. This is the draft-token selection setting, separate from client sampling of the target output.
• DFlash2 does not support pipeline parallelism or inherited INT8 draft KV on FlashAttention-V100; the latter is now rejected before loading weights (#22). A separately configured FP16 draft cache passes the configuration check but fails at KV cache initialization in 0.1.3 (#25). Verifier settings that change numerics (fused_gdn_verify, fused_gdn_norm, fused_gdn_split, fused_gdn_combined_split) remain outside the recommended Flash-Next configuration. Final image/video, preemption and long-running service coverage must be read from the release gate.
• Concurrent output can differ at near-ties as request batches change. A bitwise comparison across speculative methods is also not guaranteed. A batch-invariant all-reduce remains deferred.
• The existing tool-calling limitations remain: $ref parameters can be treated as strings; a failed required parse can still report finish_reason="tool_calls"; stop tokens inside JSON strings and very large parallel-call deltas remain problematic; a literal </function> can truncate a qwen3_coder string argument. Strict native XML structural tags remain deferred.
• Literal <|image_pad|> text can take the place of a real image, as in upstream vLLM and HF (vllm-project/vllm#57740).
• Multimodal limits apply per request, not to total host memory. A host memory guard for concurrent media bursts remains deferred. Sleep level 2 also needs enough free host memory for model buffers; it is not supported on the 128 GiB reference setup (unchanged from 0.1.2).
• Anthropic /v1/messages returns HTTP 500 / api_error for some client errors, for example a prompt longer than --max-model-len, because the router maps every exception to 500. 0.1.2 behaves the same (with internal_error). Picking the upstream fix is proposed for 0.1.4 (#26; vllm-project/vllm#52246).
• INT8 target KV cannot be used with DFlash2. The configuration check rejects the default form, but its message also suggests a separate FP16 draft cache (speculative_config.kv_cache_dtype set to float16 or auto); that form passes the check and then fails during KV cache initialization with INT8 v1 uses 16-token kernel pages dividing scheduler blocks. Use fp8_e4m3, float16 or auto KV with DFlash2, or INT8 KV with MTP. Tracked in #25.
• PLE cascade (ple_disk_cascade=true) remains unsupported in 0.1.x; keep it disabled. The whole-table disk option under New features is experimental, separate from cascade, and has only the functional check described there.

See also Support and known limits.

Release gate

These checks ran on the final candidate or the release image, against 0.1.2 on the reference 4 × V100 32 GB PCIe setup in the same window where a comparison is shown. Behaviour checks used 200 W per GPU; speed comparisons used 250 W for both versions. Rows marked "on the candidate" ran on the release candidate built from the same source; one row reports development runs and says so.

Check	Result
Greedy output and C1 fixed-seed sampling vs 0.1.2 (DFlash off)	Pass on the release image: greedy output bitwise identical to the 0.1.1 baseline, to the earlier gate and to the 0.1.3 release candidate (0.1.2's MTP4 output is bitwise identical to the same baseline); C1 fixed-seed sampling 8/8 identical, and identical on a repeat in the same process
Outputs at 4 and 8 concurrent requests, including candidate self-repeat	Review, known category: on the release image, 4 of the C4/C8 control groups differed on a second pass (0.1.2 release: 3). These are the batch-dependent near-tie differences under Known issues; the development comparisons showed the same pattern with and without the DFlash2 changes. Single-request self-repeat matched
Full AWQ gate (4/8-request stress, multimodal stress, offload readback, 211K retrieval, 512K boundary)	Pass on the release image (judge, four-stage multimodal stress, in-gate greedy vs the baseline, 512K boundary)
Speed vs 0.1.2 (512-token responses, C1/C2/C4/C8)	Same: 0.1.2 → 0.1.3 aggregate output 153.1 → 153.2 (C1), 170.7 → 169.4 (C2), 263.4 → 268.3 (C4), 389.3 → 390.4 tokens/s (C8); time to first token within the noise of request arrival order (8 repeats each, both versions at 250 W in the same window)
GPU KV pool and maximum concurrency at 512K, MTP4 and DFlash2 by format	Measured on the release image with MTP4: Uncensored AWQ + E4M3 1,153,143 tokens / 2.20×, + INT8 1,140,072 / 2.17×, NVIDIA FP4-Local + E4M3 1,321,612 / 2.52×, standard NVFP4 + E4M3 1,250,448 / 2.39×, all the same as 0.1.2. DFlash2 on AWQ + E4M3: 1,379,634 / 2.63× with the recommended settings, 1,542,558 / 2.94× with state replay on. FP16 KV and the FP4-Local and NVFP4 DFlash2 values were not remeasured
Tool-calling matrix (required / named / auto × streaming × thinking)	Pass on the release image: 128 hard checks; the one review item is by design (Anthropic forced-truncation output differs between streaming and non-streaming), as in 0.1.2
SSE keep-alive on all four streaming endpoints	Pass on the release image: 31 hard checks, including keep-alive comments before the first token on an 80K-token prompt; the review item (empty Completions text on that prompt) matches 0.1.2
Multimodal defaults, image resizing and image/video limits without audio	Pass: the reference deployment's {"image": 16, "video": 1} limit is enforced (16 images); default 1 MP resizing and the multimodal policy checks pass on the release image; the review items (reading of a 4096×4096 image and of eight 1024×768 images) appear identically on the baseline
Encoder cache memory	Pass: encoder-cache storage stays bounded over 30 greedy, 10 sampled and 10 control rounds
Small KV pool: shared-prefix recurrent state, eviction after preemption, image prefix read back from CPU	Pass on the candidate: two requests sharing a prefix both hit 50,592 cached tokens with the correct recurrent state; eviction after preemption passed for both preempted requests; text and image prefixes were read back from CPU under cache pressure
Development mode: sleep levels 0 and 1, sleep preflight, pause modes, pause during a pending KV load, transfers in flight; record sleep level 2 and disk-tier coverage	Pass: sleep levels 0 and 1 (also with a request in flight, abort and wait), sleep preflight rejection with the engine still serving, the pause matrix, pause during a pending KV load, and KV stores in flight, with the CPU tier and with CPU + disk tier. Not covered: sleep level 2 is not run on this configuration (model buffers up to 87.4 GiB vs 62.9 GiB available host memory, so it is not supported here); the disk-tier round trip and a pause that catches a disk-tier load in flight were not reached (the read-back came from the CPU tier)
Development mode with KV offload: prefix-cache reset during a CPU-to-GPU reload	Pass on the release image: the engine stays healthy and greedy output is unchanged after resets during reloads. Two answers to one long-context retrieval question are wrong in exactly the same way as in 0.1.2 and as with a full recompute without any reset, so they are a model retrieval error on that document, not a reset or reload problem; no attempt caught a reload in flight for the other two cases
GPU unit tests: top-k workspace under compute-sanitizer, NVFP4 PLE retry, host-to-device copy context, PLE worker exit inside a CUDA graph, 4-GPU PLE tests	Pass: top-k workspace padding (also under compute-sanitizer), NVFP4 PLE pinned-retry, host-to-device copy context (fails on the baseline, passes on 0.1.3), PLE worker exit inside a CUDA graph (times out on the baseline as expected, exits on 0.1.3), 4-GPU PLE pytest
DFlash2 recommended configuration on the release image: startup, tools, GPU/CPU prefix hits, long prompts, mixed prefill/decode OOM stress and C1/C4/C8 speed	Pass with state replay off ("spec_state_replay": false confirmed in the engine arguments, no replay state layout in the log): 1.17 GiB graph reserve with the actual graph pool within the estimate, GPU KV pool 1,379,634 tokens, tool calls, GPU prefix hit and CPU hit after a GPU reset, a ~100K-token prompt, both OOM stress patterns without OOM. Greedy speed at 200 W against MTP4: one request 1.24× English code, 1.20× English reasoning, 1.15× agent code, 0.97× and 0.85× on two Chinese prompts; 4 requests 1.01×; 8 requests 302.1 vs 361.6 tokens/s. A second run on the go-live DFlash2 deployment itself, which kept the accepted compile cache for the production seed, passed the same checks with the same KV pool; one-request speeds matched within 0.5% and 8 requests measured 335.8 tokens/s. The same check with replay on passed with 1,542,558 tokens and 312.6 tokens/s at 8 requests
Native MTP draft RoPE following a YaRN target	Pass on the release image with the stock native-RoPE MTP draft and an FP16 draft cache: the draft RoPE follows the target's YaRN scaling (start-up log line); draft acceptance 0.83 / 0.71 / 0.43 at 245K / 300K / 489K-token prompts, the same as 0.1.2. With an E4M3 draft cache and no calibrated draft K/V scales, start-up is refused by design
FP4-Local and standard NVFP4 smoke	Pass on the release image (start-up, profile switch lines, 5 greedy prompts twice, 2 images)
Corrupt media: HTTP 400 on Chat Completions and Responses, then a valid image request	Pass on the release image: a truncated PNG and a PNG header declaring 400 million pixels each returned HTTP 400 with BadRequestError and the loader message on both Chat Completions and Responses; a valid image request on each endpoint then returned 200 with an answer
Adjacent-media stall: completion with constrained encoder cache, concurrent ordinary request, normal multimodal parity and Mamba align coverage	Pass on the candidate with the reference align Mamba cache mode: the request with two adjacent large images (20,022 prompt tokens) finished under a constrained encoder cache, where 0.1.2 did not finish within 240 s; a concurrent text request completed; an ordinary two-image greedy request matched 0.1.2 for 64 of 64 tokens; the encoder budget check passed
Sustained C8 traffic: EngineCore/worker host-memory trend and decode throughput	Pass on the release image (MTP4 reference configuration, 200 W): 30 minutes of eight concurrent streaming completions (4096 tokens each, 249 completed, 0 errors, eight requests running in 97% of samples). Over 119 samples, EngineCore resident memory changed by +0.02 MiB, each worker by +0.26 MiB, and the container by +1.3 MiB (116.2 GiB total). Decode throughput averaged 558 tokens/s (coefficient of variation 7.8%); its variation followed the MTP acceptance length, with no sustained decline. This shows no growth on 0.1.3; 0.1.2 was not run in the same window
Anthropic errors: unknown model, overlong context, token count and mid-stream error type	Partial. Unknown model: 404 / not_found_error; count_tokens: 200. Overlong prompt: 500 / api_error instead of 400 / invalid_request_error, a known issue present since 0.1.2 (#26). A mid-stream server error could not be triggered reliably on the release image; the SSE error mapping is covered by unit tests only
INT8 draft KV + DFlash2: configuration-time rejection; INT8 + MTP4 and E4M3/FP16 + DFlash2 remain usable; record separate FP16 draft-cache coverage	INT8 KV + DFlash2 is rejected at configuration time on the release image (76 s, before model loading). INT8 KV + MTP4 starts and serves (GPU KV pool 1,140,072 tokens, 2.17× at 512K); E4M3 + DFlash2 passes the DFlash2 check above. INT8 target KV + DFlash2 with a separate FP16 draft cache passes the configuration check but fails at KV cache initialization; see Known issues (#25)
DFlash2 sharding and replay: independent and combined on/off state and greedy parity, prefix boundaries, long C8 prompts, preemption, memory stress and speed	Pass (development runs before the release image; the sampled speed result is from the release image), with the speed finding above. Replay restored 177 of 177 recurrent states bitwise; the mixed-QKV and split verifiers matched bitwise for contiguous and row-strided inputs. Single-request greedy streams matched 0.1.2 in token IDs and logprobs (24 of 24) with sharding and the graph defaults, with replay alone, and with all three, including 512 tokens after a 256K-token prompt. Eight concurrent 96K and 120K prompts finished without preemption: peak KV use was 55% and 60% with sharding and the graph defaults, 29% and 33% with replay alone, and 26% at 96K with all three (0.1.2: 60% and 66%); both memory stress patterns passed. Sampled single-request replay of 30 agent sessions: 30.6 ms per step on 0.1.2, 37.9 with replay, 30.7 without, with 30 of 30 outputs identical to 0.1.2 when replay is off
Cold-prefix prefill: 8K/32K/131K, two-request and multi-turn comparisons	Same: server prefill 3,432 → 3,439 (8K), 3,331 → 3,338 (32K), 2,978 → 2,985 tokens/s (131K); two concurrent 32K prompts 3,083 → 3,093 tokens/s combined; multi-turn time to first token 2.31–2.36 s in both versions (250 W, same window)
Final candidate/version, build manifest, native-extension comparison and artifact checksums	Pass: the image and wheel report version 0.1.3 and were built from 4abaa004b7; native sources, build files and dependencies are unchanged from 0.1.1 and 0.1.2; artifact checksums are listed under Assets

Assets

File	Content
revolta-0.1.3-cp312-cp312-linux_x86_64.whl	Wheel (Linux x86-64, Python 3.12, CUDA 12.8; debug symbols stripped), 79,263,087 bytes, sha256 d6c34243e1586b8ad321f338b54bd1905e50db69caa34a3bc2a1f4cd6fedade1
revolta-0.1.3-cp312-cp312-linux_x86_64.debug.tar.xz	Separate debug symbols for the native extensions, 86,018,096 bytes, sha256 f7dcddab74e0e6c61107ae6265a50d97867fb3948b5a81ae6b44738b9a5dbfd4
SHA256SUMS	Checksums of the files above

Docker image: ghcr.io/leonccaa/revolta:0.1.3 (digest sha256:bebb8816367ac971b54dc56df2b86038aff2b207bdff54040717aac6a97bdd24).

PyPI: pip install revolta==0.1.3. The PyPI file is revolta-0.1.3-cp312-cp312-manylinux_2_34_x86_64.whl, sha256 e0253a100cc8c2a0651753d978c884d9f88001636a9dc9699a3480ad0bf9b098. It contains the same files as the wheel above. Only the metadata differs (METADATA, WHEEL, RECORD): plain version pins for torch==2.10.0, torchaudio==2.10.0 and torchvision==0.25.0 instead of download.pytorch.org URLs, and the manylinux_2_34_x86_64 platform tag. Both wheels need glibc 2.34 or newer.

Thanks

Thanks to the vLLM contributors behind vllm-project/vllm#62704, vllm-project/vllm#60308, vllm-project/vllm#63165 and vllm-project/vllm#62294, to the contributors to #4–#9 and #21–#23, to jttdev for the community measurements in #2, and to incoai for the DFlash2 drafter.