ReVolta 0.1.4

ReVolta 0.1.4

A fix and performance release for ReVolta 0.1.3 (Qwen3.8 Flash-Next on 4 × V100 32 GB). It fixes an engine exit with CPU KV offload and DFlash2, makes MTP4 faster at two and three concurrent requests on PCIe-switch systems, and turns DFlash2 state replay into an option that now costs well under a millisecond per step. Each item links to its pull request, which has the details and measurements.

Fixes

• Engine exit with CPU KV offload and DFlash2. Under long-context traffic with CPU KV offload, DFlash2 deployments could exit with an OffloadingConnector.update_state_after_alloc assertion, after which every request failed. Block allocation for requests with both local and CPU-tier prefix hits is now two-phase. MTP4 was not affected. (#36)
• KV offload stores a prefix again after a restart from token 0. A preempted request that restarted from the beginning never re-stored a prefix the CPU tier had evicted, so later requests sharing it kept missing. Requests that are never preempted are unchanged. (#32)
• Anthropic endpoint returns 4xx for client errors. /v1/messages and /v1/messages/count_tokens return 400 for a prompt longer than --max-model-len and 404 for a missing model, instead of 500. Internal errors stay 500. (#32, fixes #26)
• INT8 KV with DFlash2 is rejected at startup. This combination used to fail after several minutes of loading; it now stops at configuration time. Use fp8_e4m3 with DFlash2, or INT8 KV with MTP. (#33, fixes #25)

Performance

• MTP4 at two and three concurrent requests, on SM70 TP4 behind a PCIe switch: the MoE all-reduce no longer uses the slow generic sum2 kernel at these batch sizes. End to end, two requests produce about 3% more tokens per second; one, four and eight requests are unchanged. DFlash2 is not affected. (#34)
• DFlash2 state replay, when turned on, now runs as a CUDA graph and costs about 0.4 ms per step with greedy drafting and about 0.8 ms with probabilistic drafting, instead of about 7 ms in 0.1.3. Outputs are identical with replay on and off. (#24)

New features

• DFlash2 state replay is optional ("spec_state_replay": true in sm70_dflash2). It keeps fewer recurrent-state blocks per request, so the GPU KV pool grows by about 12% (with greedy drafting, 1,379,634 to 1,542,558 tokens at 512K). Useful for long contexts or several long requests at once; see DFlash2. (#24)
• Opt-in stack-dump watchdog (--watchdog-config): writes the Python stacks of the engine and workers to a file when they make no progress for a set time. Off by default; a diagnostic aid, not a recovery mechanism. (#33)
• FP4-Local KV scales now include the MTP draft scales, mirroring the v1.1 pack. (#27)
• Documentation: a README first page for deciding whether ReVolta fits, NVLink community results, the experimental PLE whole-table disk option, and a round of consistency fixes. (#19, #29, #30, #35)

Upgrade notes

• DFlash2 drafting is now recommended as probabilistic ("draft_sample_method": "probabilistic"). It was validated at one to eight concurrent requests at temperature 1 with thinking, including tool calls and structured output, and the reference deployment now uses it. greedy still works with DFlash2; MTP must stay greedy. With probabilistic drafting the GPU KV pool is 1,366,121 tokens at 512K (2.61×), against 1,379,634 with greedy drafting. See Launch options.
• spec_state_replay now defaults to off. 0.1.3 turned it on by default where supported, and the recommended configuration turned it off. If your DFlash2 deployment relied on the 0.1.3 default, the GPU KV pool becomes smaller; set it to true to keep the larger pool. (#24)
• MTP4 output at two and three concurrent requests is not bitwise identical to 0.1.3, because the all-reduce summation order changed. Single-request output and DFlash2 output are unchanged. (#34)
• First start after upgrading: with a kept 0.1.3 compile cache, MTP4 and DFlash2 each compiled four graphs again on the first 0.1.4 start (about 5 s on the reference machine); later starts load them from the cache.

Known issues

• On /v1/messages, a response cut by max_tokens inside a tool call streams the partial tool_use input but returns no tool_use block when not streaming.
• DFlash2 remains workload-dependent: short English and code prompts benefit most; Chinese and long agent sessions often favour MTP4.
• Concurrent output can differ at near-ties as request batches change.
• The tool-calling limits listed for 0.1.3 remain. All limits are in Support and known limits.

Validation

The 0.1.4 release image passed the release gate on the reference 4 × V100 32 GB PCIe setup (24 arms, reference configuration):

• Single-request greedy output is bitwise identical to 0.1.3 for MTP4 and for DFlash2 with greedy drafting. Concurrent output differs only at near-ties, as before.
• The long-context replay that triggered the CPU-offload engine exit (#36) ran clean on 0.1.4: 2,808 of 2,808 requests, no engine exit. Both stress replays, 30 minutes of eight concurrent requests (flat host memory), multimodal, tools, CPU and SSD KV offload, developer-mode endpoints, INT8 KV with MTP4, the Anthropic error paths and the 512K boundary passed.
• Two checks were accepted with a reason: the eight-request top-p check of the DFlash2 arm is a false positive of its separate re-scoring prefill (the probabilities at generation time show no violations), and the 0.5 ms limit of the optional replay session check was set for greedy drafting (0.8 ms with probabilistic drafting; replay is off by default).
• Speed: MTP4 at two concurrent requests is about 3% faster than 0.1.3; other MTP4 levels are unchanged within 1%. A repeated A/B measurement at three to five requests found DFlash2 within about 2% per step of 0.1.3, with the same acceptance length. Details are in Performance.

Assets

File	Content
revolta-0.1.4-cp312-cp312-linux_x86_64.whl	Wheel (Linux x86-64, Python 3.12, CUDA 12.8; debug symbols stripped), 79,257,938 bytes, sha256 c3227094d9e2ca2de4b87c8870a2a945882f0e737573d3d808de9e6bf26e7239
revolta-0.1.4-cp312-cp312-linux_x86_64.debug.tar.xz	Separate debug symbols for the native extensions (unchanged from 0.1.3), 86,018,096 bytes, sha256 f7dcddab74e0e6c61107ae6265a50d97867fb3948b5a81ae6b44738b9a5dbfd4
SHA256SUMS	Checksums of the files above

Docker image: ghcr.io/leonccaa/revolta:0.1.4 (digest sha256:56a8e3aed3f515af575f82025823481fc86d3ff6c16928026f0be534cd4df0c0). PyPI: pip install revolta==0.1.4 (revolta-0.1.4-cp312-cp312-manylinux_2_34_x86_64.whl, sha256 ed23ca544a978486c1f6dc9cf6c2ddc9bae256b4ca890c46b1ba6ef35cf4596b).

Thanks

Thanks to the vLLM contributors behind vllm-project/vllm#52246, vllm-project/vllm#63010, vllm-project/vllm#55700, vllm-project/vllm#44409 and vllm-project/vllm#52807, and to @That80-20 and @vincentyan2023 for their test reports (#38, #37).

ReVolta 0.1.3

ReVolta 0.1.3

A fix and feature release for ReVolta 0.1.2 (Qwen3.8 Flash-Next on 4 × V100 32 GB). It returns client errors for corrupt media, fixes a multimodal scheduling stall and memory growth during sustained traffic, and makes Anthropic errors usable by Anthropic SDKs. DFlash2 gains selector codebook sharding and speculative state replay, both on by default when DFlash2 is selected and the configuration is supported. Together with smaller CUDA graph defaults, they leave more GPU memory for the KV cache. Replay slows decoding when the target samples, so the recommended DFlash2 configuration turns it off; sharding and the graph defaults still raise the KV pool by about 9% over 0.1.2. Unsupported INT8 draft KV is rejected during configuration. The reference image/video limits no longer include audio, which this model does not support. MTP4 stays the default speculative method; DFlash2 remains optional.

Behaviour changes

• DFlash2 CUDA graph defaults. With the ReVolta AWQ TP4 profile, DFlash2 now defaults VLLM_SM70_DFLASH2_TAIL_CUDAGRAPHS to 0 and VLLM_V2_CUDAGRAPH_MEM_MIB to 1200 before sizing the KV pool (#4, #23). This disables the single-request verifier tail graphs; other verifier and drafter graph settings stay as configured. The combined development run used 1.10 GiB of graph memory per GPU, within the 1200 MiB reserve. MTP, non-speculative runs and NVFP4 keep their existing profile defaults, including the 700 MiB reserve. Explicit environment values still win. After capture, a warning reports when graph memory exceeds the reserve by more than the greater of 64 MiB and 5%, and suggests a value.
• DFlash2 memory options. Selector codebook sharding (#9) and speculative state replay (#21) now default on when DFlash2 is selected and their requirements are met. Both can be disabled in sm70_dflash2. The recommended DFlash2 configuration sets "spec_state_replay": false; see New features for scope and Deployment notes for the capacity and speed tradeoff. This does not change the default speculative method from MTP4.
• Anthropic error types. /v1/messages now derives error.type from the HTTP status instead of returning internal vLLM class names (#8; vllm-project/vllm#62294). The mapping is 400 → invalid_request_error, 401 → authentication_error, 403 → permission_error, 404 → not_found_error, 413 → request_too_large, 429 → rate_limit_error, 503 → overloaded_error, other 5xx → api_error, and other 4xx → invalid_request_error. Status codes and messages stay the same. The mapping also covers token-count errors and SSE error events, so clients can handle and retry them by type.
• INT8 draft KV with DFlash2. Configuration now raises a ValueError before loading weights when DFlash uses FLASH_ATTN_V100 (or automatic backend selection), the resolved draft KV dtype is INT8, and the draft uses sliding windows or attention sinks (#22). The released DFlash2 draft uses sliding-window attention and inherits the target KV dtype unless overridden. Use E4M3 or FP16 KV, or MTP4 with INT8 KV. An INT8 target with speculative_config.kv_cache_dtype set to float16 or auto still passes this check, but in 0.1.3 it then fails during KV cache initialization (see Known issues, #25). In 0.1.2, the unsupported inherited-INT8 configuration reached worker warm-up, failed and never became healthy.
• Reference multimodal limits. The 0.1.3 reference launch settings use --limit-mm-per-prompt '{"image": 16, "video": 1}', removing the previous "audio": 16 entry; the README and multimodal guide update is tracked in #19. Qwen3.8 Flash-Next supports images and video, not audio. Generic multimodal defaults are unchanged, and the audio loader fix below does not add audio support to this model.

Fixes

• DFlash2 graph memory is reserved before KV allocation. The AWQ profile now selects the graph reserve by speculative method (#4, refined by #23). In 0.1.2, DFlash2 without an explicit 1500 MiB reserve could let KV allocation consume memory needed by its 1.37–1.39 GiB graphs, causing out-of-memory errors in concurrent long prefill steps that filled the 8192-token budget. The new default pairs a 1200 MiB reserve with the verifier tail graphs disabled; explicit overrides still need enough reserve for the graphs they enable.
• Corrupt media returns HTTP 400. A PNG with a corrupt chunk structure, or an image above Pillow's own pixel safety limit, could return 500 before ReVolta's image resizing ran (#5; vllm-project/vllm#62704, vllm-project/vllm#60308). These are now client errors. Valid images follow the same path as before. The generic soundfile loader also converts a recognised but corrupt audio file to a client error when that loader is installed and used.
• Multimodal requests can progress with EAGLE/MTP lookahead. Two adjacent media items that could not fit together in the encoder cache could leave a request scheduling zero tokens forever, blocking requests behind it (#6). The scheduler now finishes the tokens before the next item so the first can be released. The same fix covers an indivisible media item whose size exactly matches the token batch budget. Normal chat templates put tokens between media items; raw prompts can expose the adjacent-item case. Related report: InternLM/lmdeploy#5061.
• Memory during sustained traffic. Shared-memory message readers now drain their notification socket periodically while busy (#7; vllm-project/vllm#63165). Before, queued notification commands accumulated until the reader went idle. Message contents and delivery semantics do not change.
• Anthropic streaming errors. An error carried inside the chat stream is now returned with its own status and message instead of failing chunk parsing (#8). Other mid-stream failures use api_error.

New features

DFlash2 selector codebook sharding (on by default)

sm70_dflash2.sharded_selector_codebook splits the selector's vocabulary rows across tensor-parallel ranks instead of keeping both complete codebooks on every GPU (#9). At TP4 this reduces the codebooks from about 242 MiB to 61 MiB per GPU, saving about 182 MiB for the KV pool. Each row has one owner; all-reduce combines the lookups. Finite nonzero values stay bitwise identical; negative zero can lose its sign. Checkpoint shapes are checked before slicing. Set the option to false to keep the previous layout. The isolated development comparison measured 3.5% more KV capacity, identical greedy output and decode speed within noise; the combined configuration is recorded below.

DFlash2 speculative state replay (on by default where supported)

sm70_dflash2.spec_state_replay repeats the recurrent-state calculation after sampling and commits the accepted states, instead of retaining a separate state block for every draft position (#21). With Qwen3.8 Flash-Next and 7 drafts, the six GDN cache groups need 12 state blocks per running request instead of 54, leaving more cache space for concurrent requests. Replay uses the verifier's recorded inputs and launch shapes, including prefix-cache block boundaries.

Replay applies to linear DFlash verification on SM70 with Model Runner V2, align cache mode, synchronous scheduling and the mixed-QKV verifier. Other configurations keep the per-draft state blocks. When the packed verifier sm70_dflash2.fused_gdn_verify is enabled, by its automatic policy or explicitly, default replay yields to it; an explicit "spec_state_replay": true overrides an automatic fused_gdn_verify default, and explicitly enabling both is an error. VLLM_SM70_DFLASH2_FUSED_QKV_PACK is a different setting and does not disable replay. MTP and non-speculative runs keep their existing state allocation and arithmetic paths.

Development validation reported 177 of 177 kernel state comparisons bitwise identical to stored states, and matching mixed-QKV and split-verifier outputs and states for contiguous and row-strided inputs. Replay alone raised capacity from 1,268,831 tokens / 2.42× to 1,418,305 tokens / 2.71× at 512K. With replay, sharding and the new graph defaults together (the 0.1.3 code defaults), 24 of 24 single-request greedy streams matched 0.1.2 in token IDs and logprobs; eight concurrent 96K-token prompts used about 26% peak KV against 60% on 0.1.2, without preemption. These development runs precede the final candidate and release-image gate below.

Replay costs decode speed when the target samples. In a same-window replay of 30 real agent sessions (one request at a time, target sampling), a decoding step took 37.9 ms with replay on against 30.6 ms on 0.1.2; with only replay turned off, 0.1.3 took 30.7 ms and its 30 outputs were identical to 0.1.2's. Profiling traced the extra time to the CPU cost of launching replay, not to more GPU work: each step issues 36 Triton replay launches and about 150 small table-building operations, which take about 1 ms of GPU time. With greedy decoding the GPU is still busy with queued target work and hides these launches; with target sampling the GPU has already drained its queue and waits for each launch. The recommended configuration therefore keeps replay off in 0.1.3, until the launch cost is removed and replay is measured again with sampling.

Experimental: whole PLE table on disk (opt-in, off by default)

The AWQ checkpoint's PLE table (about 48 GiB) normally stays in pinned host memory. An existing code path can instead keep the whole table as file-backed mappings of the checkpoint's safetensors files and read the needed rows in a separate PLE process. 0.1.3 does not change this code; this release records its first functional check on the reference machine. It is not PLE cascade support and is not part of the reference configuration.

To try it, set VLLM_PLE_CPU_OFFLOAD=1, VLLM_PLE_DISK_OFFLOAD=1 and VLLM_SM70_QWEN38_HYBRID_PLE=0, add --safetensors-load-strategy lazy, and keep "ple_disk_cascade": false in --kernel-config (the check als...

ReVolta 0.1.2

ReVolta 0.1.2

A fix and feature release for ReVolta 0.1.1 (Qwen3.8 Flash-Next on 4 × V100 32 GB). Only Python code changed. The native extensions were rebuilt from unchanged sources; all 18 native files are byte-identical to 0.1.1. This release fixes tool calling with thinking on, parallel_tool_calls, tool_choice="none" streaming and the Anthropic Messages endpoint. It resizes large images, tightens the generic multimodal defaults (V100 keeps its own; see Multimodal), adds opt-in SSE keep-alive, and adds DFlash2 speculative decoding as an option. DFlash2 is off by default; MTP stays the default. In the release tests with DFlash off, greedy output and single-request fixed-seed sampling were bitwise identical to 0.1.1, and speed was the same. With 4 or 8 concurrent requests, outputs are not bitwise reproducible from run to run, on 0.1.1 or 0.1.2: they diverge at near-ties, depending on how the requests are batched.

Behaviour changes

Tool calling (Chat Completions and Responses API):

• Implicit <tool_call> end of reasoning. With thinking on, the model can write <tool_call> before </think>. Under a JSON grammar (tool_choice="required", a named tool, or a JSON response_format), that marker now only ends the reasoning and is not part of the content. Before, non-streaming required returned no tool calls with finish_reason="tool_calls", named calls returned arguments that began with <tool_call>, and JSON response formats returned content that was not JSON. This applies to streaming and non-streaming responses. Generation does not change.
• parallel_tool_calls=false with tool_choice="required" now limits the grammar to one call (maxItems: 1, as in vllm-project/vllm#50502). Before, the model could generate extra calls that were dropped afterwards. The first call is generated as before.
• parallel_tool_calls: null means the default (parallel calls allowed), as in the OpenAI API (vllm-project/vllm#44955, vllm-project/vllm#48098). Chat Completions no longer keeps only the first call, and the Responses API no longer rejects the request.
• A named tool without parameters. When the forced tool has no parameters, or {}, the arguments are constrained to a JSON object (vllm-project/vllm#45290). Before, a tool without parameters had no grammar, so the raw tool-call XML became the arguments, and {} allowed any JSON value.
• Tools, response_format and tool_choice="auto" together (qwen3_coder). The grammar now allows a tool call or the JSON answer. It is the union from 1CatAI/1Cat-vLLM#653 that qwen3_xml already used. Before, the JSON answer grammar blocked tool calls, so the model could not call a tool. Chat Completions only. This changes generation for these requests.
• tool_choice="none" in streaming returns tool-call markup as content, as non-streaming responses already did (vllm-project/vllm#42752). "none" is also the default for a request without tools. Before, streaming parsed the markup into tool calls with finish_reason="tool_calls". A trailing < is no longer held back for one chunk.
• Whitespace before a tool call (streaming). With automatic tool parsing, content that is only whitespace (for example the line breaks after </think>) is held. If a tool call follows, the whitespace is dropped, as in non-streaming responses. If text follows, it arrives with the first text chunk. The joined content does not change. Chat Completions only.
• Short tool calls in streaming (qwen3_coder). When one chunk carried a whole call body, which is common with multi-token MTP steps, the arguments could end as "{". The complete arguments are now sent. Calls without parameters were affected most. This also fixes Responses streams, which have no end-of-stream flush.
• Non-finite numbers in tool arguments. inf, Infinity, NaN or 1e999 in a number parameter are kept as the original string (vllm-project/vllm#43984). Before, qwen3_coder returned every call of the response as raw content (non-streaming), stopped the stream, or produced invalid JSON.

Anthropic Messages (/v1/messages):

• A forced tool (tool_choice: {"type": "tool"}) now reports stop_reason: "tool_use", not end_turn (vllm-project/vllm#47598). Clients that loop on tool_use now run the tool.
• tool_choice.disable_parallel_tool_use is now honoured. It maps to parallel_tool_calls=false (vllm-project/vllm#52021).
• A named call that was cut short (for example by max_tokens) is left out of the content. Before, the whole response failed with a 500. stop_reason is max_tokens.

Multimodal:

• V100 with FlashAttention-V100 (the ReVolta target) keeps its own default. Without --limit-mm-per-prompt, the engine allows 1 image and no video per request, as in 0.1.0 and 0.1.1. If the value leaves out "video", video is disabled. The startup log says Using SM70 Flash-V100 multimodal default. Set the option explicitly and include "video". To copy:
  • the generic 0.1.2 default: --limit-mm-per-prompt '{"image": 8, "video": 1, "audio": 1}';
  • the reference deployment (used in the README launch command): --limit-mm-per-prompt '{"image": 16, "video": 1, "audio": 16}'.
• On other hardware or attention backends, the default per-request limits are now 8 images, 1 video and 1 audio item (0.1.1: 16, 1 and 16).
• Images in earlier chat messages count toward the image limit. --limit-mm-per-prompt overrides each modality it names.
• Large images are resized. Qwen3.8 Flash-Next resizes an image larger than 1,048,576 pixels (1024 × 1024 in area) before the vision encoder. It does not reject it. HF's smart_resize keeps the aspect ratio and aligns to 32-pixel multiples. This applies on all hardware, including V100. Small text in a large image can become blurry. Crop the region or raise the budget when detail matters.
• Near the area limit, alignment can also change a slightly smaller image. For example, 977 × 1073 becomes 960 × 1056 instead of 992 × 1088.
• Explicit image size settings (size, max_pixels, min_pixels, do_resize, also inside images_kwargs) keep priority. To restore the checkpoint's 16,777,216-pixel budget, use --mm-processor-kwargs '{"images_kwargs": {"size": {"longest_edge": 16777216, "shortest_edge": 65536}}}'. Video sizing does not change.
• Video sampling defaults to 16 frames when num_frames is not set (0.1.1: 32).
• Per-request mm_processor_kwargs and media_io_kwargs that are not empty are rejected with HTTP 400 (vllm-project/vllm#58830). Start the server with --trust-request-mm-kwargs only when the clients are trusted. Omitted values, null and {} work as before. This applies to chat, completions, Responses, render, tokenize, pooling and run-batch. The Python LLM API does not change.
• The startup memory profile uses the new image budget: at most 1,024 image tokens per image instead of 16,384. On the reference setup, the KV pool did not shrink (see the top-k workspace fix below).

Other API changes:

• SSE keep-alive (opt-in). --sse-keep-alive-interval N sends an SSE comment line (: keep-alive) after N idle seconds on streaming /v1/chat/completions, /v1/completions, /v1/responses and /v1/messages responses (vllm-project/vllm#51034). A proxy with a read timeout then keeps the connection open while a request waits in the queue, runs its prefill, or pauses between tokens. SSE clients ignore comment lines. The default is 0 (off). Harmony Responses streams are excluded.
• Claude Code billing header. In Chat Completions, a system text that starts with an x-anthropic-billing-header line loses that line (vllm-project/vllm#59419). Its value changes on every request and defeated the prefix cache. /v1/messages already removed it.
• SM70 AWQ weight reload (reload_weights and weight-transfer updates) now fails before any tensor changes. Dense layers kept stale kernel state, and MoE layers could point at freed memory. Restart with the new weights instead.

Fixes

Memory and KV cache:

• Multimodal encoder cache entries could keep the memory of their whole batch alive, so partial eviction freed nothing. On the reference setup, 9.1 MiB of cache entries held 1,118 MiB of GPU memory. Each entry now owns its memory. Output values do not change.
• With MTP and synchronous scheduling, the draft QSA layer now shares the target's top-k workspace instead of allocating its own. The saved memory goes to the KV pool: on the reference setup it grows from 1,145,881 to 1,153,143 tokens (+7,262).
• With KV offload, a finished request no longer marks shared recurrent-state checkpoints (prefix hits and the shared-prefix boundary) for early eviction. Other requests with the same prefix keep their CPU cache hits.
• With --mamba-cache-mode align and async scheduling (vLLM's default), every prefill chunk kept one superseded recurrent-state block per Mamba cache group until the request ended. These blocks are now freed (from 1Cat-vLLM, by cookys). In an upstream test on 4 × V100, a 253,731-token cold prefill went from 343 s with 6 preemptions to 60 s with none. Synchronous scheduling (--no-async-scheduling, as in the reference setup) is unchanged.

Engine robustness:

• If the PLE offload worker process exits, the GPU worker now stops at once and reports the exit code. Before, the engine could hang, for example in a CUDA graph that waited for a PLE result. An exit during startup also reports the exit code.
• With all PLE rows in host memory, raw host-to-device copies now run on the destination GPU. PLE cascade (ple_disk_cascade=true) is still unsupported in 0.1.x.
• Development mode: /sleep?level=1 could kill the engine. The CUDA runtime lookup matched TileLang's libcudart_stub.so (vllm-project/vllm#47586). Library file names are now matched exactly.
• Sleep levels 1 and 2 now check their prerequisites on every worker first. A failed check returns an error and leaves requests and caches as they were.
• Devel...

ReVolta 0.1.1

ReVolta 0.1.1

A bug-fix release for ReVolta 0.1.0 (Qwen3.8 Flash-Next on 4 × V100 32 GB). Only Python code
changed. The native extensions were rebuilt from unchanged sources: their code and data are
identical to 0.1.0, and three nvcc-built files differ only in local symbol names and the
debug-file checksum. On requests that do not hit one of the
fixes, greedy output and fixed-seed sampling are identical to 0.1.0.

Behaviour changes

• Stop strings with thinking on match only after the reasoning ends. A stop string
  repeated inside the reasoning no longer ends the request with empty content, and a
  stop of </think> no longer ends a request at the end of the reasoning.
• Reasoning/content split uses the </think> token, not its text. A literal
  </think> written in the reasoning or the answer no longer moves the split. With
  thinking off, the whole output is content, in streaming and non-streaming responses.
• qwen3_coder tool calls stream string arguments as they are generated, instead of
  sending a long argument only at the end. A repeated parameter name keeps its first
  value (a warning is logged).
• Invalid request parameters return 400 with a one-line warning in the log, without
  an ASGI traceback.

Fixes

API and output processing:

• Streaming with n ≥ 2 kept prompt log-probabilities for every choice.
• A structured-output failure while generating now ends only that request; the engine
  keeps serving.
• An output buffer could be reused while it was still being sent to the API server.
• The end-of-stream tool-call flush could join a sent prefix and the full arguments into
  invalid JSON.

Scheduling, KV offload and state:

• Resetting the prefix cache (development endpoints and LLM.reset_prefix_cache) with KV
  offload enabled no longer crashes the engine; preemptions outside the step loop are
  flushed correctly.
• Without MTP, a prompt no longer takes one extra prefill step.
• Draft warm-up runs no longer write QSA ring state into the null block.
• Loading an AWQ checkpoint with a missing tensor fails at startup instead of running.

Memory and startup:

• PLE layers no longer allocate speculative state blocks that are never used
  (KV pool 1,145,881 tokens on the reference setup, from 1,140,505).
• NCCL communicators are created before memory profiling, so they no longer allocate
  memory inside the profiling window.
• Standard NVFP4 checkpoints get their validated kernel settings from the ReVolta
  profile; they no longer need manual environment variables.
• FP4-Local with a data-parallel vision encoder profiles the encoder at startup.
• Two blocking GPU synchronizations in mixed batches (PLE metadata, QSA prefill bounds) are gone.
• A 512K view whose outer config has no rope_parameters now loads (#2).
• The startup profile line no longer reports pex_p2p before the workers have resolved
  the interconnect, and the profile's own compile setting no longer triggers a
  diagnostic warning (#2).

Multimodal:

• A repeated image or video is no longer evicted from the encoder cache while another
  occurrence still needs it.
• Media preprocessing runs on its own thread and no longer blocks tokenization.
• A multimodal request rejected after rendering no longer leaves its media registered.
• Videos with an unknown or invalid frame rate no longer fail.
• Startup checks that the visual token IDs in the config match the tokenizer.
• Faster placeholder matching for long multimodal prompts.

Documentation:

• The README's 512K view script failed to load. Fixed in the README on 2026-10-21 (#2).
• Host memory guidance for large image bursts (#2).
• The cause of batch-dependent output differences: the order of the tensor-parallel
  all-reduce sums, amplified near routing and block-selection ties.

Release gate

All results come from the reference machine (4 × V100 32 GB PCIe behind one PCIe switch).
Baselines were measured with 0.1.0 in the same window.

Check	Result
Greedy output and C1 fixed-seed sampling vs 0.1.0	Identical
Full AWQ gate (4/8-request stress, multimodal stress, offload readback, 211K retrieval, 512K boundary)	Pass
Speed vs 0.1.0 (512-token responses, C1/C2/C4/C8)	Throughput ratio 1.004 / 1.019 / 1.008 / 0.999
FP4-Local and standard NVFP4 smoke (release candidate)	Pass
Offload connector GPU unit tests	158 passed; 2 failures also present in 0.1.0 (sliding-window layout, not used by this model)

Known limitations

See README, Support and known limits.
New in this list: a qwen3_coder string argument containing the literal text
</function> is cut at that point.

Assets

File	Content
revolta-0.1.1-cp312-cp312-linux_x86_64.whl	Wheel (Linux x86-64, Python 3.12, CUDA 12.8; debug symbols stripped)
revolta-0.1.1-cp312-cp312-linux_x86_64.debug.tar.xz	Separate debug symbols for the native extensions
SHA256SUMS	Checksums of the files above

Docker image: ghcr.io/leonccaa/revolta:0.1.1 (digest sha256:dce36236fa4018c0d45fcf1a33284b8cdbb20638713ce5fdd2a77fca686cf084).

PyPI: pip install revolta==0.1.1 (added 2026-10-21). The PyPI file is
revolta-0.1.1-1-cp312-cp312-manylinux_2_34_x86_64.whl, sha256
87ad51a5aac165edc0dc094b96a8d9e861668930504ecbf0788512e9e7305b19. It contains the
same files as the wheel above. Only the metadata differs: plain version pins for
torch==2.10.0, torchaudio==2.10.0 and torchvision==0.25.0 instead of
download.pytorch.org URLs, the manylinux_2_34_x86_64 platform tag and build tag 1.
Both wheels need glibc 2.34 or newer.

The first PyPI file, revolta-0.1.1-cp312-cp312-manylinux_2_34_x86_64.whl, has a
malformed RECORD: paths that contain commas are not quoted. uv refuses it and pip
prints warnings. pip and uv prefer the build-1 file, which fixes this.

Thanks

Thanks to @jttdev for the first NVLink test and the bug
reports in #2.

ReVolta 0.1.0

ReVolta 0.1.0

First public release of ReVolta, a vLLM-based inference engine for NVIDIA Tesla V100
(SM70). The release targets Qwen3.8 Flash-Next (multimodal) on 4 × V100 32 GB PCIe.
See the README for the quick start,
support matrix and known limitations.

Highlights

• Validated configuration: TP4 + DCP2, four-token MTP drafting, E4M3 KV cache, 512K
  context through YaRN, multimodal input, up to 8 concurrent requests, CPU KV offload.
• Weight formats: AWQ g32 (primary), standard NVFP4 and NVIDIA FP4-Local (vision tower
  TP4).
• KV cache: E4M3 (default) and INT8.
• KV offload: off, CPU, or CPU + SSD tier.
• Interconnect: PCIe switch with P2P; PCIe without P2P (validated by disabling P2P);
  NVLink path present but unverified.
• An automatic "ReVolta profile" sets the tuned SM70 kernel switches; the startup log
  lists each value and its source.

Release gate

All results come from one reference machine (4 × V100 32 GB PCIe behind one PCIe
switch, 128 GiB container memory limit).

Check	Result
AWQ g32 full gate (greedy/fixed-seed parity, 4/8-request stress, multimodal stress, offload readback, 211K retrieval, 512K boundary)	Pass; greedy output identical to the previous reference build
Release wheel / image (this build) full gate	Pass, including the 512K boundary; greedy output identical to the validated source-overlay build
Offload off / CPU + SSD	Pass / Pass (8 GiB CPU pool + 40 GiB disk tier)
No P2P	Pass: correct output, about 30–55% of P2P throughput
INT8 KV, standard NVFP4, FP4-Local	Pass / Pass / Pass

Decode throughput (512 output tokens, thinking off, client aggregate):
C1 152.5, C2 167.2, C4 262.2,
C8 383.6 tok/s. C2 varies between runs (159.7–173.7 tok/s with the same code); see the README.

Assets

File	Content
revolta-0.1.0-cp312-cp312-linux_x86_64.whl	Wheel (Linux x86-64, Python 3.12, CUDA 12.8; debug symbols stripped)
revolta-0.1.0-cp312-cp312-linux_x86_64.debug.tar.xz	Separate debug symbols for the native extensions
SHA256SUMS	Checksums of the files above

Install with pip install and the wheel's release URL (see the README). ReVolta 0.1.0 is not on PyPI.

Docker image: ghcr.io/leonccaa/revolta:0.1.0 (digest sha256:bc53a7f3bf041f19ed5544d534c57f8bae042745706acac91554a83ba5d48f29).

Known limitations

• Greedy outputs at C4/C8 can differ between runs (batch non-invariance); C1 is
  reproducible.
• INT8 KV decode is about 5–8% slower than E4M3; INT8 with the SSD tier is not
  supported.
• NVLink, TP2/TP8, V100 16 GB and other models are unverified.
• ple_disk_cascade=true is not supported in 0.1.0; keep the default false.
• With thinking on, avoid stop strings: one that appears in the reasoning ends the
  request with empty content, and in streaming responses </think> can leak into
  content (both inherited from upstream); fix planned for 0.1.1.

Full list: README, Known limitations.

Acknowledgements

ReVolta is derived from 1Cat-vLLM and
vLLM. Third-party components and licenses are
listed in NOTICE and LICENSES/.