ReVolta 0.1.2

ReVolta 0.1.2

A fix and feature release for ReVolta 0.1.1 (Qwen3.8 Flash-Next on 4 × V100 32 GB). Only Python code changed. The native extensions were rebuilt from unchanged sources; all 18 native files are byte-identical to 0.1.1. This release fixes tool calling with thinking on, parallel_tool_calls, tool_choice="none" streaming and the Anthropic Messages endpoint. It resizes large images, tightens the generic multimodal defaults (V100 keeps its own; see Multimodal), adds opt-in SSE keep-alive, and adds DFlash2 speculative decoding as an option. DFlash2 is off by default; MTP stays the default. In the release tests with DFlash off, greedy output and single-request fixed-seed sampling were bitwise identical to 0.1.1, and speed was the same. With 4 or 8 concurrent requests, outputs are not bitwise reproducible from run to run, on 0.1.1 or 0.1.2: they diverge at near-ties, depending on how the requests are batched.

Behaviour changes

Tool calling (Chat Completions and Responses API):

• Implicit <tool_call> end of reasoning. With thinking on, the model can write <tool_call> before </think>. Under a JSON grammar (tool_choice="required", a named tool, or a JSON response_format), that marker now only ends the reasoning and is not part of the content. Before, non-streaming required returned no tool calls with finish_reason="tool_calls", named calls returned arguments that began with <tool_call>, and JSON response formats returned content that was not JSON. This applies to streaming and non-streaming responses. Generation does not change.
• parallel_tool_calls=false with tool_choice="required" now limits the grammar to one call (maxItems: 1, as in vllm-project/vllm#50502). Before, the model could generate extra calls that were dropped afterwards. The first call is generated as before.
• parallel_tool_calls: null means the default (parallel calls allowed), as in the OpenAI API (vllm-project/vllm#44955, vllm-project/vllm#48098). Chat Completions no longer keeps only the first call, and the Responses API no longer rejects the request.
• A named tool without parameters. When the forced tool has no parameters, or {}, the arguments are constrained to a JSON object (vllm-project/vllm#45290). Before, a tool without parameters had no grammar, so the raw tool-call XML became the arguments, and {} allowed any JSON value.
• Tools, response_format and tool_choice="auto" together (qwen3_coder). The grammar now allows a tool call or the JSON answer. It is the union from 1CatAI/1Cat-vLLM#653 that qwen3_xml already used. Before, the JSON answer grammar blocked tool calls, so the model could not call a tool. Chat Completions only. This changes generation for these requests.
• tool_choice="none" in streaming returns tool-call markup as content, as non-streaming responses already did (vllm-project/vllm#42752). "none" is also the default for a request without tools. Before, streaming parsed the markup into tool calls with finish_reason="tool_calls". A trailing < is no longer held back for one chunk.
• Whitespace before a tool call (streaming). With automatic tool parsing, content that is only whitespace (for example the line breaks after </think>) is held. If a tool call follows, the whitespace is dropped, as in non-streaming responses. If text follows, it arrives with the first text chunk. The joined content does not change. Chat Completions only.
• Short tool calls in streaming (qwen3_coder). When one chunk carried a whole call body, which is common with multi-token MTP steps, the arguments could end as "{". The complete arguments are now sent. Calls without parameters were affected most. This also fixes Responses streams, which have no end-of-stream flush.
• Non-finite numbers in tool arguments. inf, Infinity, NaN or 1e999 in a number parameter are kept as the original string (vllm-project/vllm#43984). Before, qwen3_coder returned every call of the response as raw content (non-streaming), stopped the stream, or produced invalid JSON.

Anthropic Messages (/v1/messages):

• A forced tool (tool_choice: {"type": "tool"}) now reports stop_reason: "tool_use", not end_turn (vllm-project/vllm#47598). Clients that loop on tool_use now run the tool.
• tool_choice.disable_parallel_tool_use is now honoured. It maps to parallel_tool_calls=false (vllm-project/vllm#52021).
• A named call that was cut short (for example by max_tokens) is left out of the content. Before, the whole response failed with a 500. stop_reason is max_tokens.

Multimodal:

• V100 with FlashAttention-V100 (the ReVolta target) keeps its own default. Without --limit-mm-per-prompt, the engine allows 1 image and no video per request, as in 0.1.0 and 0.1.1. If the value leaves out "video", video is disabled. The startup log says Using SM70 Flash-V100 multimodal default. Set the option explicitly and include "video". To copy:
  • the generic 0.1.2 default: --limit-mm-per-prompt '{"image": 8, "video": 1, "audio": 1}';
  • the reference deployment (used in the README launch command): --limit-mm-per-prompt '{"image": 16, "video": 1, "audio": 16}'.
• On other hardware or attention backends, the default per-request limits are now 8 images, 1 video and 1 audio item (0.1.1: 16, 1 and 16).
• Images in earlier chat messages count toward the image limit. --limit-mm-per-prompt overrides each modality it names.
• Large images are resized. Qwen3.8 Flash-Next resizes an image larger than 1,048,576 pixels (1024 × 1024 in area) before the vision encoder. It does not reject it. HF's smart_resize keeps the aspect ratio and aligns to 32-pixel multiples. This applies on all hardware, including V100. Small text in a large image can become blurry. Crop the region or raise the budget when detail matters.
• Near the area limit, alignment can also change a slightly smaller image. For example, 977 × 1073 becomes 960 × 1056 instead of 992 × 1088.
• Explicit image size settings (size, max_pixels, min_pixels, do_resize, also inside images_kwargs) keep priority. To restore the checkpoint's 16,777,216-pixel budget, use --mm-processor-kwargs '{"images_kwargs": {"size": {"longest_edge": 16777216, "shortest_edge": 65536}}}'. Video sizing does not change.
• Video sampling defaults to 16 frames when num_frames is not set (0.1.1: 32).
• Per-request mm_processor_kwargs and media_io_kwargs that are not empty are rejected with HTTP 400 (vllm-project/vllm#58830). Start the server with --trust-request-mm-kwargs only when the clients are trusted. Omitted values, null and {} work as before. This applies to chat, completions, Responses, render, tokenize, pooling and run-batch. The Python LLM API does not change.
• The startup memory profile uses the new image budget: at most 1,024 image tokens per image instead of 16,384. On the reference setup, the KV pool did not shrink (see the top-k workspace fix below).

Other API changes:

• SSE keep-alive (opt-in). --sse-keep-alive-interval N sends an SSE comment line (: keep-alive) after N idle seconds on streaming /v1/chat/completions, /v1/completions, /v1/responses and /v1/messages responses (vllm-project/vllm#51034). A proxy with a read timeout then keeps the connection open while a request waits in the queue, runs its prefill, or pauses between tokens. SSE clients ignore comment lines. The default is 0 (off). Harmony Responses streams are excluded.
• Claude Code billing header. In Chat Completions, a system text that starts with an x-anthropic-billing-header line loses that line (vllm-project/vllm#59419). Its value changes on every request and defeated the prefix cache. /v1/messages already removed it.
• SM70 AWQ weight reload (reload_weights and weight-transfer updates) now fails before any tensor changes. Dense layers kept stale kernel state, and MoE layers could point at freed memory. Restart with the new weights instead.

Fixes

Memory and KV cache:

• Multimodal encoder cache entries could keep the memory of their whole batch alive, so partial eviction freed nothing. On the reference setup, 9.1 MiB of cache entries held 1,118 MiB of GPU memory. Each entry now owns its memory. Output values do not change.
• With MTP and synchronous scheduling, the draft QSA layer now shares the target's top-k workspace instead of allocating its own. The saved memory goes to the KV pool: on the reference setup it grows from 1,145,881 to 1,153,143 tokens (+7,262).
• With KV offload, a finished request no longer marks shared recurrent-state checkpoints (prefix hits and the shared-prefix boundary) for early eviction. Other requests with the same prefix keep their CPU cache hits.
• With --mamba-cache-mode align and async scheduling (vLLM's default), every prefill chunk kept one superseded recurrent-state block per Mamba cache group until the request ended. These blocks are now freed (from 1Cat-vLLM, by cookys). In an upstream test on 4 × V100, a 253,731-token cold prefill went from 343 s with 6 preemptions to 60 s with none. Synchronous scheduling (--no-async-scheduling, as in the reference setup) is unchanged.

Engine robustness:

• If the PLE offload worker process exits, the GPU worker now stops at once and reports the exit code. Before, the engine could hang, for example in a CUDA graph that waited for a PLE result. An exit during startup also reports the exit code.
• With all PLE rows in host memory, raw host-to-device copies now run on the destination GPU. PLE cascade (ple_disk_cascade=true) is still unsupported in 0.1.x.
• Development mode: /sleep?level=1 could kill the engine. The CUDA runtime lookup matched TileLang's libcudart_stub.so (vllm-project/vllm#47586). Library file names are now matched exactly.
• Sleep levels 1 and 2 now check their prerequisites on every worker first. A failed check returns an error and leaves requests and caches as they were.
• Development mode with KV offload: reset_prefix_cache(reset_running_requests=True) could crash the engine on an assertion. This happened when a request preempted outside the step loop was re-admitted with a fresh CPU-tier load in the same step. Introduced in 0.1.1.

Speculative decoding:

• A native MTP draft with plain RoPE now follows a YaRN-scaled target with the same rope_theta. It uses the target's scaling and context length (from 1CatAI/1Cat-vLLM#1242). A draft that already uses scaled RoPE, or a different rope_theta, keeps its settings.
• If the QSA key ring did not divide the attention block (possible with an explicit --block-size), startup stopped at an assertion. The ring now widens by whole groups, up to 4 times its minimum size (logged). Beyond that, startup fails and names the block multiple to use.

Checkpoints:

• Standard NVFP4 and FP4-Local checkpoints with a missing or unexpected tensor now fail at startup. 0.1.1 checked only AWQ checkpoints.
• Standard NVFP4: if pinning the PLE table in host memory fails, a retry no longer finds a half-built table.

Documentation:

• README: install from PyPI.
• Multimodal input limits: the new defaults, image resizing and per-request kwargs.

New features

DFlash2 speculative decoding (off by default)

ReVolta can use a DFlash2 drafter instead of the native MTP draft. The drafter proposes a block of tokens from the target's hidden states, and the target verifies them in one step.

• Off by default. Without a dflash speculative config, the KV cache groups, block size and MoE routes are the same as in 0.1.1. MTP remains the default and the reference configuration.

• Drafter. incoai/Qwen3.8-Flash-Next-DFlash2 (Apache-2.0, one 1.2 GB model.safetensors), tested revision a22a1744e85c17364dcc54a6a420af7135d4c7f6: 5 sliding-window layers, 8-token block, 2048-token window.

• Download and draft view. Download the tested revision, then make a draft view: a directory that links the weights and has its own config.json with two more settings, "target_feature_reduction": "hc_mean" inside dflash_config and "max_position_embeddings": 524288 (the served context). Startup refuses a drafter that does not declare hc_mean. The hf command comes with the huggingface_hub package that ReVolta installs. The paths follow the README quick start; the README section "DFlash2 speculative decoding (optional)" has the same steps as a Python script.

  hf download incoai/Qwen3.8-Flash-Next-DFlash2 \
    --revision a22a1744e85c17364dcc54a6a420af7135d4c7f6 \
    --local-dir /data/revolta/models/Qwen3.8-Flash-Next-DFlash2
  sha256sum /data/revolta/models/Qwen3.8-Flash-Next-DFlash2/model.safetensors
  # d6723f2a359eaf33ec0c95f22ae8edda896774020c552296a85e1913b4d3186f
  
  mkdir -p /data/revolta/views/dflash2-512k
  ln -s /data/revolta/models/Qwen3.8-Flash-Next-DFlash2/model.safetensors \
    /data/revolta/views/dflash2-512k/
  python3 - <<'PY'
  import json
  src = "/data/revolta/models/Qwen3.8-Flash-Next-DFlash2/config.json"
  dst = "/data/revolta/views/dflash2-512k/config.json"
  config = json.load(open(src))
  config["dflash_config"]["target_feature_reduction"] = "hc_mean"
  config["max_position_embeddings"] = 524288
  json.dump(config, open(dst, "w"), indent=2)
  PY

  With Docker, mount the directory at the same path (-v /data/revolta:/data/revolta) so the link resolves inside the container.

• Recommended config for the reference setup (TP4, DCP2, 512K context, CPU KV offload, --no-async-scheduling). It replaces the MTP --speculative-config, together with two environment variables:

  export VLLM_SM70_DFLASH2_FUSED_QKV_PACK=1
  export VLLM_V2_CUDAGRAPH_MEM_MIB=1500
  
  --speculative-config '{"method": "dflash", "num_speculative_tokens": 7,
    "attention_backend": "FLASH_ATTN_V100", "draft_sample_method": "greedy",
    "model": "/data/revolta/views/dflash2-512k",
    "dflash_concurrency_drafts": {"4": 3, "5": 4},
    "sm70_dflash2": {"context_kv_graph": true, "context_pipeline": true,
      "verify_fastpath": true, "fused_gdn_metadata": true,
      "fused_smallq_metadata": true, "grouped_smallq_metadata": true}}'

  With {"4": 3, "5": 4}, one to three running requests verify all 7 drafts, four verify 3, and five or more verify 4.

• VLLM_SM70_DFLASH2_FUSED_QKV_PACK=1 packs the verifier's Q/K/V rows with one copy kernel. It acts only on SM70 with DFlash2. In tests, greedy output was bit-identical and throughput about 1.5% higher. Like every VLLM_* variable, it is part of the torch.compile key, so a deployment that sets it builds its own compile cache.

• VLLM_V2_CUDAGRAPH_MEM_MIB=1500 reserves GPU memory for DFlash2's CUDA graphs before the KV pool is sized. The ReVolta profile sets 700 MiB for AWQ, which fits MTP4's graphs (0.68 GiB per GPU). DFlash2's graphs take about 1.37–1.39 GiB per GPU. Without this setting, the KV pool takes about 0.7 GiB per GPU too much. Concurrent long prompts that fill a mixed prefill step (up to the 8192-token batch budget), with or without decoding, can then run out of GPU memory. An explicit value overrides the profile; it is also part of the compile key. Alternative: --gpu-memory-utilization 0.93.

• Enable these six sm70_dflash2 options: context_kv_graph, context_pipeline, verify_fastpath, fused_gdn_metadata, fused_smallq_metadata and grouped_smallq_metadata. They are off unless set.

• Do not enable fused_gdn_verify, fused_gdn_norm, fused_gdn_split or fused_gdn_combined_split. They change the numerics, and they lowered sampled acceptance in our tests.

• Leave --block-size unset. The automatic block size fits the drafter cache into the main KV slots: 1664 tokens with 7 drafts at DCP2 (1632 with 4 drafts). An explicit block can leave the drafter only small blocks or its own pages. Startup logs a warning in that case.

• Do not set VLLM_SM70_AWQ_QWEN38_GROUPED_QPN_MAX_TOKENS. With DFlash, the AWQ MoE layers read VLLM_SM70_AWQ_QWEN38_GROUPED_QPN_DFLASH_MAX_TOKENS instead (default 48, the kernel maximum; you do not need to set it). A verify batch above the band runs as at most two grouped calls instead of the slower TurboMind route. The older variable changes only MTP and non-speculative routes.

• Verify width at high concurrency. dflash_concurrency_drafts maps a number of running requests (the threshold) to the number of drafts the target verifies per request (the width). The drafter still drafts 7 tokens, so single-request acceptance does not change.

• The thresholds form a ladder. The entry with the largest threshold that is not above the number of running requests sets the width. Below the lowest threshold, all drafts are verified. A higher entry can make the width wider again, as "5": 4 does after "4": 3.

• CUDA graphs are captured only for the request counts that use each width.

• Thresholds must be 2 or more, and widths 1 to num_speculative_tokens.

• Against {"5": 4}, the ladder {"4": 3, "5": 4} gave identical output at 1 request and 9.7% more throughput at 4 requests (English 219.7 → 261.0, Chinese 172.1 → 209.1, code 368.4 → 363.7 tokens/s). At 5–8 requests both use the same width, and the results were within noise. These are the author's A/B runs on the 0.1.2 code on the reference setup, with English, Chinese and code weighted equally.

• For loads that are mostly Chinese, {"3": 3} verifies 3 drafts from three requests on. At three requests, that was about 15% faster than verifying all 7 in an earlier width sweep. For loads that are mostly code, {"8": 4} measured best at 8 requests.

• Setting dflash_concurrency_drafts turns off async scheduling. --async-scheduling together with it is an error.

• Output. Greedy output with DFlash2 is not bitwise identical to MTP4. Differences start at near-ties: DFlash2 verifies a different number of tokens, and batches of 41 to 96 rows use the grouped MoE route instead of TurboMind.

• Pipeline parallelism is not supported with DFlash2. Startup fails.

Reference numbers, measured with the 0.1.2 release candidate on the reference setup: 4 × V100 32 GB PCIe, TP4, DCP2, 512K context, CPU KV offload, Uncensored AWQ target. DFlash2 used the recommended config above. Greedy decoding, thinking off, 512-token responses. For 1 request: decode tokens/s after the first token, one prompt per category, mean of two runs. For 4 and 8 requests: aggregate tokens/s over mixed prompts, mean of two rounds. With one prompt per category, the per-category values are indicative. This run had no graph reserve: its KV pool of 1,445,845 tokens is not safe (see Known issues). The reserve changes only the KV pool size; the speed rows were measured without it.

	MTP4	DFlash2, 7 drafts
KV cache tokens, with the 1,500 MiB graph reserve	1,153,143	1,268,831 (+10%)
1 request, English code	162.7	200.5
1 request, English reasoning	163.1	195.4
1 request, coding-agent task	158.2	181.1
1 request, Chinese explanation	104.6	100.8
1 request, Chinese chat	99.6	84.8
4 requests, mixed prompts	236.0	237.6
8 requests, mixed prompts	357.2	331.1 (0.93×)

Earlier runs with the 0.1.2 code before the ladder change ({"5": 4}), 8 prompts per category, aggregate tokens/s, no profiling. The 2-request row is one round; the 8-request row is the mean of two rounds.

	MTP4	DFlash2, 7 drafts
2 requests, 7 drafts verified: code / English / Chinese	232 / 193 / 141	275 / 203 / 129
8 requests, 4 drafts verified: code / English / Chinese	492 / 332 / 327	505 / 336 / 285

GSM8K and MBPP samples (12 problems each), 1 request, all 7 drafts verified, on 0.1.1 plus the DFlash2 changes: 206 and 192 tokens/s against 156 and 139 with MTP4. These runs had a profiling fence that added about 2.6 ms per step to both arms; the ratio is still valid.

On a replay of real agent sessions (one request at a time, sampling, long reasoning, contexts up to 128K and beyond; 87 runs weighted by actual depth), DFlash2 and MTP4 were on par: acceptance length 2.93 vs 2.92, speed 0.97× MTP4 (95% interval 0.90–1.01). DFlash2 was ahead on tool-call and answer segments (+6%), contexts up to 64K (+5%) and file-writing turns with reasoning off (+9%). It was behind in reasoning segments (−4%), contexts above 128K (−2 to −3%) and turns with Chinese text (acceptance −5.5%, speed −9%). The short-prompt benchmark gains in the table above therefore do not carry over to long agent sessions as they are; MTP4 stays the default.

• With the release candidate, DFlash2 passed startup, tool calls (thinking on and off), GPU and CPU prefix-cache hits and a 101K-token prompt.
• Chinese is slower than MTP4 in almost every measured case. The drafter was trained on English, and its acceptance on Chinese is lower.
• Very long prompts favour MTP4 (the drafter's window is 2048 tokens).

Deployment notes

The reference deployment of 0.1.2 keeps MTP4 and adds these settings:

• --sse-keep-alive-interval 15: keep-alive comments for streams behind a proxy with a read timeout.
• --limit-mm-per-prompt '{"image": 16, "video": 1, "audio": 16}': keeps the per-request limits the reference deployment had with 0.1.1. Large images are still resized (the new default).
• --structured-outputs-config '{"backend":"xgrammar"}': pins the structured-output backend. With auto, the engine keeps the backend chosen for its first structured-output request (vllm-project/vllm#49625).
• TILELANG_CACHE_DIR=<persistent directory>: keeps TileLang's compiled kernels when the container is re-created. Without it, TileLang compiles them again in every new container.

Other notes:

• On V100 with FlashAttention-V100, always set --limit-mm-per-prompt and include "video" in it, for example '{"image": 8, "video": 1, "audio": 1}' for the generic default. Without the option the engine allows 1 image and no video.
• The first start of 0.1.2 builds a new torch.compile cache, because the model code changed. It takes longer than a restart that reuses the cache.
• Keep --trust-request-mm-kwargs off unless the API clients are trusted.

Known issues and deferred work

• Tool calls with required or a named tool still use a JSON grammar. Upstream vLLM's strict native XML structural tags (default since vllm-project/vllm#45003) are deferred to a separate evaluation. They would change the output of every required and named request.
• The implicit <tool_call> marker is handled in the API layer. Masking it in the engine (vllm-project/vllm#59627, vllm-project/vllm#61574) changes sampling and is deferred to the same evaluation.
• Known tool-calling issues that are not fixed yet: a parameter that uses $ref is treated as a string; a required response that fails to parse still has finish_reason="tool_calls" (vllm-project/vllm#59324); xgrammar allows a stop token inside a JSON string (vllm-project/vllm#49227, vllm-project/vllm#60749); with very large deltas, the last of several parallel calls can be lost (most cases were removed by the short-call fix above).
• A qwen3_coder string argument that contains the literal text </function> is cut at that point (unchanged from 0.1.1).
• Text that contains the literal image placeholder <|image_pad|> can take the place of a real image. Upstream vLLM and HF behave the same (vllm-project/vllm#57740). ReVolta follows them.
• On V100 with FlashAttention-V100, the multimodal default without --limit-mm-per-prompt stays 1 image and no video, not the generic 8 images, 1 video and 1 audio item. This behaviour is kept. Set the option explicitly and include "video" (see Multimodal under Behaviour changes).
• A host memory guard for multimodal bursts is deferred to 0.2. The defaults above are per-request limits, not a process memory limit; concurrent requests add up (#2).
• PLE cascade stays unsupported in 0.1.x. 0.1.2 adds only the fail-fast and host-to-device fixes. Full support (GPU reserve on 32 GB cards, KV estimate, placement modes) is a 0.2 goal.
• With several concurrent requests, outputs are not bitwise reproducible from run to run. This is the same in 0.1.1 and 0.1.2: in the release tests, two runs of the same 0.1.2 build at 4 and 8 requests also diverged at near-ties. The cause is the order of the tensor-parallel all-reduce sums, which depends on how requests are batched. A batch-invariant all-reduce is a 0.2 goal.
• Sleep level 2 (development mode) is not supported on the reference setup (128 GiB host, PLE tables in host memory): it needs about 87 GiB of free host memory for the model buffers while about 63 GiB is available. Sleep level 1 works.
• Update after release: the 8-request throughput gap on NVLink reported in #2 did not reproduce in later runs. On that system the CPU KV pool size moves the 8-request number by about 3% (see the README).
• DFlash2 without VLLM_V2_CUDAGRAPH_MEM_MIB=1500 can run out of GPU memory. The ReVolta profile reserves 700 MiB per GPU for CUDA graphs, sized for MTP4. DFlash2's graphs take about 1.37–1.39 GiB, so the KV pool gets about 0.7 GiB per GPU too much. The trigger is concurrent long prompts that fill a mixed prefill step (up to the 8192-token batch budget), with or without decoding. On the reference setup this happened with one DFlash2 decode plus two long prefills, and with four long prompts and no decoding, where the first such step ran out of memory. Workaround: set VLLM_V2_CUDAGRAPH_MEM_MIB=1500, as in the recommended config, or use --gpu-memory-utilization 0.93. The DFlash2 KV pool is then 1,268,831 tokens on the reference setup, about 10% more than MTP4's 1,153,143, instead of 1,445,845. MTP4 is not affected: its reserve and its actual graph memory are both 0.68 GiB. A code change that sizes the reserve per speculative method is planned for the next release.
• DFlash2: the first request after an empty compile cache compiles the DFlash and QSA verify kernels (there is no DFlash-shape warm-up yet). The verify width follows the number of running requests, not the content; a width that adapts per request is planned. Not yet tested on GPU with DFlash2: images and video, preemption, INT8 KV and long-running service.

See also README, Support and known limits.

Release gate

All results come from the reference machine (4 × V100 32 GB PCIe behind one PCIe switch). Baselines were measured with 0.1.1 in the same window.

The candidate was the 0.1.2 Python code on the 0.1.1 image, before the reset-during-reload fix was added (its own row below); the native extensions have no changes. The reference deployment's 0.1.2 settings were used (see Deployment notes).

Check	Result
Greedy output and C1 fixed-seed sampling vs 0.1.1 (DFlash off)	Pass: bitwise identical (C1 sampling 8 of 8)
Outputs at 4 and 8 concurrent requests	Not bitwise reproducible run to run, on either version: two runs of the same candidate also diverged at near-ties
Full AWQ gate (4/8-request stress, multimodal stress, offload readback, 211K retrieval, 512K boundary)	Pass. The gate's own C2 speed sample (0.967) was reviewed: C2 is bimodal (whether the two requests share their first prefill step), and the speed run below shows no change
Speed vs 0.1.1 (512-token responses, C1/C2/C4/C8)	Throughput ratio 1.005 / 1.002 / 1.002 / 0.996
KV pool size vs 0.1.1 (1,145,881 tokens)	1,153,143 tokens (+7,262)
Tool-calling matrix (required / named / auto × streaming × thinking)	Pass after review, no regression. Two checks failed on test expectations: streamed arguments keep whitespace (also in 0.1.1 and upstream vLLM), and the joint-grammar probe expected one call where the model correctly made two parallel calls. The 0.1.1 streaming whitespace difference and the Anthropic truncated-call 500 are fixed
SSE keep-alive	Pass on all four endpoints. One check failed because its raw /v1/completions test prompt stopped after one token; keep-alive itself worked
Multimodal defaults and image resizing	Pass. Large images are resized; with the 16-megapixel budget restored, image prompt tokens match 0.1.1 exactly. With the reference limits, 16 images are accepted and 17 rejected. The tests also found the V100 default of 1 image (see Known issues)
Encoder cache memory	Pass: cache entries own their memory
Small KV pool (630,307 tokens): shared-prefix recurrent state, eviction after preemption, image prefix read back from CPU	Pass
Development mode: sleep levels 0 and 1, sleep preflight, pause modes, pause during a pending KV load, transfers in flight	Pass; output before and after identical. Failed on the candidate: prefix-cache reset during a CPU-to-GPU reload with running requests (also in 0.1.1; fixed in 0.1.2, next row). Not run: sleep level 2 (host memory). Inconclusive: disk-tier round trip
Development mode with KV offload: prefix-cache reset during a CPU-to-GPU reload, with the fix	Pass on the release image: the engine stays healthy and keeps serving through resets of running requests during a reload (two needle answers in this case were also wrong with a full recompute and no reset: a model retrieval error on that document)
GPU unit tests: top-k workspace under compute-sanitizer, NVFP4 PLE retry, host-to-device copy context, PLE worker exit inside a CUDA graph, 4-GPU PLE tests	Pass (0.1.1 fails the host-to-device and worker-exit probes, as expected)
DFlash2 with the recommended config, on the release image	Pass: startup, tool calls (thinking on and off), GPU and CPU prefix-cache hits, a 101K-token prompt, and the out-of-memory triggers from Known issues (one decode plus two long prefills, twice; four prompts of 38K to 40K tokens at once): all requests succeeded, no out-of-memory error. KV pool 1,268,831 tokens (block 1664). Speed vs MTP4 on the same image, 512-token responses: 1 request 1.15× to 1.23× for code and English, 0.85× and 0.96× for Chinese; 4 and 8 requests 0.95× and 0.91×
Native MTP draft RoPE following a YaRN target	Not run: the test view lacked its calibrated KV scales and was refused at startup (test setup)
FP4-Local and standard NVFP4 smoke	Not run for 0.1.2

Assets

File	Content
revolta-0.1.2-cp312-cp312-linux_x86_64.whl	Wheel (Linux x86-64, Python 3.12, CUDA 12.8; debug symbols stripped), 79,251,371 bytes, sha256 96a31cc9e20e335226ad80ace1a3c62a15423c164408336bf78b7cecea9c314a
revolta-0.1.2-cp312-cp312-linux_x86_64.debug.tar.xz	Separate debug symbols for the native extensions, sha256 2546aa36d69ba4b4e1b2203225d0d01ea06d7a8536741fa721efc318b625d461
SHA256SUMS	Checksums of the files above

Docker image: ghcr.io/leonccaa/revolta:0.1.2 (digest sha256:a367af30016922bc272e4efa2fa4b44993a42e5daba794566a367f77ab539634).

PyPI: pip install revolta==0.1.2. The PyPI file is revolta-0.1.2-cp312-cp312-manylinux_2_34_x86_64.whl, sha256 ab6f06605cfe76e02d4f83ccca3e1da39f39ed507e02d76a176b490133ecda71. It contains the same files as the wheel above. Only the metadata differs (METADATA, WHEEL, RECORD): plain version pins for torch==2.10.0, torchaudio==2.10.0 and torchvision==0.25.0 instead of download.pytorch.org URLs, and the manylinux_2_34_x86_64 platform tag. Both wheels need glibc 2.34 or newer.

Thanks

Thanks to jttdev for the multimodal memory report in #2, which led to the image resizing and the new multimodal defaults. Thanks to the authors of the upstream changes ported here: Juan Pérez de Algaba (vllm-project/vllm#58830), Dieter Schwarzmann (vllm-project/vllm#51034), Maroon Ayoub and Robert Shaw (vllm-project/vllm#59419), Zihua Wu (vllm-project/vllm#47586), the authors of the other vLLM changes listed above, and the 1Cat-vLLM contributors of 1CatAI/1Cat-vLLM#653 and 1CatAI/1Cat-vLLM#1242. Thanks to incoai for the DFlash2 drafter.