Speedy 0.3.1

Qwen3.8-Flash-Next (125B MoE) on one or two consumer RTX cards. Fixes for long sessions and long prompts, tool-call parsing fixes, and a new ready-made engine for RTX 20 (Turing) cards.

Downloads

File	For
strata-linux-x64-rtx20.zip / strata-windows-x64-rtx20.zip	New: RTX 20 series (sm_75: 2060, 2070, 2080, 2080 Ti, Titan RTX), built, not tested yet
strata-linux-x64-rtx30.zip / strata-windows-x64-rtx30.zip	RTX 30 series (sm_86), tested on 2x RTX 3080
strata-linux-x64-rtx40.zip / strata-windows-x64-rtx40.zip	RTX 40 series (sm_89), built, not tested yet
strata-linux-x64-rtx50.zip / strata-windows-x64-rtx50.zip	RTX 50 series (sm_120), built, not tested yet

CUDA 13, NVIDIA driver 580+. python setup.py picks the zip for your card automatically, now including RTX 20 cards.

RTX 20 note: Turing has no BF16/TF32 tensor cores, so a few kernels use fp32 fallbacks and prompt reads are slower than on RTX 30. Most RTX 20 cards have 8 GB; 11-12 GB or more is recommended. Reports from RTX 20/40/50 owners are welcome.

Fixes

• Long prompts on the layer-major prefill path (#4, thanks @WilliamStone): with STRATA_PF_GROUP_SCRATCH=1, a large routed expert group could overflow the MoE scratch and turn the layer into NaN rows, stopping the engine on prompts of ~29K tokens and up (HTTP 503). Oversized groups are now split. Prefill speed unchanged.
• Long sessions slowly losing their expert cache (upstream #4844): under --pipeline-windows 2 with --adapt-async, a stale copy-back range made the engine read experts from the model file again after warm-up. Now 0 file reads after 190K exchanges.
• Split-window DMA ordering (upstream #5096): an empty group no longer overtakes the earlier group's uploads.
• Expert cache auto keeps the ~256 MiB the allocator needs after it (#4474). On 10 GB cards set --vram-reserve-mib lower (we use 444) to keep the same slot count.
• The engine waits up to 20 s for RAM to be given back by an earlier engine instead of failing (#4051); clearer memlock / PLE lock messages (#4671, #4846).
• Layer-major prefill copies an expert with no mapped blob from its source instead of ending the request (#4695).

Tool calls and the server

• Tool-call recovery is on by default: damaged arguments, a stray closing think tag or a missing </function> no longer lose the call (#4427, #4077); "tool_call_recovery": false turns it off.
• A tool the conversation already called counts as declared on all routes (#4663); tools.write / functions.write / default_api.write map to the declared tool (#4751).
• Calls shown inside code fences or quotes are not run (#4702, #4703); a file that lists closing tags is written whole (#4701); quoted tool-call text in a string argument stays exact (#4078).
• /v1/messages context overflow uses Anthropic's wording, so Claude Code compacts instead of retrying (#4838).
• Repeat / frequency / presence penalties count the reply's tokens, not the end of the prompt (#4590; STRATA_PENALTY_PROMPT=1 restores the old window).

Performance

• Hyper-connection read kernels for 5-8 token launches: decode at c=8 +5-7 %, same bits.
• Opt-in tensor-core GDN scan on 99 KB cards (STRATA_GDN_TC, STRATA_GDN_CONVL2=2), pinned per preset for 2x RTX 3080.

Build

• HIP build fixes (vmm_chunk_new, alignas on int4 arrays); CI builds HIP by hand.

Speedy 0.3.0

Qwen3.8-Flash-Next (125B MoE) up to 1M context on one or two RTX 3080 10 GB. 2× 3080: up to 4,270 tok/s prefill, ~190 tok/s decode (Q2_0).

Downloads

File	For
strata-linux-x64-rtx30.zip / strata-windows-x64-rtx30.zip	RTX 30 series (sm_86), tested on 1x and 2x RTX 3080
strata-linux-x64-rtx40.zip / strata-windows-x64-rtx40.zip	RTX 40 series (sm_89), built, not tested yet
strata-linux-x64-rtx50.zip / strata-windows-x64-rtx50.zip	RTX 50 series (sm_120), built, not tested yet

CUDA 13, NVIDIA driver 580+. python setup.py picks the zip for your card automatically (and checks the SHA-256 GitHub publishes).

Or unzip it and point a preset's "exe" at it. The RTX 30 release engine measured the same as our local build (Q2_0: 4,187-4,211 tok/s prefill, 188-202 decode on 2x 3080).

What's new

• One codebase for one or two GPUs. The single-GPU edition (1GPU 0.2.0) and the dual-GPU edition are merged. A
  preset with --layer-split runs two cards; without it the engine runs on one. Production presets for both are in
  configs/cards/rtx3080-10gb/1x/ and 2x/, including the 1M tier on two GPUs.

• #3559 stage-local HC_Q8 upload (havok2-htwo): on a trimmed split each card uploads only its own Q8_0 rows.
  This gives UD-IQ4_XS and UD-Q4_K_XL +7-11 % expert slots per card (292 / 345 MiB saved).

• STRATA_PREFILL_STREAM_MIN=512 on UD-IQ4_XS (two GPUs): follow-up turns of 730 tokens take 2.15 -> 1.86 s, and 958 tokens 1.94 -> 1.77 s.

• "parallel": 2 presets in configs/cards/rtx3080-10gb/2x/parallel2/. These are measured, not the default: slower on
  2x 10 GB (two clients 42-48 s against 33 s queued; single-user decode -15-19 %).

• Evals: evals/ has the IFBench and LiveCodeBench harness. UD-Q4_K_XL scores 81.0 on IFBench (card BF16: 81.3).

• Releases and updates now point at DevDock-AI/Speedy.

• Rejected after measurement: STRATA_MMQ_DIRECT (neutral), #2427 and split QSA attention (tolerance gate),
  EXP_MODE=8, HELP_FRAC 0.45, NT=256, Talus union-KV, StrataFast-style prompt split.

• Per-card configs: configs/cards/<card>/{1x,2x} with CARD.md, a template for new cards and a validator in CI.