Skip to content

Speedy 0.3.0

Choose a tag to compare

View all tags
@DevDock-AI DevDock-AI released this 01 Nov 19:05
· 53 commits to main since this release
v0.3.0
54638bc

Qwen3.8-Flash-Next (125B MoE) up to 1M context on one or two RTX 3080 10 GB. 2× 3080: up to 4,270 tok/s prefill, ~190 tok/s decode (Q2_0).

Downloads

File For
strata-linux-x64-rtx30.zip / strata-windows-x64-rtx30.zip RTX 30 series (sm_86), tested on 1x and 2x RTX 3080
strata-linux-x64-rtx40.zip / strata-windows-x64-rtx40.zip RTX 40 series (sm_89), built, not tested yet
strata-linux-x64-rtx50.zip / strata-windows-x64-rtx50.zip RTX 50 series (sm_120), built, not tested yet

CUDA 13, NVIDIA driver 580+. python setup.py picks the zip for your card automatically (and checks the SHA-256 GitHub publishes).

Or unzip it and point a preset's "exe" at it. The RTX 30 release engine measured the same as our local build (Q2_0: 4,187-4,211 tok/s prefill, 188-202 decode on 2x 3080).

What's new

  • One codebase for one or two GPUs. The single-GPU edition (1GPU 0.2.0) and the dual-GPU edition are merged. A
    preset with --layer-split runs two cards; without it the engine runs on one. Production presets for both are in
    configs/cards/rtx3080-10gb/1x/ and 2x/, including the 1M tier on two GPUs.

  • #3559 stage-local HC_Q8 upload (havok2-htwo): on a trimmed split each card uploads only its own Q8_0 rows.
    This gives UD-IQ4_XS and UD-Q4_K_XL +7-11 % expert slots per card (292 / 345 MiB saved).

  • STRATA_PREFILL_STREAM_MIN=512 on UD-IQ4_XS (two GPUs): follow-up turns of 730 tokens take 2.15 -> 1.86 s, and 958 tokens 1.94 -> 1.77 s.

  • "parallel": 2 presets in configs/cards/rtx3080-10gb/2x/parallel2/. These are measured, not the default: slower on
    2x 10 GB (two clients 42-48 s against 33 s queued; single-user decode -15-19 %).

  • Evals: evals/ has the IFBench and LiveCodeBench harness. UD-Q4_K_XL scores 81.0 on IFBench (card BF16: 81.3).

  • Releases and updates now point at DevDock-AI/Speedy.

  • Rejected after measurement: STRATA_MMQ_DIRECT (neutral), #2427 and split QSA attention (tolerance gate),
    EXP_MODE=8, HELP_FRAC 0.45, NT=256, Talus union-KV, StrataFast-style prompt split.

  • Per-card configs: configs/cards/<card>/{1x,2x} with CARD.md, a template for new cards and a validator in CI.

Assets 8
Loading