v2.0: a 2.8 bpw DeepSeek-V4.1-Flash-1017 build and a faster engine

v2.0 ships a new DeepSeek-V4.1-Flash-1017 build, the closest to native 1017 of the five 1017 packs we measured, and a new engine that serves it faster than TensorFold on code and structured output on the same two DGX Sparks. The engine also makes v1.2's 2.7 bpw build faster, ahead of TensorFold on all three prompts.

Weights: sudoingx/DeepSeek-V4.1-Flash-1017-EXL3-2.8bpw, 204 GB, plus 1017's two Engram shards from DeepSeek's repo (the same ones v1.2 uses). Experts at 2.80 bpw with a cap of 3.10 bpw per decoded token, dense linears at K5 with wq_b at K4, the head at K5, the drafter with its own K4 head, and a draft-depth model fitted to the pack.

Quality against native 1017, scored by DeepSeek's reference implementation with each pack's own tensors loaded:

• KLD 0.0198 on our panel and 0.0213 on a second panel of native 1017 replies, the lowest of the five 1017 packs we measured: HeNryous 2.75 bpw 0.0201 / 0.0216, object-oriented-architects 2.7 bpw 0.0232 / 0.0249, our v1.2 0.0243 / 0.0262, LibertAIDAI 2.9 bpw 0.0304 / 0.0322. Top-1 agreement 95.94%, level with HeNryous's 95.96%.
• HumanEval+ 97.0 / 89.6, MBPP+ 90.7 / 77.8, IFEval prompt strict 90.8, each within noise of native 1017 (paired p 0.61 to 1.00).
• On 300 IFEval prompts with native replies HeNryous's pack is a little closer: 0.0974 against 0.0980.

Speed, kit_bench.py set b, single stream, DSpark on, median of 9 reps a prompt, against TensorFold with object-oriented-architects' 1017 2.7 bpw pack on the same boxes with the same client:

• The 2.8 bpw build, one session with alternating fresh boots: code 103.8 vs 98.5 tok/s (+5.3%), structured 144.1 vs 141.1 (+2.1%), prose 57.8 vs 58.2 (-0.6%). A second session (the release gates): 103.7 vs 97.7 (+6.1%), 144.4 vs 140.4 (+2.8%), 57.8 vs 57.5 (+0.5%).
• v1.2's 2.7 bpw build on the v2.0 engine, the second session: 106.4 / 61.8 / 149.1 tok/s, +8.9% / +7.5% / +6.2% over TensorFold (v1.2's own engine: 100.7 / 59.2 / 141.3).
• Against TensorFold's newest DeepSeek TP2 commit (af26d42b, its v0.6 release plus two fixes), a third session with alternating fresh boots: code 103.9 vs 98.4 (+5.6%), structured 143.4 vs 140.5 (+2.0%), prose 57.8 vs 57.9 (-0.3%); v1.2's build 106.2 / 62.0 / 147.6 (+7.9% / +7.0% / +5.0%). The first two sessions used 8e0966e2, the commit lucataco's 1017 recipe pins.

Engine (image dsv41-serve:v5): the draft depth from the confidence head plus each draft's logit margin and top-1 probability through a small model fitted to the pack; the drafter's own head; greedy verify from a vocabulary-shard argmax; Engram's projection row-parallel on the RDMA ring; a row GEMM for the attention inputs; five drafter-phase changes; every SWA metadata build of a decode step in one launch; the critical-first L2 prefetch order; RoPE tables and the indexer's prefill buffer sized to the serve (1.4 GiB a rank, after HeNryous's patches). Patches 0023-0038.

Upgrading from v1.x: stop the serve (scripts/launch/dsv41_tp2.sh down), run git pull and ./install.sh (images only, the wheels are kept), download the 2.8 bpw weights and up with MODEL_DIR on the new folder; the launcher's defaults are this build's. v1.2's weights on v2.0: DOCKER_ENV="VLLM_DSPARK_ROUND_MS=27.3,31.3,34.7,38.1,41.1,43.8".

Also in v2.0:

• data/: the plan, KLD on three panels for five 1017 packs, the suites, the same-box and gate files, the round-cost table, the depth model's log, and the native-1017 panel.
• FINDINGS 24-29: compare engines in one session, where the confidence head is weak, targeted bit moves cost several times their predictions, small kernels under PDL, an L2 discard that changed outputs.
• scripts/dspark/: the depth model's fitter.

Limits: after load the 2.8 bpw build leaves 6 / 6 GiB free with vision on, 4 / 4 after a 128K prompt; the 256K needle rung has not run on it (needles pass at 8K, 32K, 64K and 128K). The 2-hour agent soak has not run.

v1.2: DeepSeek-V4.1-Flash-1017

DeepSeek's Oct 17 update, deepseek-ai/DeepSeek-V4.1-Flash-1017, keeps the launch checkpoint's config, tokenizer and tensor layout and changes the content of every shard. v1.2 re-runs the whole pipeline on it and makes the 1017 build the default. The engine and the serving image are the same as v1.1.

Weights: sudoingx/DeepSeek-V4.1-Flash-1017-EXL3-2.7bpw, 196 GB, plus 1017's own two Engram shards from DeepSeek's repo (the launch checkpoint's shards differ). The launch build stays at sudoingx/DeepSeek-V4.1-Flash-EXL3-2.7bpw, served by the same engine.

Quality against native 1017, on the same panel and reference implementation as the launch build:

• KLD 0.0243, top-1 agreement 95.47%. LibertAIDAI's 1017 EXL3 2.9 bpw pack scores 0.0304 / 94.76% there.
• HumanEval+ 96.3 / 90.2, MBPP+ 89.7 / 76.2, IFEval prompt strict 89.1, each within noise of native 1017 (paired, p >= 0.14).
• On 300 IFEval prompts with native replies, LibertAIDAI's pack is closer to native: KLD 0.1135 against 0.1199.

Speed, kit_bench.py set b, single stream, DSpark on, median of 9 reps a prompt:

• 100.7 / 59.2 / 141.3 tok/s on code / prose / structured; the launch build read 103.0 / 58.0 / 142.6 tok/s in the same session.
• Against TensorFold 8e0966e2 with the object-oriented-architects 1017 EXL3 2.7 bpw pack, on the same two Sparks with the same client, two boots each: code +2.0%, prose +0.9%, structured level.

Switching from v1.1: download the 1017 weights and 1017's Engram shards (quickstart step 3), then run up with MODEL_DIR on the new folder and DOCKER_ENV="VLLM_DSPARK_ROUND_MS=28.99,32.76,36.11,39.34,42.43,45.06", the depth table measured on 1017. No image rebuild.

Also in v1.2:

• data/: the 1017 plan, statistics, KLD, eval, set b and gate files, the same-box run, and two heavier dense / head compositions on the same experts (KLD 0.0227 and 0.0219, 2.4-7.8% slower than this build in one session; data/speed/variants_1017).
• FINDINGS 22 (memory fragmentation after days of uptime) and 23 (sensitivity across checkpoints).
• guards.sh is executable (#1, reported by @eduffd).

Not run on the 1017 build yet: the 256K needle rung (needles pass at 8K, 32K, 64K and 128K; the launch build passed 256K). The 2-hour agent soak has not run on either build.