DS41RT v15

DS41RT v15

V15 restores dSpark speculation for requests that offer tools and adds a live engine console. Any request with tools used to verify no drafts in its free text (prose, answers, reasoning after the think block), because grammar truncation treated "the grammar could end here" as "stop drafting"; agent turns decoded at about half speed. Drafts now stop only where the grammar rejects a token. Opening the API port in a browser shows the new console: rates, lane occupancy, KV and host-offload use, a per-round ticker of accepted and rejected drafts, prefill, pipeline micro-step timings and dSpark acceptance, streamed over a WebSocket at no measurable cost. Official checkpoint deepseek-ai/DeepSeek-V4.1-Flash (dba1be0a40aa45a94ad051997016db3960a90277), four Spark workers.

Image	Architecture	Registry manifest digest
ghcr.io/tpurtell/ds41rt-coordinator:v15	amd64	sha256:edc93580ac95bf6c33f97f430fc442761a23fa7aeaca394515384552c620d62a
ghcr.io/tpurtell/ds41rt-spark-expert:v15	arm64	sha256:92e8a70dca41787da332aa567078769f605220dadcff5bb1ff630494d5804988

latest resolves to the same pair. Image source: bd06bec42e219a65097235dd9092e4a0537d4b7a (SparkInfer 7fcc094; native code identical to v14). The tag adds measurements, documentation and promoted defaults.

Tool-enabled decode, two RTX, greedy, tokens/s (v14 code path → v15):

Request	C1	C4
Tools offered, thinking off	54–57 → 98–111	142 → 210
Tools offered, thinking on	60–63 → 101–109	154 → 220
No tools (control)	104–109 → 89–109	216 → 207

On the published images, tool-enabled requests match the no-tools control on both layouts. Schema-JSON dSpark decode, remeasured with v14's prompts: 138.74 (1 RTX) and 148.66 (2 RTX); a fresh v14 control measured 147.85 on two RTX, so v14's 159.10 was one high sample. The console costs nothing measurable (C1 174.1 v14, 173.8 v15, 175.3 with a viewer). Tool evaluation on the v15 images: 155/176. The text view of the console is off unless the server runs with DS41RT_CONSOLE_TEXT=1.

v15-evidence.tar.gz (SHA-256 6d36416f901197d6f42ea53e0450f038d8c0a882f1f446afb85ae0cd3705a107) holds the focused campaign on the final images, the v14 control, publication records, the build log and the development-session measurements. SHA256SUMS verifies its 93 payload files; v15-release-assets.sha256 verifies the tarball and payload manifest. See the release notes.

DS41RT v14

DS41RT v14

V14 shortens the verification cycle, the time from one committed token block to the next, without changing the checkpoint, precision or output contract. Verification passes order their stages on the GPU with CUDA events instead of draining each stage on the host, on one and two RTX. The window-KV and compressor producers overlap the query projections. Two RTX gains a device sampler, TP2 per-token route sums, and device-timed dSpark policy costs. Target argmax, router, draft attention and the decode route planner use faster kernels with identical results. Official checkpoint deepseek-ai/DeepSeek-V4.1-Flash (dba1be0a40aa45a94ad051997016db3960a90277), four Spark workers.

Image	Architecture	Registry manifest digest
ghcr.io/tpurtell/ds41rt-coordinator:v14	amd64	sha256:c49493737243adb693023d02d0dabbe0b64c56668ada7e55b3f70657811bf75e
ghcr.io/tpurtell/ds41rt-spark-expert:v14	arm64	sha256:41d6cd3f66ac492a6e5d5318b9ae81c309ae16430a5ae5deba62f14a955f5b4c

latest resolves to the same pair. Image source: af9d96923f70af1cd385960e28450e5ec0653ff8 (SparkInfer 7fcc094). The tag adds measurements, documentation and promoted defaults.

Release decode (C1 dSpark, tokens/s):

Measurement	1 RTX	2 RTX
Weighted decode	104.69	129.67
C1 code decode	146.12	179.86
Counting decode	182.39	219.86

Against v13 in interleaved A/B sessions (four per arm per layout), v14 is faster on every family on both layouts: greedy C1 +16.6% (1 RTX) and +18.2% (2 RTX), sampled C1 +16.4% and +37.7%, code C1 +12.3% and +13.3%, topic C1 +16.4% and +18.5%, and code, topic and mixed C2–C16 +6–17%. The only key that did not improve is two-RTX code at C16 (−1.0%). No session had a quality failure. Target-only decode rises from about 47–50 to 60 tok/s.

v14-evidence.tar.gz (SHA-256 9303c2360dae7b2a6d838d13ac785fc2e1db989857b0da749a7ba3fe52f32273) holds every A/B session and analysis, the release campaign, publication records, the final build log and the cycle-time program's tooling and checklist. SHA256SUMS verifies its 443 payload files; v14-release-assets.sha256 verifies the tarball and payload manifest. See the release notes and the performance report.

DS41RT v13

DS41RT v13

V13 replaces every adaptive dSpark length selector with one online bandwidth-balance policy. Each round it chooses how many drafted tokens each request verifies by maximizing expected committed tokens per unit of predicted round time, pricing routed-expert weight traffic over an effective bandwidth fitted continuously from the lane's own layer timings. It needs no offline calibration, so quantization, TP width, clock and thermal changes are absorbed at runtime; the drafter's confidence is recalibrated per position online. Official checkpoint deepseek-ai/DeepSeek-V4.1-Flash (dba1be0a40aa45a94ad051997016db3960a90277), four Spark workers.

Image	Architecture	Registry manifest digest
ghcr.io/tpurtell/ds41rt-coordinator:v13	amd64	sha256:f0a7c245be0224ee6d472bc57c6e42eb578ba6ba8b5b2195afe558937c5282de
ghcr.io/tpurtell/ds41rt-spark-expert:v13	arm64	sha256:42d867515f16d37794127477d16d3d2de9cc40e0e6bac6cade4a784548a9ba86

latest resolves to the same pair. Immutable image build source: 00b25e7025410cdb1dadc7960b8e94ae9b89db48. The annotated tag's checkout adds measurements, documentation and promoted defaults; it is not claimed byte-identical to the image source.

Release decode (C1 dSpark, tokens/s):

Measurement	1 RTX	2 RTX
Weighted decode	91.10	108.91
C1 code decode	134.01	161.20
Counting decode	159.35	192.56

Against v12 in interleaved A/B sessions (four per arm per layout): on two RTX, +2–5% on per-case greedy and sampled C1 decode, topic C1, and code, topic and mixed C2–C16, with weighted batteries tied; on one RTX, +2–3% on per-case greedy and sampled C1 decode and ties elsewhere. Measured losses: topic C1 on one RTX (−4.8%), code C1 on two RTX (−2.1%), and two-RTX counting (about 193 versus 221), because dSpark now drafts its trained five-token block by default on both layouts. --dspark-draft-limit 7 restores the seven-token block with per-round 5/7 choice (opt-in; it measured 2–5% slower on code concurrency). No session had a quality failure.

Removed: the placement cost profiles, DS41RT_ADAPTIVE_COST_*, --dspark-confidence-cutoff, --dspark-reuse-floor and the offline fit scripts. /v1/stats now reports the policy's fitted costs, prediction error, draft-length histogram and per-position confidence reliability under dspark_policy.

v13-evidence.tar.gz (SHA-256 97f320a60a79d8f9acce105823cdb6091ffc5cca0ad21bb81908e504c9a4531d) holds every A/B session and analysis, the release campaign, publication records and build logs. SHA256SUMS verifies its 1,170 payload files; v13-release-assets.sha256 verifies the tarball and payload manifest. See the release notes, the performance report and the policy design.

DS41RT v12

DS41RT v12

V12 serves stochastic target-token sampling on the GPU for ordinary and constrained decoding. The public controls remain temperature, top_p, top_k, min_p, and deterministic seed. Greedy, min-p, top-k, and nucleus paths are qualified on the official deepseek-ai/DeepSeek-V4.1-Flash checkpoint (dba1be0a40aa45a94ad051997016db3960a90277).

Image	Architecture	Registry manifest digest
ghcr.io/tpurtell/ds41rt-coordinator:v12	amd64	sha256:ea80ce9f313d9e708d34007cc21d4cc2c09b051936e94905fa6f7b7a19cd6814
ghcr.io/tpurtell/ds41rt-spark-expert:v12	arm64	sha256:d28bd3fe888e691378e85a88a9b2af1292ec6c4b6b2c7cb3ca89d3d50e6bec71

latest resolves to the same pair. Immutable image build source: b23a13011020286d5e1f9d7477c6f6c71ab637cc. The annotated tag's checkout adds measurements, documentation and promoted defaults; it is not claimed byte-identical to the image source.

The 1 RTX + 4 Spark dSpark campaign passed its validator on all 15 raw reports. Weighted medians of three interleaved repeats, in observed decode tokens/s:

Mode	v12	v11 paired output
Greedy	94.17	95.46
T0.2, top-p 0.95	89.65	80.58
T0.7, top-p 0.9	89.73	83.29
T0.7, min-p 0.05	90.91	88.99
T0.7, top-k 40	89.64	87.74

All 150 paired request bodies, response hashes and completion-token counts match the v11 final campaign. The live five-vector sampling qualifier passed 92 checks with zero failures, including constrained JSON and strict tool calls. The native sampler selftest passed 401 cases and 43,648 assertions. Synthetic seeded CPU/GPU floating-point residuals are measured per cell in the performance report; they are not hidden as bitwise parity.

v12-evidence.tar.gz (SHA-256 ace298b38b684e84d332e0c29f5d85a02e1c72b745c16fb737d89b50e3b9c29b) includes the raw campaigns, qualifiers, profiler results, build/image checks and publication records. SHA256SUMS verifies its 104 payload files; v12-release-assets.sha256 verifies the tarball and payload manifest. Full provenance and limitations are in the release notes.

DS41RT v11

DS41RT v11

Serves the official native deepseek-ai/DeepSeek-V4.1-Flash (revision
dba1be0a40aa45a94ad051997016db3960a90277) on one RTX PRO 6000 plus four DGX
Spark workers with dSpark speculative decoding and FP4 KV.

Images

Role	Tag	Registry manifest digest
Coordinator (linux/amd64)	ghcr.io/tpurtell/ds41rt-coordinator:v11	sha256:e0e5d631a54e84f36cd1cd99e2a7cf4e2092c15919c8c79403007ba0852d0d89
Spark expert (linux/arm64)	ghcr.io/tpurtell/ds41rt-spark-expert:v11	sha256:fc50ac1cf7f309727d406962168fbe278807848566ae84abde0709b9ca5ba27a

latest resolves to the same pair. Immutable image build source
fb5115466a8c70c063280e25957577f284e903e3; the tag checkout is not claimed
byte-identical to it. The runtime default is promoted to :v11.

Measured performance (tok/s, weighted median of three repeats)

Profile	Baseline 9d9b3e0	Final fb51154
greedy	94.41	95.46
temp0.2 + top_p0.95	47.35	80.58
temp0.7 + top_p0.9	47.52	83.29
temp0.7 + min_p0.05	88.26	88.99
temp0.7 + top_k40	47.14	87.74

Only greedy is clearly first; the four stochastic modes form one cluster and are
not ranked. Weighted token counts and per-case completion tokens are identical
between the two datasets, so the gain is reduced elapsed time. Both campaigns pass
the validator with zero failures. A baseline --no-dspark control found no
profile faster without dSpark. Full detail, the optimization record, the control
decomposition and the rejection-sampling decision: docs/release-v11-performance.md.

Evidence

v11-evidence.tar.gz (sha256 4e7058c41bec7b433128526e7259edc14b1cfed52a43b284f1edff34672c6629)
packages the raw campaign reports for the final and baseline datasets, the
no-dSpark control, the dSpark debug traces, the re-qualification artifacts, the
validator and aggregator outputs, the build and source-gate logs, and the
publication records. SHA256SUMS verifies every member.

Known limits

Bitwise speculative/non-speculative parity is not claimed, matching the upstream
non-guarantee; the optimization is not faster on every input (fully-tied rows use
the radix branch and can be up to ~1.5x slower than the old structured sort while
still far below baseline). See docs/release-v11-performance.md section 7.

DS41RT v10

DS41RT v10 release notes

STATUS: PUBLISHED. The v10 images are built, published to GHCR as v10 and
latest, and each role was pulled anonymously with a fresh DOCKER_CONFIG and
no host credentials, returning the digests stated below. Model weights are
host-mounted and are never in the images.

What v10 changes

• Promotes the runtime default to the v10 pair. ds41rt.config, the five
  older native examples/configs/* files and the README.md pull/pair lines name
  :v10. The two v10 TP3 profiles (tp3ep1-native.config,
  exl3-compact-tp3.config) were already at :v10 and are no longer exempt from
  published-pair equality.
• Does not switch the checkpoint. v10 serves the same native
  deepseek-ai/DeepSeek-V4.1-Flash revision; the release change is the image pair
  and the runtime default that names it, not a new model or quantization.
• Keeps the universal Spark role set. The Spark image advertises
  io.ds41rt.v41.spark_tp_roles=tp2;tp3;tp6 on top of the default TP4 shard, so
  one pair serves every approved native topology and ./run.sh selects the role
  from SPARK_TP.
• Ships the v10 TP3 profiles and their measured reports (below). Packaging is
  not qualification: each report's own status banner governs.

Images and source identity

Artifact	Tag	Arch	Registry digest
Coordinator	ghcr.io/tpurtell/ds41rt-coordinator:v10	amd64	sha256:2236d94317eb393cd78940efb117bcca14ae06e6d1113c6aeb188b0b22424689
Spark expert	ghcr.io/tpurtell/ds41rt-spark-expert:v10	arm64	sha256:98ddf9cd83626d92297169b04561b722d136de27ac8213a1a9f5702ebe40ec30

v10 and latest resolve to the same digest per role. The digests are the
registry's own Docker-Content-Digest responses; the coordinator (an amd64 OCI
index) was re-verified by an anonymous pull on the amd64 host, and the Spark image
(a single arm64 manifest) on the arm64 worker kiwi, because an amd64 host cannot
pull the arm64 manifest.

• Image source commit 3dd9a4ac2be9fd17ecf4cb8b7746efdc900d38f0 (bare, no
  -dirty-), version v10, role labels coordinator / expert, CUDA arch 120
  / 121, SparkInfer 4b0954148523b5a2e93813f963d483ffd350b9c9, XGrammar
  557becfb64c503ae9c04344b0047661f43f44320.
• Local build identity at that commit: coordinator image
  sha256:2236d94317eb393cd78940efb117bcca14ae06e6d1113c6aeb188b0b22424689; Spark
  image sha256:d1b668cd7e87079b5b57e381bdd45dfb4858646533611f18f22061f58ae18dec
  on all four workers (the local Spark id is the config digest; the registry
  reports the manifest digest, which is not expected to be equal).
• Dist artifacts and labels verified inside the running containers:
  runs/v10-release/build/ (including RUN-SUMMARY.md,
  10-label-assertions.txt, 10-dist-artifact-verification.txt); dist/SHA256SUMS
  verifies clean over 28 entries and is attached to this release.

v10 TP3 reports (measured, published)

• Official native TP3EP1 1x/3-Spark — INCOMPLETE / INFORMATIONAL. 309 / 359
  expected performance records and 88 / 264 tool-eval scenario-runs, measured on
  the pre-KV_POOL_SIZE-pin profile. Published as measured and incomplete; it is
  not a formal release qualification.
  report
• Compact EXL3 TP3 1x/3-Spark — MEASURED, strict PASS. 359 / 359 expected
  performance records and 264 / 264 tool-eval scenario-runs, with 0 retained
  non-capacity and 0 capacity-gated failures. Its network and change-threshold
  cells remain explicitly PENDING. Pass is not a release-readiness claim.
  report
• campaign status

The headline table in README.md keeps the historical v6/v7/v8/v9 columns
unchanged; v10 does not re-campaign the official default.

Rollback

• Pre-push latest was the v9 pair (coordinator index
  sha256:786d1d6704e4cdaaf12ae59f5324bb1a43ce2238c9ec79ff7884fd3c83e8eb7f,
  Spark manifest
  sha256:f0c67407adb4228200c1fcb97e3f7210501db120d1e1ee11697f66cfb1ca4ea9).
  Re-point latest at those digests to roll the consumer-facing tag back.
• A bad v10 is corrected by publishing v10.1 and promoting it; do not
  overwrite or delete the published v10 manifest.
• Full ordering, evidence and rollback live in
  docs/release-v10-checklist.md.

DS41RT v9

DS41RT v9 release notes

STATUS: PUBLISHED. The v9 images are built, functionally validated and published to GHCR
as v9 and latest. Both were pulled anonymously with a fresh DOCKER_CONFIG and no host
credentials, returning the digests stated below. Model weights are host-mounted and are never
in the images.

What v9 changes

• Adds the pure unreplicated TP6EP1 Spark topology. Six ranks form one group;
  each rank holds a disjoint 2304 / 6 = 384 column slice of every routed expert
  (SPARK_COUNT=6, SPARK_TP=6, SPARK_EP=1). This is tensor parallelism over
  the official intermediate, not expert parallelism and not a reuse of the
  TP2xEP3/TP3xEP2 replicated-group contract.
• Keeps the default four-rank TP4EP1 path unchanged. The release default
  remains SPARK_COUNT=4; TP4 needs no baked role label and is always exported.
• Supports the approved explicit topologies TP2EP2, TP3EP2, TP2EP3, TP4EP1 and
  TP6EP1. The v9 Spark role set is tp2;tp3;tp6, with the default TP4 path
  implicit.
• Widens cleanup scope to every configured Spark host (stop.sh /
  release_select_stop_hosts). Cleanup is a superset of the active ranks, so
  stop.sh reaches containers a previous six-rank run left on the fifth/sixth
  hosts even when the configuration selects four ranks. Launch and restart keep
  cleaning only the active ranks.
• Names the fifth/sixth hosts in ds41rt.config (rhea, moa). The default
  SPARK_COUNT=4 selection does not launch them. The release and candidate
  launchers pass only LANE_A; the secondary LANE_B values are configuration
  annotations: on the coordinator (raptor) both lanes share one physical port
  and HCA, while each Spark has two separate HCAs. No dual-rail gain is
  claimed.

Images and source identity

Artifact	Tag	Arch	Local image id
Coordinator	ghcr.io/tpurtell/ds41rt-coordinator:v9	amd64	sha256:786d1d6704e4cdaaf12ae59f5324bb1a43ce2238c9ec79ff7884fd3c83e8eb7f
Spark expert (all six)	ghcr.io/tpurtell/ds41rt-spark-expert:v9	arm64	sha256:9e8c8248b00366a493023687a42dea9c2544fe8eb7cc1a3c46656a91d9ea3ff9

• Engine revision 5d0d209509bf1f26731bc588dfe8dc72d1373ad0 (bare, no
  -dirty-), version v9, SparkInfer 4b0954148523b5a2e93813f963d483ffd350b9c9,
  io.ds41rt.v41.spark_tp_roles=tp2;tp3;tp6.

• Dist binaries (verified inside the running containers): coordinator daemon
  8c14fd85bb6eb532a196dc8ddd30e8de1ac7128a120e82ab9d75f0eedf416b35, native lib
  b14cac3ea246fe50369221d97aae984cafd72417ef536538813d7f653434b101; Spark daemon
  c176f2c8db03dfd36a179b28bf45e464ac2a41a1a57b34809eeb13914a32231d, native lib
  e1e71c51d1c74a6ef3ae38fb68e0c40a497ad28dfa14f54ddf3e901cb230520a.

• Registry publication: PUBLISHED and anonymous-verified (2026-09-21). v9 and
  latest resolve to the same digest per role:

  • coordinator ghcr.io/tpurtell/ds41rt-coordinator index
    sha256:786d1d6704e4cdaaf12ae59f5324bb1a43ce2238c9ec79ff7884fd3c83e8eb7f
    (linux/amd64 child sha256:649e5c84b25d309457950d3e680b84cd2527434f156aecf8483076c4aa72807c);
  • spark-expert ghcr.io/tpurtell/ds41rt-spark-expert manifest
    sha256:f0c67407adb4228200c1fcb97e3f7210501db120d1e1ee11697f66cfb1ca4ea9
    (arm64; local config id sha256:9e8c8248…).

  The coordinator's local .Id equals its index digest (buildkit stores the
  index+attestation locally); the Spark local .Id is its config digest while the
  registry reports the manifest digest — this is not a discrepancy. Both images were
  pulled anonymously with a fresh DOCKER_CONFIG and no host credentials (rc=0,
  matching digests). The container publication is complete; this GitHub release is the corresponding source release.

Official quantization vs container distinction

• The official quantization is the checkpoint itself —
  deepseek-ai/DeepSeek-V4.1-Flash at
  dba1be0a40aa45a94ad051997016db3960a90277. Its published numbers are the
  historical official campaign; v9 does not re-campaign the official default for
  qualification purposes.
• The container images are the release artifacts built from the frozen source
  commit. They bundle binaries, Python tooling and AOT packages; they do not
  bundle weights. Weights are mounted read-only from the host Hugging Face cache
  and Engram tables are mapped from host storage.
• A container-image measurement therefore proves the shipped code, not the
  checkpoint license, and vice versa. Every number below names which of the two it
  belongs to.

Final-image functional validation (PASS)

All four production geometries launched with the production ./run.sh from the
clean build tree. The loaded container binaries equal the fresh dist artifacts
above, so no old or scratch binary was injected. The API and constrained smokes
return rc=0 on every geometry, and the strict-schema 400 rejection is genuinely
exercised (not forced): json_schema=true, combined_response_tools=true,
strict_tool_stream=true, validation=true.

Geometry	Placement applied	Worker contract	api-smoke	constrained	Content
TP6, 1 RTX, 5/35	RTX GPUs 1, RTX layers 5; Spark first layer 5, remote_layers=35	world=6 role=7 intermediate=384 first_layer=5 layers=35	rc=0	rc=0	9 categories + counting, 3 repeats, 30/30 PASS (nonce 61001)
TP6, 2 RTX, 20/20	RTX GPUs 2, RTX layers 20; Spark first layer 20, remote_layers=20	world=6 role=7 intermediate=384 first_layer=20 layers=20	rc=0	rc=0	9 categories + counting, 1 each, 10/10 PASS
TP4 implicit default, 1 RTX	coordinator dispatch placement not captured	worker LOAD only: world=4 role=1 intermediate=640 first_layer=0 layers=40	rc=0	rc=0	10/10 PASS
TP4 implicit default, 2 RTX	RTX GPUs 2, RTX layers 20; Spark first layer 20	world=4 role=1 intermediate=640 first_layer=20 layers=20	rc=0	rc=0	10/10 PASS

TP4 1 RTX placement. The worker LOAD contract is first_layer=0 layers=40, but
a worker's load count does not say which layers the coordinator dispatches to it.
start-tp4-1x.log prints no placement line and the coordinator container logs were
not captured, so the dispatch split for this geometry is unknown and is
recorded as unknown rather than inferred. In particular it is not an
"all-40-remote" claim: the historical baseline also loads 40 while dispatching
5 local / 35 remote.

Operational prerequisite. GB10 unified memory counts clean page cache against
device-free bytes. The established drop-page-cache step on kiwi and rhea
restored ~113–117 GiB free, after which every geometry launched. This is recorded
as an operational prerequisite, not as tuning; no dial, config or threshold was
changed.

Performance (warm candidate)

These are the v9 TP6 measurements. They were taken on the warm candidate binaries
(daemons a41a76ba/884f4454, libs e9736c3e/c98b392d), which are distinct
from the final clean-build dist binaries above. They are validated candidate
measurements; they are not final-image performance numbers. This headline table
is deliberately warm-candidate only. A final-image 1 RTX headline-shaped functional
run (30 samples) was performed and is preserved, but it is not used as a
performance replacement and is not paired against any other arm.

Metric	1 RTX + 6 TP6	2 RTX + 6 TP6
Weighted decode, median of three	96.83	110.19
Weighted decode, repeats	92.90 / 97.91 / 96.83	110.19 / 110.70 / 109.20
Counting decode, median	174.54	231.51
Prefill, best completed cell	7,131	8,577

The headline protocol uses the fixed corpus 1972e572…, tokenizer c90dfa01…
and nonce 61001, with 30 timed samples per arm and 15/15 objective checks passing.
Mixed traffic completes on both arms (all_http_200): median 175.84 token/s at
1 RTX and 188.47 at 2 RTX. No causal speedup or qualification claim is made:
the candidate throughput stands as measured, and "no speedup" means no causal
TP6-versus-TP4 qualification claim, not that these numbers are void.

Baselines and comparison framing

• Published-v8 clean baselines: 1 RTX weighted 92.3024, 2 RTX 112.5899.
• The 1 RTX primary default comparison against published v8 is cost-confounded:
  the v8 baseline's builtin-calibration resolution is inferred from source
  (profile=None), not observed in a captured resolved log, while the TP6
  launcher passes explicit --spark-tp/--spark-ep and resolves legacy-heuristic.
  The confound is stated, not erased.
• The 2 RTX primary comparison is conservatively unverified (the v8 2x resolved
  cost model is not observed), so no same-cost claim is made.
• A new controlled-cost side arm exists for context: TP4 1 RTX legacy
  legacy-heuristic 88.5130 versus TP6 1 RTX legacy-heuristic 96.8271.
  This is a side arm, not a replacement for the default baseline.

Component evidence (provenance, not numerical TP6 proof)

Earlier SM120/SM121 coordinator and expert packing builds and their FFI/mask
contract audits are preserved in the evidence archive
(component-evidence/COMPONENT-EVIDENCE-INVENTORY.md). They come from the same
source lineage but are not numerical TP6 test proof: build logs and the
RealFullConstraintMasks packing paths record compilation, not a TP6 numerical
result. Only the actual geometry selftests plus the qualifier numerical passes from
the earlier valid artifacts count. The recorded ffi-symbol-shortfall diagnostic is
historical: it was taken against an earlier broken library (1c3) and was
superseded by the full coordinator library e9736c3e and then the final
b14cac3e; it is not a current missing-ABI claim. Warm-candidate kernel/expert
tiling evidence is likewise distinct from the final dist binaries and does not
transfer as final-image proof.

Known limitations

• **1 RT...

DS41RT v8

DS41RT v8 release notes

V8 enables the NVFP4 W4A4 optimizations prepared on the development branch and
re-measures that quant on the published images. Only the NVFP4 configurations
were re-campaigned this cycle; the official MXFP4 and EXL3 figures in the README
are the v6 and v7 campaigns respectively and are labelled as such there.

What changed

The NVFP4 routed-expert path previously used the generic deterministic fused
kernel. This cycle wires the optimizations that were reachable without changing
the checkpoint format:

• Load-time padding of the expert payload (export_b12x_v41_nvfp4_aot.py,
  v41_experts/nvfp4.rs, the FFI launch contract).
• Adaptive NVFP4 splitting exposed through the exporter, with the
  corresponding SM121 Spark share-input kernels (SparkInfer revision
  4b0954148523b5a2e93813f963d483ffd350b9c9, pushed to the fork before pinning).
• An NVFP4 speculative cost profile selected by
  configure_cost_model(&transport, catalog.nvfp4().is_some()).

Measured result

Both layouts, published images, three samples per cell, C1 dSpark decode.

Measurement	NVFP4 1x v7	NVFP4 1x v8	NVFP4 2x v7	NVFP4 2x v8
C1 code decode	86.58	112.55 (+30.0%)	106.38	145.96 (+37.2%)
Counting decode	114.06	144.66 (+26.8%)	149.98	192.42 (+28.3%)
Weighted decode	66.81	81.74 (+22.4%)	78.57	100.78 (+28.3%)
Best prefill	4,141	5,237 (+26.5%)	7,370	7,371

Prefill gains are shape-dependent. On 1x they are broad (+16 to +22% median at
every suffix length); on 2x they are concentrated in short suffixes (+9.2%
median at +1K, +0.5% at +32K), and because "best prefill" is the maximum cell —
the 0K/+32K cell — the 2x headline barely moves even though most of the matrix
improved. The full 30-cell matrices are in the report.

Both layouts completed the full battery on the published images: retained-context
decode with its separate 2K control, counting/code/topic concurrency scaling
through C16, mixed traffic, target-only decode, startup and post-readiness
memory, and three high-effort tool-call evaluations each (1x 156/157/152, 2x
156/157 over 176).

Costs

The gain is not free. On the Sparks, per-layer residency rises from
1,911,035,904 to 2,123,372,544 bytes (+11.1%, about 68.8 GB to 76.4 GB over the
36 layers) and layer loading slows from roughly 935 ms to 2,000–2,263 ms. That
is why expert readiness for NVFP4 1x now measures ~171 s end to end against
~47 s at v7, and it reduces the headroom left on a 100 GiB Spark budget.

The NVFP4 2x battery was measured with a 512 MiB global KV pool (the v7
campaign used the default), which leaves the TP2 pair enough device headroom for
its load-time and request-time allocations. This is part of the measured
configuration, recorded here as the compact EXL3 profile's 2 GiB pool is.

Container images

• ghcr.io/tpurtell/ds41rt-coordinator:v8 (linux/amd64), digest
  sha256:08c2d6df9a0a6a365eff2c014172478b40d9f39d06437a1c9244c566181b9e40
• ghcr.io/tpurtell/ds41rt-spark-expert:v8 (linux/arm64), digest
  sha256:9907983992916bb8e0f35ab869e12706cfe4613cc6dcd93f4ff53796c2d80bf6

Both also carry latest. Both roles are built from one staged tree at
5a56f0e, so they share org.opencontainers.image.revision
(5a56f0eb9d8caea050f84f96332c66786e177607) and
io.ds41rt.sparkinfer.revision
(4b0954148523b5a2e93813f963d483ffd350b9c9); run.sh requires that equality
across every Spark before it will deploy. ds41rt.config points both roles at
:v8.

Reading

The optimizations are real and reproduce on both layouts, with the 2x path
gaining slightly more on decode than 1x. The prefill headline for 2x should be
read together with its matrix, for the suffix reason above. The startup and
expert-residency costs are measured, not estimated, and should be weighed against
the throughput gain when sizing a deployment.

See the NVFP4 performance report for the full
tables, raw-record digests and per-configuration qualification.

DS41RT v7

V7 adds native support for two more quantized DeepSeek V4.1 Flash publications
— NVIDIA's ModelOpt NVFP4 (W4A4) and diffbot's EXL3 2.0 bpw — while the
official full checkpoint stays the launcher default and its measurements are
unchanged from v6.

Key features

• NVIDIA nvidia/DeepSeek-V4.1-Flash-NVFP4 (W4A4). E2M1 weights with
  E4M3 K16 block scales, activations quantized in the kernel from BF16 rows.
  The loader writes no repacked weight: the checkpoint payload lands in the
  kernel-native [up; gate] order with one device-to-device copy per expert,
  and both scale planes are re-laid into 128x4 atoms on device. Runs in the
  same one- and two-card topologies as the official checkpoint, with the
  remainder of the layers on four Sparks.
• diffbot DeepSeek-V4.1-Flash-EXL3-2.0bpw-2x-RTX-PRO-6000. Uniform K=2
  trellis. On two cards all forty routed-expert layers are resident and no
  Spark worker is involved. On one card the new compact profile holds the
  card to a 32 GiB budget including headroom and serves the rest of the model
  from exactly two Sparks, which is how the checkpoint runs on a 32 GB card.
• One GPU image for every part in the family. The AOT artifacts are not
  SM-count specific — the host's SM count only bounds the cooperative grid —
  so the engine accepts a same-capability device with fewer SMs and clamps
  the launch cluster cap to the card. A 5090 needs no rebuild and no separate
  AOT set. (No physical 5090 was available to test; the grid behaviour was
  verified on an RTX PRO 6000 at 188 and 170 SM inventories.)
• Placement now budgets NVFP4 layers at their own measured cost rather than
  the native one, which had over-filled a card at the margin.
• EXL3 routed-expert support, added in v5 and disabled by default, is enabled
  and qualified for this checkpoint.

Container images:

• ghcr.io/tpurtell/ds41rt-coordinator:v7 (linux/amd64), digest
  sha256:d85608bbe14ce655a3bae4fd61655a0020099f056d357a40b8a3ac5115275c5e
• ghcr.io/tpurtell/ds41rt-spark-expert:v7 (linux/arm64), digest
  sha256:aa477ff1c74fe4e815734c1e57d3fad325c5126ddbb324c92fd58f6d02fe856b

Both also carry latest. Both roles are built from one staged tree, so they carry the same engine revision; each image re-verifies the packaged EXL3 families
against the pinned SparkInfer revision at build time.

All reported RTX measurements use a 400 W power limit per card, standard
memory speed, and three samples per cell. See the
NVFP4 report and the
EXL3 K2 report for exact results,
configuration detail, and what is still owed; the
configuration chart shows each profile's
memory composition.

Against the official checkpoint on the same two-card topology, W4A4 trades
decode throughput for half the routed-expert bytes per rank (weighted 80.1 vs
109.4 tok/s, best prefill 7,432 vs 8,355), and EXL3 2 bpw is the fastest
decode measured here at 145.1 weighted and 337.4 counting tok/s, at a lower
best prefill of 5,702. The single-card compact profile reaches 88.1 weighted
and 2,015 best prefill within its 32 GiB ceiling.

DS41RT v6

DS41RT v6 adds RAM-backed retained KV caching and safer admission under pressure while preserving the fastest native dual-RTX serving layout as the default. It also ships independently selectable TP2 experiments for further development without enabling paths that did not win end-to-end measurements.

Key features

• Automatic pinned-RAM backing sizes GPU-plus-host logical capacity above 20 retained entries × the configured maximum context. The standard dual recipe provides 14,680,064 GPU tokens plus 6,291,968 RAM tokens.
• Token-aware admission, bounded HTTP queueing, cancellation, explicit overload responses, and safe cache-pressure progress integrate PR #4 and resolve issues #2 and #3.
• Budget-driven bottom-up RTX expert placement can preserve a requested KV budget without requiring exactly 20 RTX layers.
• Opt-in TP2 attention with replicated KV, query projection, output projection, and dSpark routed experts are available independently. They remain off by default because qualification found no consistent end-to-end win.
• The official DeepSeek V4.1 Flash checkpoint remains the native default. EXL3 remains compatible; its historical performance data is linked from the v6 README rather than presented as v6 qualification.
• Standard serving uses port 8000, concurrency 16, adaptive K7 drafting on dual RTX, 20 retained turns, FP4 compressed source KV, and FP8 SWA.

Native performance

Measurements use 400 W per RTX and standard memory speed. Dual RTX reaches 109.44 weighted dSpark tok/s versus 92.00 on one RTX (+19.0%), 1,195.37 C16 code tok/s (+11.8%), 1,508.81 C16 counting tok/s (+19.1%), and 8,355.22 best prefill tok/s (+6.8%). A fresh same-machine v5/v6 control measured mixed decode at 327.02 versus 326.01 tok/s (-0.31%) and 256K retained decode at 98.23 versus 96.36 tok/s (-1.91%).

Validation and artifacts

• Full release campaign: native one/two-RTX decode, retained context through 256K, prefill, target-only controls, combined TP2 smoke, basic EXL3 smoke, and a two-minute tiny-KV pressure torture run.
• Script gate: 209 passed, 1 skipped, 46 subtests passed.
• Coordinator image: ghcr.io/tpurtell/ds41rt-coordinator:v6 (sha256:22d74a117613fa3ed1b75912b3e3732c82b31a55487ffb45efa28b08d53abd00).
• Spark image: ghcr.io/tpurtell/ds41rt-spark-expert:v6 (sha256:9b8a420939ab5e2b5b81047b0b7b1e399c6e0b035b05f824026faea207a130d5).
• Attached binary bundles include exact provenance and internal checksums; the evidence bundle contains the raw reports and telemetry behind the published tables.