v12.1.0 — dev748, 7-model fleet validation, 242 t/s on 2×A5000

sndr-platform v12.1.0 — dev748, fleet-validated

325-patch orchestration platform for vLLM on consumer GPUs. This release ships the dev714→dev748 pin promotion, a 7-model fleet validation, three battle-validated patch fixes, four new operator docs, and a green CI.

Headline (2× RTX A5000, pin dev748, 2026-07-04 promotion gate)

Metric	Value
wall_TPS (35B AWQ, TQ k8v4, MTP K=5)	242.5 t/s (CV 6.9%, +3.5% vs dev714)
decode TPOT / TTFT	3.9 ms / 84.5 ms
Tool-calls	7/7 (qwen3_xml)
MTP accept-rate	0.653 (floor 0.55)
Ctx-scaling 1K→32K	LINEAR_OK — no cliff

Fleet: 7 models, one pin, failed=0 patch applies everywhere

35B AWQ 242.5 t/s · 35B FP8 231.2 t/s · 27B INT4 TQ (+PN520) ~130 t/s · 27B fp8kv (+P100) ~108 t/s · Gemma4-26B ~140 t/s · Gemma4-31B (+PN351) ~87 t/s · DiffusionGemma ✓ — full table in docs/BENCHMARKS.md.

Battle-validated this cycle: PN520 (cured the INT4-27B boots-clean-but-garbage GDN loader regression, 96 shards routed), P100 (FlashInfer FULL-CG spec-decode, runtime-proven), PN351 (dev748 anchor variant on head_dim=512).

Pins (SSOT sndr/pins.yaml)

• current: 0.23.1rc1.dev748+g2dfaae752 — image vllm/vllm-openai:nightly-2dfaae752
• rollback: 0.23.1rc1.dev714+g09663abde · stable: v0.24.0

vLLM image archive (Docker Hub nightly tags get purged — this pin is preserved)

The exact promoted engine image is archived off-registry:

file:   vllm-nightly-2dfaae752_dev748.tar   (8.8 GB, compressed OCI layers)
sha256: a52eb6aa9341500dbdc79c4eb5768d06bd07b779cbac1dbd51bca7ff7ce631ab
load:   docker load -i vllm-nightly-2dfaae752_dev748.tar

Stored on the reference rig at /home/sander/releases/ (33 layers, RepoTag vllm/vllm-openai:nightly-2dfaae752, checksum-verified). Too large for GitHub release assets — mirror it to your own storage alongside this source release for full reproducibility.

New documentation

ARCHITECTURE · HOST_SETUP · ADDING_MODELS · OPERATIONS

Quality

14 802 tests green locally · CI fully green (pytest 3.10/3.12, ruff, doc-sync, memory-postgres) · doc gates: 325-count consistency + preset-key resolution · known low-severity note: dependabot #27 (torch, no patched version exists yet).

What's Changed

• fix: dry-run pull crashes on uncreatable models dir + redact LAN host (greens CI) by @Sandermage in #35
• docs: discoverability & onboarding uplift — naming, ramps, CITATION, OG, Pages site, promotion (W0–W3) by @Sandermage in #34
• v12 consolidation: dev748 promotion, PN520/P100/PN351 battle-validated, fleet sweep, docs waves by @Sandermage in #44
• Wave 3 docs (manuals+architecture+SEO README) + hardware pin fix + CI green restore by @Sandermage in #45
• release: v12.1.0 by @Sandermage in #46

New Contributors

• @Sandermage made their first contribution in #35

Full Changelog: v12.0.0...v12.1.0

v12.0.0

Full Changelog: https://github.com/Sandermage/genesis-vllm-patches/commits/v12.0.0

Full Changelog: https://github.com/Sandermage/genesis-vllm-patches/commits/v12.0.0

Full Changelog: https://github.com/Sandermage/genesis-vllm-patches/commits/v12.0.0

Full Changelog: https://github.com/Sandermage/genesis-vllm-patches/commits/v12.0.0

Full Changelog: https://github.com/Sandermage/genesis-vllm-patches/commits/v12.0.0

Full Changelog: https://github.com/Sandermage/genesis-vllm-patches/commits/v12.0.0

Full Changelog: https://github.com/Sandermage/genesis-vllm-patches/commits/v12.0.0

Full Changelog: https://github.com/Sandermage/genesis-vllm-patches/commits/v12.0.0

Full Changelog: https://github.com/Sandermage/genesis-vllm-patches/commits/v12.0.0

Full Changelog: https://github.com/Sandermage/genesis-vllm-patches/commits/v12.0.0