bloomery 0.3.12

bloomery 0.3.12 makes long prompts much faster on the cards our contributors run, and serves Mistral Large 4
(1T parameters, 52B active) on one 24 GB card. Numbers below are each PR author's own measurement on the card named;
the Mistral Large 4 rows are ours, measured as stated under its table.

Faster prompts and steadier serving (@juliankang4)

Change	Card	Before → after
Qwen3.6-35B-A3B + MTP, prompt of 4,096 ids, prefill (#1073)	RTX 5090	2,071.6 → 8,747.2 tok/s; first token 1,979 → 470 ms
Qwen3.6-35B-A3B Q4_K_M, prompt of 4,096 ids, prefill (#1127)	RTX 5090	14,441.9 → 18,712.5 tok/s
Qwen3.8 FP4-Local + MTP, second request on a shared 9,000-id prefix, prompt time (#1012)	DGX Spark	4,642 → 1,003 ms
Qwen3.8, --parallel 2: the longest stall of a decoding stream while a 16,385-id prompt arrives (#1069)	DGX Spark	6,308.9 → 1,719.7 ms
--hf download of a 4.48 GB GGUF (#1037: 16 resumable, verified connections)	RTX 5090 host	36.1 → 20.3 s

Qwen3.8 also gained --parallel auto with --slot-ctx (#1011, #1123), slot save and restore (#1027), KV memory that grows with the sequence
(#993), and FP4-Local and NVFP4 loads split across card, host and drive (#1034, #1036, #1103).

Mistral Large 4

• Both files: unsloth's UD-IQ1_S and Mix-256G. --place a runs UD-IQ1_S on one RTX 3090: the dense and attention
  weights on the card, the routed experts streamed from host memory.
• Batched prefill, the expert stream beside a tier card, and vision (Pixtral).
• The DSpark draft (BLOOMERY_DRAFT=dspark, opt-in): its first draft is accepted 74.7 % of the time on our
  reference prompts, measured against upstream's draft on the same file.

UD-IQ1_S, the release's own bloomery-serve at the serving defaults (no levers, --ctx-size unset), one request
after a prompt of D ids, from the release sitting (on the binary of 2cd109a, whose source the tag's binary
shares; the tag adds the README status only):

Cards	--place	D	Decode tok/s	Prompt tok/s
RTX 3090	a	4,096	10.85	36.01
RTX A6000 + RTX 3090	bp	512	11.03	34.82
RTX A6000 + RTX 3090	bp	4,096	11.48	40.11

The 3090's D = 512 row read the experts cold from the drive (11,205 major faults) and is left out.

Known issues

• GLM-5.3 at --place bp: the same short request sent a second time, after a long one, can answer different text
  (the release check that replays it was red in 0.3.11 too). Decode and prompt speed are unaffected.
• DeepSeek V4.1 on one 24 GB card, from a file that has an r8 sidecar beside it: the server refuses at load by name
  (the NVMe tier does not fill from the sidecar yet). BLOOMERY_R8=off reads the source file with the same bits. A
  plain GGUF and two cards (--place bp) are not affected.
• The GB10 archive (linux-aarch64-cuda-sm121a) follows this release. Until it is attached, install.sh on a DGX
  Spark stops at the download by name; install.sh --version v0.3.11 installs the last one.

Also in this release

• Server: per-key scopes and rate limits (#1003), Ollama-style /api/tags, /api/show and /v1/models/{m} (#994),
  gzip and deflate request bodies (#1049), reasoning_loop (#1009), more decode and draft detail on /metrics (#1007).
• Apple Silicon: Qwen3.8-27B decodes 20 % faster on an M1 Pro (#1047) and long contexts decode faster (#1060); MLX
  8-bit, DWQ and F16-scale folders (#1057), Qwen2, Qwen2.5-Coder and tied-head folders (#1131), Llama 3 tokenizers (#1052).
• @wsj040303's first contribution makes the q3pp profiling plans read their batch limits from the model header (#1072). Thank you!

Thanks

To @juliankang4 for the prompt speed work on the RTX 5090 and DGX Spark, Qwen3.8's serving slots, and the FP4 loads,
and to @wsj040303 for a first contribution.

Written with Claude Code (Claude Opus 5.5); reviewed by the maintainer.