bloomery 0.3.7

bloomery 0.3.7 adds three things:

• Raon-1-mini-preview on one card, with its DFlash2 draft;
• MiniMax-M3, with its experts on the card and the host;
• Solar-Open2 on two cards (--place bp).

Install

Linux x86-64 with AVX2 (glibc 2.34+), an NVIDIA GPU of compute capability 8.6+ and driver R580 or newer (CUDA 13).
On Windows, run inside WSL2. Docker and Homebrew as in the README.

curl -fsSL https://raw.githubusercontent.com/midagedev/bloomery/main/tools/release/install.sh | sh
bloomery-serve -m PATH/raon-1-mini-preview-Q4_K_M.gguf --port 8080

New

• Raon-1-mini-preview (#598, #613).
  • The card runs the RDA delta-rule layers, the gated NoPE latent attention, SiTU-GLU experts with a 128-expert sigmoid router, and the four-stream mHC mix.
  • Measured on one RTX 3090 at the serving defaults (ctx 262,144), on a build of this code with the same kernels (e8bd14c08):
    • draft off: decode 131 tok/s;
    • the defaults, with the draft beside the file: 122–126 tok/s.
  • Tape: a Korean answer of 2,048 tokens at 122.4 tok/s.
• DFlash2 draft for Raon (#601).
  • Put the draft file (dflash-raon-1-mini-preview-Q8_0.gguf) beside the model. The server then drafts a thinking-on request behind its width chooser.
  • The answer is the same as without the draft.
  • BLOOMERY_DRAFT=off runs plain; BLOOMERY_DRAFT=dspark drafts every greedy request.
• MiniMax-M3 (#599). This covers its block-sparse attention (MSA), the card body and the prompt batch. The prompt-batch walker is one walker that Solar-Open2 shares.
  • Measured with UD-IQ4_XS (208.5 GB) at the serving defaults on the RTX A6000:
    • the plan puts 662 experts on the card and 6,634 on the host, ctx 131,072;
    • decode is 14.3–14.5 tok/s;
    • a 197-token prompt runs at 37.7 tok/s prefill.
  • Tape: Korean, 512 tokens.
• Solar-Open2 on two cards (#602). Plan bp runs the stage on one card and an expert tier on the second.
  • The tier code is common with GLM-5.3-Flash.
  • The two-card gate gives the same decode, call and group bits as the one-card run of the same plan.
• From contributors:
  • BF16 dense weights in their own words, picked per tensor (#592).
  • Checkpoint BF16 matrices stay BF16 by default; the Q8_0 rewrite is behind BLOOMERY_PREPARE_Q8 (#607).
  • FP8 weights with row, tensor or 32×32 E8M0 block scales (#579, #562).
  • Qwen3.8 served from the NVFP4 checkpoint on one card (#561).
  • A type's card columns read from one row (#588).
  • The GLM tensor-parallel rank body on the fixture (#589).

Fixed

• The serve API authorizes request heads and bounds network image decoding (#590). apply-template exposes the guarded prompt ids (#603).
• The Q8 quantizers refuse non-finite values (#597).
• Fixture budgets that are stale are refused before planning (#606).
• Failed cache snapshots are rejected (#577).
• Developer tooling (#574, #575, #576, #586, #604, #610): see the merged pull requests since v0.3.6.

Known limits

• Raon prompt processing is slow. The prompt runs through the rows pass in chunks of 8, at about 250–300 tok/s.
  • On the same RTX 3090 and file, the model's own llama.cpp fork measures pp512 at 2,675 tok/s and tg128 at 102 tok/s (llama-bench, -ngl 99 -fa 1).
  • Moving Raon to the common chunked prefill is the next step.
• Raon's draft rarely pays at these prompts. A width-1 verify window costs 18.8 ms against a plain step of 8.0 ms, so the chooser keeps the gate mostly closed.
  • The draft's first-block check against its reference reads 0 of 7 on this build: gate-gpu-raon1-dflash is red. Corrected 2026-10-22: the gate compared the vendor's second block with the first block's ids. The draft matches the reference, 7 of 7 on both blocks (#615, #619).
  • Output is not affected, because every drafted row is verified by the target.
  • The verify window's cost is open work for 0.3.8 (#617).
• From 0.3.6:
  • gate-qdot's IQ4_XS clause is red by 5 ULP (the box's ik reference build predates ik PR 2664).
• From 0.3.5:
  • V4.1 on two cards (--place bp) is not faster than one card.
• From 0.3.4:
  • Bolt needs a 24 GB card and refuses --place bp.

Checked before the tag

Release head fe986cb21: the union of #599, #602 and #613, then the version (e3df22ceb) and the README status.

• The fixture-tier batch over every gate just affected selects ran 139 items: 2 red. Both were Raon meta tests that pinned the state before the body; they were re-pinned with FAIL-first and ran green.
• The static tier ran green on the second Mac.
• After the review fixes, these ran green on the real files:
  • gate-gpu-lib;
  • gate-gpu-raon1-e2e;
  • the Solar-Open2 prefill, e2e and serve gates;
  • the MiniMax-M3 prefill, e2e and serve gates.
• gate-gpu-raon1-dflash is red, as above. It was a gate defect, corrected above.
• This release skipped three steps:
  • the real-tier deferred list (94 gates);
  • the user-facing pass from the tarball;
  • the pull requests' contrib-check t0 runs, which had not finished at the merge.
• The release builds from this commit. A defect found after the tag is fixed in place, with the notes and assets replaced.

Thanks

• @juliankang4 for the BF16 and FP8 weight paths, the NVFP4 Qwen3.8 checkpoint, the QROWS columns, the GLM tensor-parallel fixture, and the quantizer and budget checks.
• @Burntgogi for the serve request guards, the apply-template prompt ids, the host CPU diagnostics, and the tooling fixes.