Skip to content

GRIMOIRE v1.10.0 - multi-user MTP on two GPUs, Arc Pro B60, oneDNN

Pre-release
Pre-release

Choose a tag to compare

View all tags
@doopeworld doopeworld released this 28 Oct 16:45
v1.10.0
3fb9df7

GRIMOIRE v1.10.0 is built from main at 3fb9df7. Its main changes are multi-user MTP on two GPUs, Arc Pro B60 support, oneDNN, and the newest Intel toolchain.

curl -LO https://github.com/doopeworld/GRIMOIRE/releases/download/v1.10.0/grimoire-b70-v1.10.0.tar.gz
docker load < grimoire-b70-v1.10.0.tar.gz        # -> grimoire-b70:v1.10.0

sha256 6786c9c1595ea677878823346d8a3ef36fda6c4dd5489697f31b93dde7355efa

New since v1.9.0

  • Two GPUs (Megatron tensor parallel) for models that do not fit one card. Measured on Qwen3.8-27B-FP8 with two Arc Pro B70s: prompt 3,530 tok/s at 4K, and 29.5 tok/s decode for one user without MTP. Qwen3.8-Flash-Next runs tensor plus expert parallel at 44.6 tok/s decode.
  • MTP for several users at once on two GPUs. Before this release it served one request at a time. With Qwen3.8-27B-FP8 + MTP, 1 / 2 / 4 / 8 users get 56.5 / 62.9 / 95.0 / 161.9 tok/s in total (llama-benchy, pp4096 / tg128). On the coding test it decodes at 88 tok/s, and the answers pass their unit tests.
  • Intel oneDNN for the prompt GEMMs, on by default. Prompts run 5-26% faster on two cards. GRIMOIRE_ONEDNN=0 turns it off.
  • Arc Pro B60 / Arc B580 (Battlemage G21) support. The same image carries code for the B70 and for G21, so the B60 no longer needs its own rebuild. The kernels are tuned on the B70, and we have not measured a B60 ourselves.
  • KV cache fitted to the card. GRIMOIRE now prints KV cache: N slots x --ctx M, X GiB on this card at start. When GRIMOIRE_SEQ_SLOTS conversations at --ctx do not fit the card, it holds fewer at once and says why; the other requests wait for a free slot. Before, the driver silently moved memory to system RAM and every step crawled; this was the 4 x 200K test that never finished. GRIMOIRE_SLOTS_FIT=0 keeps the requested slots.
  • No more misleading start-up lines. v1.9.0 printed "... unavailable; using fallback" five times at every start. Those lines described optional outside libraries that GRIMOIRE does not ship; GRIMOIRE's own fast kernels were running all along. The lines now appear only with GRIMOIRE_BRIDGE_DEBUG=1.
  • Newest Intel stack: oneAPI DPC++ 2026.2, GPU driver (compute-runtime) 26.35 with IGC 2.41.5, and oneDNN 3.14. We checked it against the previous stack: 43 of the 44 regression answers are identical, the 44th changes one word with the same meaning, and speed is the same on one and two cards.
  • MTP with GRIMOIRE_SEQ_SLOTS=1: the v1.9.0 bug that could skip or repeat parts of an answer is fixed.

Test system

Beelink GTi15 Ultra (Core Ultra 9 285H, 96 GB) with two Arc Pro B70 32 GB cards: one on PCIe Gen4 x8, one on Gen4 x4 through an M.2-OCuLink redriver. The cards have no peer-to-peer link. Numbers come from llama-benchy 0.4.0 through the OpenAI-compatible server. See the README for launch commands, including the two-GPU multi TP mode.

This is still an experimental release (pre-release).

Assets 3
Loading