Mach X v0.2.26

Everything since v0.2.25: GLM-5.3-Flash in ie serve reads a 9.6K-token agent prompt in 32.4 s (was 243 s), answers later turns from the prompt cache in 9.5–9.7 s (was 273–311 s) and decodes 13.5–14.3 tok/s (was 10.8–11.5), and the re-published glm5-next GGUF loads. Host↔device copies on the Qwen3.8-27B and 35B-A3B two-card splits, the samplers and --spec's draft head go through pinned memory, at most 64 MiB a copy, for issue #2.

Release gate: the public tree builds from scratch to the same ie binary (md5) as the one tested and installed, and 11 host test suites pass. On two Arc Pro B70 cards, greedy replies are identical to v0.2.25's on Flash-Next, the 35B-A3B, the 27B, MiMo-V2.6, Laguna and DeepSeek-V4.1 (V4.1 with its CPU expert path off, which makes it deterministic), at the same speed within run-to-run variation. GLM-5.3's greedy replies match its own gate, the prompt-cache restore included. Inkling-Small was not re-run; its code is unchanged since v0.2.25.

Mach X v0.2.26

Everything since v0.2.25. Measured
October 19–20, 2026, on two Arc Pro B70 cards.

GLM-5.3-Flash in ie serve: first token 7.5× sooner, later turns 28–33× sooner

On an agent conversation (a 9.6K-token system prompt with three tools, then ~1.4K-token tool results), with the
server's defaults and the model card's sampling:

	v0.2.25	v0.2.26
first turn, 9.6K tokens cold	243.2 s (39 tok/s)	32.4 s (300 tok/s)
later turns	273–311 s (the whole conversation read again)	9.5–9.7 s
decode	10.8–11.5 tok/s	13.5–14.3 tok/s

• Prompt cache: a turn that extends the conversation restores both cards' state from system RAM and reads only the
  new tokens. Re-sending a turn restored 10,989 of 10,991 tokens and answered in 0.57 s with the same reply.
• Pipelined prefill: card 1 reads the next prompt chunk while card 2 finishes the current one, in 2048-token chunks
  (was 256; --prefill-chunk up to 2048 sets it, and 1024 gives back about 4 % of decode).
• MTP draft on by default (IE_G5_PIPEDRAFT=0 turns it off).
• Prefill on XMX: routed experts with 32 or more rows in a chunk, the dense Q8_0 projections and the MLA absorbs
  run as XMX GEMMs; experts below 256 rows split K four ways (gemm_fp16_splitk), and Q4_K/Q5_K dequant is tiled
  through shared local memory (bit-identical). Switches: IE_G5_XMX_ROWS, IE_G5_Q8_GEMM_ROWS,
  IE_G5_ABSORB_XMX_ROWS, IE_G5_SPLITK (0 = off each).
• Checks: greedy with the CPU expert leg off (IE_G5_CPU_MISS=0), the restored turn, pipelined prefill against
  serial prefill and the MTP draft against plain decode each give byte-identical replies. The XMX prefill changes the
  summation order, and benign arithmetic changes move this model's PPL by about 0.002; split-K is no worse (wikitext
  at 16K context: 1.9461 with it, 1.9469 without).
• The re-published GGUF loads. Unsloth's GLM-5.3-Flash files were rewritten on October 5 with the architecture
  named glm5-next; v0.2.25 refused them ("unsupported architecture"). Both names load now.
• Prefill chunks above 2048 are refused: at 4096 a kernel's 32-bit range overflowed.

Measurements.

Host↔device copies through pinned memory (issue #2)

A B70 wedged after about two hours of ie serve --gpus 2 --spec on the Qwen3.8-27B (a copy-engine fault, then a
failed GPU reset). It matches an open driver issue (intel/compute-runtime#948,
drm/xe#8390) that sustained load exposes on B70 and B580, in other engines too. In #948 the copy engine faults on
single copies of 512 MiB or more from pageable memory, so the engine now copies through pinned memory, at most 64 MiB
a copy:

• Every host↔device copy of the Qwen3.8-27B and 35B-A3B two-card splits, including the card-to-card hand-off, the
  samplers' per-token copies and the dense loaders, goes through a 64 MiB pinned buffer per card. No single copy is
  larger than 64 MiB.
• --spec's MTP draft head loaded token_embd and the packed output.weight (about 1 GB each) in single pageable
  copies, in exactly the issue's configuration. Both go through the pinned buffer now.
• Small device-to-host reads (up to 1 MiB) run as a kernel on the compute queue: as copy-engine copies they waited
  for the compute → copy-engine hand-off after every step and cost the 27B's two-lane decode 0.55 %.
• Checks: the 27B in the issue's configuration (--ctx 262144 --parallel 1 --spec --spec-k 3), on one card with
  --spec, and with two lanes; the 35B-A3B with one request and with lanes; with and without repetition penalties:
  byte-identical replies to v0.2.25 at the same speed or slightly faster. The prompt-cache slot stash restores
  bit-exactly. No kernel GPU fault in any run.
• Not shown: a multi-hour soak. A soak can show the fault; it cannot prove it is gone.
• Switches: IE_NO_PINNED_STAGE=1 restores the direct copies, IE_NO_D2H_KERNEL=1 the copy-engine reads,
  IE_COPY_STATS=1 prints calls and time per copy path at exit.

What is still slow, and not checked

• DeepSeek-V4.1-Flash runs on two cards only. On a machine with more Arc cards it takes all of them, and past two
  cards prompts over 2,048 tokens fail and attention past 16K tokens of context is wrong. Run it with --cards 0,1
  (ie cards lists the numbers). Four-card support is planned.
• GLM-5.3-Flash prefill is bound by streaming the host expert bank over PCIe 5.0 x8 and by per-expert compute;
  dequantizing inside the split-K GEMM is the next step. ie serve runs GLM one request at a time.
• The other models (Flash-Next, the DeepSeeks, MiMo, GLM, Gemma and more) still copy host memory directly, mostly
  whole tensors at load. Laguna and Inkling have used the pinned buffer since v0.2.25. Each conversion needs its own
  old-against-new gate.