Repository navigation
Mach X v0.2.6
Third tagged release of Mach X — a C++/SYCL LLM inference engine for Intel Arc GPUs, written from the metal up. No llama.cpp fork, no PyTorch, no vendor runtime.
v0.2.6 is about agent swarms on one engine: a lead and up to 15 sub-agents that read files and write at the same time, on the Qwen3.6-35B-A3B class (Q8_0) split over two Arc Pro B70 cards. A replay of that workload which did not finish in 336 s now takes 139.7 s. The release also returns tool calls and reasoning in the OpenAI fields on the Qwen templates, moves the Q8_0 kernels to the compiler's native integer dot (bit-identical; Qwen3.8-27B at 16 lanes 113.8 → 153.5 tok/s), lets the engine pick the number of request lanes, makes an orderly stop take about a second, and runs Q6_K and Q5_K natively on the 27B and 35B-A3B splits.
Built and tuned on two Arc Pro B70 cards (64 GB VRAM). Figures were measured September 29 – October 1, 2026 on the driver stack v0.2.0 shipped on; the 35B-A3B-class figures, and the Qwen3.8-27B's 16-lane and Q6_K / Q5_K figures, are on community fine-tunes of those models. One run per figure unless a line says A-B-A.
Highlights
Agent swarms: up to 16 agents that read and write at once
v0.2.0's 16-lane figure was measured with short prompts and long replies. Agents do the opposite: each turn appends 1–3K tokens of tool output and writes a short reply, so the cards spend most of their time reading. The engine's replay of that workload (tools/swarm_replay.py) runs 15 worker conversations for 3 turns each (prompts growing from 1.4K to about 8K tokens), then a wave of 15 new conversations that share one 12.3K-token system prompt, with one follow-up each: 75 requests against ie serve --gpus 2 --ctx 262144 --parallel 16 --thinking on, the flags the Dream Agent Harness starts it with. Its worker phase reads 87.8K new prompt tokens and writes 8.6K.
| build | whole replay | decode, summed over the run | decode per request, workers (median) | the shared-prompt wave |
|---|---|---|---|---|
| September 30, before this work | not finished at the 336 s cut (45 of 75 requests) | 25.4 tok/s | 3.87 tok/s | 0 of 30 requests done: each of the 15 agents read the shared prompt itself |
| v0.2.6's scheduler and kernels, October 1 | 139.7 s, 75 of 75 | 101.2 tok/s | 8.10 tok/s (5th percentile 6.64) | 30 of 30 done |
Step by step, each pair of runs made back to back (the same build measured 146 s and 158.0 s in two sessions, so compare within a row):
| change | whole replay | decode, summed over the run |
|---|---|---|
| a shared prompt read once, conversations kept per lane, tiled prefill attention (September 30) | not finished at 336 s → 212.7 s | 25.4 → 67.0 tok/s |
| native int-dot Q8_0 MoE kernels (October 1) | 214.9 → 169.6 s | 66.3 → 84.2 |
| both cards busy during a deep prefill, short requests first (October 1) | 166.2 → 154.3 s | 86.3 → 92.7 |
| 512-row prefill pieces beside running lanes, were 2,048 (October 1) | 156.1 → 144.6 s | 91.6 → 99.1 |
| native int-dot Q8_0 GEMVs (October 1) | 144.8 → 139.7 s | 99.0 → 101.2 |
- A shared prompt is read once. When several agents arrive with the same system prompt and tools, the first one reads it and the others wait for it and restore it (the prefill FIFO, now on by default for the 35B-A3B class). In the replay's wave, all 15 agents used to read the 12.3K-token prompt themselves and none had finished a reply after 142 s; now 14 of 15 restore 12,279 tokens and the wave's 30 requests finish in 69 s (September 30; 53.2–58.4 s on the October 1 builds).
- A conversation resumes in its lane. Each lane keeps a checkpoint of its conversation and restores the next turn in place; the prompt cache is sized for the lanes within a VRAM budget, and a conversation's new snapshot replaces its older ones. Worker follow-ups that restored their previous turn: 6 of 30 before, 30 of 30 now; the worker phase reads 87.8K new tokens instead of 126.6K. A new user query after a tool loop restores up to the previous query's end (373 cached tokens instead of 0 in a five-round test).
- Long prompts prefill faster at depth. Tiled prefill attention from 6,144 positions: a cold 55,923-token prompt in 94.5 s instead of 200.4 s; a follow-up with 1,164 new tokens at 56K depth in 6.38 s instead of 11.70 s. Not bit-identical: perplexity 2.3529 against 2.3525 (+0.017 %), needles 6 of 6 at 32K and 54K.
- Both cards busy during a deep prefill. A lane's next piece enters card 0 while card 1 runs the piece before it. A 51,459-token prompt beside a decoding lane: first token in 64.1 s instead of 119.3 s (×1.86), the cards 93 / 96 % busy instead of 50 / 51 %, the lane decoding beside it at 18.1 tok/s instead of 10.9, with the same bytes.
- Short requests go first. A prompt with at most 16,384 tokens left to read goes ahead of a long prompt's next piece; the long prompt waits for at most 2 short pieces or 10 s. With an 80K-token lead prefilling and six ~2.2K-token workers arriving 2 s later, the workers' first tokens came at 53 s; with the lane pipeline alone at 208 s, with neither at 267 s. The lead's own first token came at 175.9 s, against 208.0 s and 267.7 s.
- Workers keep writing during a deep prefill. A long prompt that started alone in 8,192-row pieces is cut into 512-row pieces once another lane decodes or reads a short prompt (re-cut), and its next piece waits until every decoding lane has made 32 steps, for at most 2 s (the decode quota).
The same lead and six workers (--parallel 16 --ctx 262144 --thinking on, a 79,952-token lead, October 1):
| workers' replies | build or setting | workers' first token | workers' decode while the lead prefills | workers done | lead's first token |
|---|---|---|---|---|---|
| 69–319 tokens | lane pipeline and short-first only | 53.2 s | 0.09 tok/s each | 176.8 s (median) | 175.8 s |
| 69–319 tokens | v0.2.6 defaults | 12.6 s | 20.1 tok/s each (median) | 18.6 s (median; the last at 29.5 s) | 169.8 s |
| 1,200 tokens | lane pipeline and short-first only | 34.2 s | 0.09 | 202.7 s | 175.9 s |
| 1,200 tokens | v0.2.6 defaults | not recorded | 11.6 | 116.1 s | 214.2 s |
| 1,200 tokens | IE_Q35MOE_DECODE_QUOTA=0 |
12.7 s | 0.51 | 187.5 s | 162.4 s |
The short-reply row for the defaults was measured with those values set by environment, on the build before they became the defaults. The quota is a trade: with short, tool-call-sized replies it costs the lead nothing (169.8 s against 175.8 s); with long worker replies the lead's prefill takes 214 s, 22 % more than before and 32 % more than with the quota off, and all the work is done at 215 s instead of 190 s.
Tool calls and reasoning in the OpenAI fields
- XML tool calls come back as structured
tool_calls. Models whose GGUF chat template teaches the<function=XML tool form (Qwen3.8-27B, the 35B-A3B class) get that template's own tools block and history form, and their calls are parsed intotool_callswith parameters typed by the request's schema. A captured Dream explorer request went from 0 of 8 alone and 0 of 8 at once to 7 of 8 and 8 of 8 structured (35B-A3B class,--parallel 16, September 30).IE_QWEN_TOOLS_JSON=1restores the Qwen3 JSON preamble. - Calls are taken only from the answer. With thinking on, a call block the model quotes while it reasons is never a call; a function name the request does not list is rejected;
True,02and Python-quoted lists are typed by the schema. A reply that ends with a complete call without ever closing its reasoning is accepted as that call. - Thinking goes to
reasoning_contenton the Qwen thinking templates (Qwen3.8-Flash, Qwen3.8-27B, the 35B-A3B class), streamed and non-streamed alike;contentholds the answer only. A reply cut off before the model closes its reasoning is allreasoning_contentwith an emptycontent, as on DeepSeek, MiMo and GLM. With thinking off the bytes are unchanged. - Checked on October 1 on Qwen3.8-Flash (1 and 4 lanes), Qwen3.8-27B, the 35B-A3B class (16 lanes), MiMo-V2.6-Flash and DeepSeek-V4.1-Flash: one-call requests 4 of 4 structured on each, and requests that paste a tool-call snippet as text gave 0 calls.
Faster kernels
In the one-token Q8_0 GEMV, the engine's portable integer-dot helper compiled to 28 instructions per dot on the B70, 24 of them byte moves; the compiler's native dot takes 4. v0.2.6 moves the Q8_0 kernels of the 35B-A3B class, Qwen3.8-27B and Qwen3.8-Flash to the native dot, each in a new kernel beside the old one, with the same per-lane sums in the same order: the replies checked on all three models, and perplexity on the 35B-A3B class, are byte-identical with the switches on or off.
| kernel | measured | switch (=0 = the previous kernel) |
|---|---|---|
| Q8_0 MoE prefill, down and gate+up (35B-A3B class) | per launch at 2,048 rows: down 17.56 → 3.15 ms, gate+up 10.57 → 6.26 ms. One request's prefill, A-B-A: 2,038 tokens 2.20 → 1.41 s, 8,065 tokens 9.04 → 5.96 s, 16,369 tokens 22.72 → 16.44 s, 32,161 tokens 60.86 → 48.38 s | IE_Q8_MOE_DOWN_V2, IE_Q8_MOE_GATEUP_V2 |
| Q8_0 MoE decode and lane rows (35B-A3B class) | 16 lanes 390.6–391.3 → 414.1 tok/s together (+5.9 %, 25.9 per lane), 8 lanes 315.2–316.7 → 331.9, one request 80.8–81.1 → 81.9–82.4; A-B-A, --ctx 32768 --parallel 16 |
IE_Q8_MOE_DECODE_V2 |
| Q8_0 GEMVs, lane rows and one token (Qwen3.8-27B, 35B-A3B class, Qwen3.8-Flash) | Qwen3.8-27B 16 lanes 113.7–113.8 → 153.5 tok/s together (+34.9 %), 8 lanes 92.3 → 111.3; Qwen3.8-Flash 16 lanes 124.6–124.8 → 128.0 (+2.6 %); one request +0.4–0.6 %; A-B-A, --ctx 8192 --parallel 16 |
IE_Q8_SOA_BATCHED_V2, IE_Q8_SOA_GEMV_ND, IE_Q8_SOA_GEMV_G_ND |
| XMX decode attention at head size 256, from 4,096 tokens of context (27B and 35B-A3B splits) | decode at a 32,979-token prompt: 35B-A3B class 42.01 → 57.95 tok/s, Qwen3.8-27B Q8_0 13.75 → 14.56, Q6_K 16.68 → 17.89. Not bit-identical: perplexity moves by at most 0.002 nats | IE_Q35_XMX_DECODE |
| K/V prefetch and a wider combine pass in decode attention | at a 32,979-token prompt: 35B-A3B class 36.25 → 41.67 (prefetch) and 57.92 → 60.91 (combine), Qwen3.8-27B 12.69 → 13.65 and 14.57 → 14.82; same bytes | IE_FA2_VEC_PF, IE_FA2_TILE_PF, IE_FA2_COMBINE_SPLIT |
| shared-expert gate in one pass (35B-A3B class) | decode after a 1K-token prompt 62.0 → 82.6 tok/s; same bytes | IE_Q35MOE_SHEXP_GATE_V0=1 = the previous kernel |
| lane rows' decode attention in one launch per pass (27B and 35B-A3B splits) | 35B-A3B class 16 lanes 348.9 → 390.7 tok/s (+11.7 %), four 16K lanes +5.3 %; Qwen3.8-27B four 16K lanes +2.9 %; same bytes | IE_Q35_ROWS_ATTN |
The attention and shared-expert rows are from September 29–30, the int-dot rows from October 1. Kernel-level tables, the estimates that did not hold, and how to reproduce are in PERFORMANCE.md.
- Qwen3.8-27B shared-prefix cache on by default, cut on the model's 512-row prefill grid so single-request output does not change: four new agents behind a 13.6K-token shared prefix got their first tokens in 5.6–8.5 s instead of 72–75 s.
IE_QWEN35_SHARED_PREFIX=0turns it off.
Serving
- The engine picks the lane count.
ie servewithout--parallel, or with--parallel auto, sizes the request lanes at load and logs why ([lanes] auto ...);/propstotal_slotsand/healthparallelreport the pick, and--max-queuedefaults to the pick + 8. Measured picks (October 1): the 35B-A3B class at--gpus 2 --ctx 262144takes 16 lanes of 32,768 positions (9.44 GiB free per card against 8.68–8.69 needed); Qwen3.8-27B Q8_0 takes 16 at--ctx 8192, 12 at--ctx 16384and 1 at--ctx 32768; MiMo-V2.6-Flash, DeepSeek-V4.1-Flash and Qwen3.8-Flash take a fixed 4 lanes of 16,384 positions. One card,--int8-kv,ie run, a switch that refuses the lanes, and every other architecture give 1. The replay ran the same with the pick as with an explicit 16 (167.2 s against 166.7 s). /propsreportsslot_ctx: the positions each lane after the first holds (lane 0 holdsn_ctx), 0 with one lane, so a client can size a sub-agent's context to its lane instead of to the lead's window.- A stop is fast and safe. SIGTERM, SIGINT or
POST /admin/shutdownends every request at once, drops the queued work and waits only for the steps already on a card; a running 35B-A3B prefill piece stops at its next layer. The log ends with[ie] stopped in X.X s (waited for N in-flight GPU steps).
| stop request to process exit | before | v0.2.6 |
|---|---|---|
| 4 lanes, four ~18.8K-token prompts mid-prefill | 14.58 s | 0.43 s |
16 lanes at --ctx 262144, a 79,738-token prompt mid-prefill beside six ~15.5K-token prompts |
49.70 s | 1.08 s |
| 16 lanes, the 80K lead and six workers, on the release's defaults | — | 0.45–0.53 s |
| Qwen3.8-27B, 4 lanes mid-prefill | — | 0.82 s |
| Qwen3.8-Flash, 4 lanes mid-prefill (it waits for the running chunk) | — | 3.08 s |
Every stop left 0 GPU fault lines, and a server started right after each 35B-A3B stop loaded and answered. The bound is one layer of the running prefill piece, measured down to 43K tokens of depth.
Native Q6_K and Q5_K on the 27B and 35B-A3B splits
- Qwen3.8-27B (two-card split): Q6_K decodes at 22.0 tok/s instead of 5.2, and a Q5_K_M file, which did not load, at 24.0 tok/s; Q8_0 is unchanged at 17.2 with the same bytes (one request, September 29).
- 35B-A3B class: Q6_K / Q5_K dense weights and experts run from the file's own bits instead of being requantized to Q8_0 at load, row batching included. On a test file requantized from the Q8_0 model to pure Q5_K: decode 62.1 → 70.1 tok/s, a 3.4K-token prefill 867 → 934 tok/s, perplexity 5.9314 against 5.8671 for Q8_0. A published Q6_K or Q5_K_M file of this class was not measured.
IE_QWEN35_SPLIT_KQ=0keeps the previous paths on both.
The models it is tuned for
| model | prefill | decode |
|---|---|---|
| Qwen3.6-35B-A3B class (Q8_0), new in this table | one request, cold: 2K tokens in 1.41 s, 8K in 5.96 s, 32K in 48.4 s (October 1) | 85.7 tok/s on a short prompt, 60.9 at a 33K-token prompt; 16 requests at 414.1 tok/s together, 25.9 each (October 1) |
| Qwen3.8-27B (Q8_0) | 945 tok/s at 2K (August 15) | 24.5 tensor-parallel + speculative (August 26); 16 requests at 153.5 tok/s together with --ctx 8192 lanes (October 1; v0.2.0 measured 108.0 with --ctx 16384 --slot-ctx 4096); Q6_K 22.0 and Q5_K_M 24.0 on the two-card split |
| MiMo-V2.6-Flash, DeepSeek-V4.1-Flash, DeepSeek-V4-Flash, GLM-5.3-Flash, Qwen3.8-Flash | unchanged from v0.2.0 | unchanged from v0.2.0; Qwen3.8-Flash 16 requests at 128.0 tok/s together with --ctx 8192 lanes (October 1; v0.2.0 measured 118.4 with --ctx 32768) |
Conditions and methods are in the README's Benchmarks section. These come from one machine; your hardware will differ.
Upgrading from v0.2.0
--paralleldefaults toauto. A load without--parallelnow picks its request lanes instead of serving one request at a time. On MiMo-V2.6-Flash, DeepSeek-V4.1-Flash and Qwen3.8-Flash that is 4 lanes of 16,384 positions, taken out of the expert cache, with the lanes' documented differences from one lane: MiMo-V2.6-Flash and Qwen3.8-Flash refuse images above one lane, MiMo's prompt lookup is off, and DeepSeek-V4.1 withIE_DS41_PROFILE_OUTset stays at 1. Pass--parallel 1for the one-request server of v0.2.0. An explicit--parallel Nloads exactly as before.--max-queuedefaults to the pick + 8 underautoand stays 8 beside an explicit--parallel. A layout file'sparallelkey takes a number only (leave it out for the pick), andie capabilitiesstill reportsdefaults.parallelas 1.- Thinking arrives in
reasoning_contenton Qwen3.8-Flash, Qwen3.8-27B and the 35B-A3B class with thinking on. A client that read the reasoning out ofcontentmust readreasoning_content; stop sequences now count from the start ofcontent. History sent back with the reasoning insidecontent(a session saved before the upgrade) still works. - Tool calls on Qwen3.8-27B and the 35B-A3B class are prompted with the template's XML tool form and returned as structured
tool_calls.IE_QWEN_TOOLS_JSON=1restores the JSON preamble. - The prefill FIFO is on by default for the 35B-A3B class (
IE_LANES_PREFILL_FIFO=0turns it off); it stays opt-in (=1) on Qwen3.8-Flash and Qwen3.8-27B. On the 35B-A3B class the lane pipeline, short-first, 512-row pieces, re-cut, the decode quota, sticky lanes and the lane-sized prompt cache are all on by default; each has its own switch (below). - The Qwen3.8-27B shared-prefix cache is on by default (
IE_QWEN35_SHARED_PREFIX=0turns it off). - Output at long context changes on the 27B and 35B-A3B splits. The tiled prefill attention (from 6,144 positions, 35B-A3B class) and the XMX decode attention (from 4,096 tokens of context) are not bit-identical to the kernels they replace;
IE_Q35MOE_NO_FA2_TILE(set to any value) andIE_Q35_XMX_DECODE=0restore the previous bytes. On the 35B-A3B class, a prompt longer than 512 tokens that arrives beside running lanes is read in 512-row pieces and can give other bits than the same prompt alone. - A stop no longer waits for queued work on the 35B-A3B class, Qwen3.8-27B and Qwen3.8-Flash lanes: requests end with finish reason
abort. Do not kill a server that still has work on a card. - The first request after the upgrade compiles the new kernels once: 4.87 s to the first token instead of 0.29–0.35 s on the 35B-A3B class, 6.95 s instead of 1.85 s on Qwen3.8-Flash. The result is cached on disk.
/propsgainsslot_ctxand/healthgains lane counters (lookaheads,short_bypass,short_guard,short_guard_wait,recuts,quota_holds,quota_guard,snapshots,drains,drain_ms,drain_max_ms). Existing fields are unchanged.
New switches
| switch | default | what the other value does |
|---|---|---|
--parallel |
auto: the load picks the lanes |
--parallel N (1–16): exactly N, as in v0.2.0 |
IE_LANES_PREFILL_FIFO |
on for the 35B-A3B class, off on the other lane models | =0 every waiting prompt's pieces at once; =1 turns it on for Qwen3.8-Flash and Qwen3.8-27B |
IE_Q35MOE_STICKY_LANES |
on | =0 no per-lane checkpoint |
IE_Q35MOE_LANES_CACHE |
on: up to max(12, 2 × lanes + 8) entries within a VRAM budget | =0 12 entries, no budget; IE_PROMPT_CACHE_MAX_ENTRIES and IE_PROMPT_CACHE_VRAM_MIB set the limits |
IE_Q35MOE_CACHE_SUPERSEDE |
on | =0 a conversation keeps its older snapshots |
IE_Q35MOE_ANCHOR |
on | =0 no snapshot kept at the last user query |
IE_Q35MOE_NO_FA2_TILE |
unset: tiled prefill attention from IE_Q35MOE_FA2_TILE_MINCTX (6144) positions |
set to any value: the previous kernel and output |
IE_Q35MOE_LANE_PIPELINE |
on | =0 one piece of a lane in the pipe at a time |
IE_Q35MOE_SHORT_FIRST |
on, with IE_Q35MOE_SHORT_ROWS 16384, IE_Q35MOE_SHORT_SLOTS 2, IE_Q35MOE_SHORT_WAIT_MS 10000 |
=0 the FIFO window and arrival order alone |
IE_Q35MOE_MIX_CHUNK |
512 rows | =<rows> another piece size; =0 the plan's 8,192-row pieces, which also turns re-cut off |
IE_Q35MOE_RECUT |
on | =0 a prompt that started alone keeps its pieces |
IE_Q35MOE_DECODE_QUOTA |
32 steps, with IE_Q35MOE_QUOTA_MAX_MS 2000 |
=0 off: the most throughput, agents stall while a deep prompt prefills |
IE_Q8_MOE_DOWN_V2, IE_Q8_MOE_GATEUP_V2, IE_Q8_MOE_DECODE_V2 |
on | =0 the previous MoE kernels, same bytes |
IE_Q8_SOA_BATCHED_V2, IE_Q8_SOA_GEMV_ND, IE_Q8_SOA_GEMV_G_ND |
on | =0 the previous GEMV kernels, same bytes |
IE_Q35_XMX_DECODE |
on from IE_Q35_XMX_DECODE_MIN (4096) tokens of context |
=0 the previous decode attention and its bytes |
IE_FA2_VEC_PF, IE_FA2_TILE_PF, IE_FA2_COMBINE_SPLIT |
8, on, 16 | =0 each: no prefetch, the original combine kernels |
IE_Q35_ROWS_ATTN |
on | =0 the lane rows' decode attention one row at a time |
IE_Q35MOE_SHEXP_GATE_V0 |
unset | =1 the previous shared-expert gate kernel |
IE_QWEN35_SHARED_PREFIX |
on (Qwen3.8-27B) | =0 off |
IE_QWEN35_SPLIT_KQ |
on | =0 Q6_K / Q5_K through the previous fallbacks |
IE_QWEN_TOOLS_JSON |
unset | =1 the Qwen3 JSON tool preamble |
IE_LANES_TRACE, IE_DECODE_PROF |
off | =1 one log line per card step; a per-token decode breakdown (27B and 35B-A3B splits) |
Requirements
Intel oneAPI 2026.x and an Intel Arc GPU. The agent-swarm configuration needs two Arc Pro B70 cards (64 GB VRAM); the largest models want about 256 GB of system RAM. Measured on Intel compute-runtime 26.35.39758.10 with the kernel's xe driver, the stack v0.2.0 shipped on.
Getting started
source scripts/env.sh
cmake -S . -B build -G Ninja && cmake --build build -j
./build/src/ie serve <35B-A3B-Q8_0.gguf> --gpus 2 --ctx 262144 --thinking on # no --parallel: the load picks the lanes
curl -s localhost:11435/props # total_slots and slot_ctxie <command> --help lists every option. QUICKSTART.md has the container route; the prebuilt image dates from July 2026 and predates this release, so build from source. The Dream Agent Harness drives this engine as a local agent, with its sub-agents on the request lanes.
Known limitations
- Many agents reading at once. With 15 agents that each append 1–3K tokens of tool output per turn, an agent decodes at a median 8.1 tok/s (5th percentile 6.6), against 25.9 tok/s per agent when 16 only write. The replay's conversations stay under about 8K tokens and it has no lead. A live session is slower still: with a lead that started 15 file-reading workers in one reply (the Dream Agent Harness, October 2, one session, the build before
slot_ctx), the 57 replies of the first ten minutes read 145K new prompt tokens and decoded at a median 6.1 tok/s; the 37 that ended with 15 lanes busy at a median 4.0 (slowest 3.2), and replies with one lane busy at 61.8. - A deep prefill beside workers that write long replies is about 22 % slower than before (the decode quota).
IE_Q35MOE_DECODE_QUOTA=0reverts it. - The new scheduling is for the 35B-A3B class only. On Qwen3.8-27B, Qwen3.8-Flash, MiMo-V2.6-Flash and DeepSeek-V4.1-Flash a long new prompt still slows or pauses the other lanes while it prefills. The Qwen3.8-27B gets no lanes from the pick at
--ctx 32768with the prompt cache on (12 at--ctx 16384, 16 at--ctx 8192). - A default load of MiMo-V2.6-Flash or Qwen3.8-Flash has 4 lanes, and their lanes do not take images. Pass
--parallel 1for image input. - Tool calls: a complete call block that the model writes in its answer is taken as a call even when prose follows it, so a prompt that asks the model to reproduce a call snippet can produce a call.
- Stops deeper than 43K tokens were not measured; the bound grows with the depth of the running prefill piece. DeepSeek-V4.1-Flash and MiMo-V2.6-Flash stop as in v0.2.0, at the next token.
- DeepSeek-V4.1-Flash greedy replies can differ between two server sessions of the same build (seen with the previous build too; within a session both builds agree). The cause is not established.
- The 35B-A3B-class figures (and the Qwen3.8-27B's 16-lane and Q6_K / Q5_K figures) come from community fine-tunes, and from one run each unless marked A-B-A; one build of the replay measured 146 s and 158.0 s in two sessions.
- Verified on Arc Pro B70. Other Arc cards are untested by the maintainer. The limitations listed for v0.2.0 still apply where this release does not say otherwise.
- This is a 0.x release: interfaces may change.
Full changelog: v0.2.0...v0.2.6