Skip to content

Mach X v0.1.0

Choose a tag to compare

View all tags
@Red-Weasel Red-Weasel released this 21 Sep 01:47
· 22 commits to main since this release
v0.1.0
17902ed

First tagged release of Mach X — a C++/SYCL LLM inference engine for Intel Arc GPUs, written from the metal up. No llama.cpp fork, no PyTorch, no vendor runtime.

Built and tuned on two Arc Pro B70 cards (64 GB VRAM), with host RAM holding the experts that do not fit.

The five models it is tuned for

model prefill decode
DeepSeek-V4.1-Flash (475 GB) 304 tok/s at 2K, 319 at 32K–223K 12.8 chat, 14.0 in agent loops
DeepSeek-V4-Flash (155 GB) 571 tok/s at 4K 26.1 at 4K, 32.7 short
GLM-5.3-Flash 156 tok/s at 16K 14.5 at 16K
Qwen3.8-Flash (104 GB) 467 tok/s pipelined 35.0 chat, 41.8 code
Qwen3.8-27B (Q8_0) 945 tok/s at 2K 24.5 tensor-parallel + speculative

Conditions and methods are in the README's Benchmarks section. These come from one machine; your hardware will differ.

What is in it

  • One OpenAI-compatible server (ie serve) with native tool calls, and image input on checkpoints that carry a vision tower.
  • A prefix cache that answers a follow-up turn in 1–2 s instead of re-reading the whole conversation.
  • Tensor-parallel multi-GPU, running models larger than one card.
  • Hand-written SYCL kernels — XMX matrix engines, int-dot quantized GEMV, tiled FlashAttention.
  • Dense, MoE and hybrid architectures: GLM-5.3, DeepSeek-V4 and V4.1, Qwen3.8-Flash-Next, Qwen3.6, Qwen3 / Coder / Tongyi, Qwen3-Next, Gemma-4, gpt-oss, and Llama-compatible dense models.

Requirements

Intel oneAPI 2026.x and an Intel Arc GPU — most of the engine only runs on one. A container path is in QUICKSTART.md if you want to try it without building.

Getting started

See the From source section of the README, or QUICKSTART.md for the container route. The Dream Agent Harness drives this engine as a local agent, including tool calls.

Known limitations

  • Verified on Arc Pro B70. Other Arc cards are untested by the maintainer.
  • Large-model configurations assume substantial host RAM — the 475 GB checkpoint wants around 256 GB.
  • GLM-5.2 and Tencent Hy4-preview have experimental standalone runners rather than the main path.
  • This is a 0.x release: interfaces may change.
Assets 2
Loading