mlx-omarchy

MLX on Apple GPU under Linux.

MLX is Apple's array framework. Upstream it runs on Metal. mlx-omarchy is the Omarchy GPU backend that keeps import mlx.core as mx and mx.gpu on Apple Silicon Linux through Mesa's Honeykrisp Vulkan 1.4 stack. There is no Metal. GPU work never falls back to CPU tensors.

This repo is a patch-set and overlay, not a hard fork of MLX history. Upstream source is fetched by pin (mlx.lock + scripts/prepare-mlx.sh); project files live under overlay/, and edits to upstream files stay in a small patches/ series. The Python module name remains mlx. Do not install upstream mlx beside this wheel.

Open defects: docs/known-defects.md.

One-command install on an M1 running Omarchy, first model download, streamed answer with measured tokens/sec, and the launcher entry — 2:47, unedited. Also at joshuaswarren.github.io/mlx-omarchy.

On Omarchy (Apple Silicon), one command installs the release wheel into a private venv under ~/.local/share/mlx-omarchy, puts mlx-omarchy, mlx-omarchy-demo, and mlx-omarchy-parakeet on ~/.local/bin, and registers MLX Chat (Apple GPU) in the Omarchy launcher. It never replaces Mesa or edits Omarchy package files.

curl -fsSL https://raw.githubusercontent.com/joshuaswarren/mlx-omarchy/main/install.sh | bash

Uninstall with bash install.sh --uninstall. Latest release: v0.7.4 (prerelease: installed-from-release gates pending, see the release notes). Wheel filenames carry the build commit; pin the exact URL and check SHA256SUMS on the release.

Manual install (or any other Linux box):

# Apple Silicon (Honeykrisp) — Python 3.14 + aarch64 wheel from the latest release
python3 -m venv ~/.venvs/mlx
~/.venvs/mlx/bin/pip install <cp314 linux_aarch64 wheel URL>

# x86_64 dev box (software Vulkan, no ANE) — Python 3.11 + cp311 wheel
python3 -m venv ~/.venvs/mlx
~/.venvs/mlx/bin/pip install <cp311 linux_x86_64 wheel URL>

mlx-lm depends on upstream mlx, so install it with pip install --no-deps mlx-lm and add its own deps as install.sh does. Build-from-source and the Honeykrisp fork driver (required for verified M1 Omarchy numbers — stock Mesa is not enough): docs/install-omarchy.md.

import mlx.core as mx

x = mx.array([[1.0, 2.0], [3.0, 4.0]])
w = mx.array([[0.5], [0.25]])

def loss(w):
    return mx.exp(x @ w).sum()

value, grad = mx.value_and_grad(loss)(w)
print(value, grad)

Text generation: the qualified model is mlx-community/Qwen3.8-27B-4bit (metadata verified 2026-09-20: Apache-2.0, ungated; tested revision 10c35caafbb80f7dc6a7a432cdd11af10a6d4818). The verified smoke ran on an M2 Max with candidate wheel 0.32.3.dev202609201346+a1251aaa, mlx-vlm 0.7.1 and mlx-lm 0.31.3 — this candidate is not yet a published release. From this checkout, in a Python 3.14 environment containing that wheel (installed with --no-deps):

python -m pip install --no-deps -r receipts/2026-09-20-qwen38-text-install/requirements.txt
env -u MLX_DISABLE_COMPILE python -m mlx_vlm.generate \
  --model mlx-community/Qwen3.8-27B-4bit \
  --max-tokens 32 --prompt "Say READY."

The receipt and raw output record exit 0 and READY. with compilation enabled. The checkpoint downloads ~15 GB of weights; a 16 GB M1 has not passed this model's memory and generation gates.

Local OpenAI-compatible serving (catalog CLI + qualification limits): docs/serve.md. Kernel / serve-patch flags: docs/kernel-flags.md. Serving tooling in this tree is not all in a published release yet — docs/serve.md marks what is qualified.

Verified on Omarchy with Mesa Honeykrisp / Vulkan 1.4:

Apple GPU and Apple ANE are separate lanes. GPU qualification does not qualify ANE. Later SoCs follow. Receipts and dated detail live under receipts/ and in joshuaswarren/ane-linux-experiments.

Qwen3.8-2B (4-bit mlx) on Apple M-series: upstream Metal on macOS vs omarchy Vulkan on Asahi/Omarchy. Protocol for every cell: greedy (temperature 0), 32 new tokens, 2 warmups, 512-token pure-prefill leg; decode is the median of 30 measured runs. Rows are marked by source build — † ‡ § are three different builds and two battery protocols, so rows with different marks are not comparable:

Receipt per row:

• † M1 / M1 Max Linux: v0.7.2 release receipt (mirrored at the same path in ane-linux-experiments) — records these exact rows on the tag build, wheel mlx_omarchy-0.32.3.dev202609221309+fa103c86, 10-prompt × 3-pass subset of the standing battery, ordered-record digests ac1b2695… identical on both hosts. The M1 row is the Honeykrisp fork-driver leg (stock Mesa measured 32.9 pure-prefill tok/s on the same wheel); Omarchy installs must use the fork driver (docs/install-omarchy.md, "Honeykrisp driver with the fork fixes").
• ‡ M2 Max Linux: the 2026-09-21 100-prompt-corpus battery on wheel 0.32.3+5b18306 — public matrix (row "M2 Max", adapter Apple M2 Max G14C). Raw per-run receipt qwen38-2b-m2-omarchy-firstpass.json (label M2Max-T6021-Omarchy-q4-2B-stable-corrected, ordered-records digest 6f21e665…), kept outside the repository.
• § macOS rows: upstream mlx 0.32.2 Metal, idle login-window runs from the same 2026-09-21 battery. Raw per-run receipts outside the repository: qwen38-2b-m1-macos-idle.json (label M1-T8103-macOS-q4-2B-idle-corrected), qwen38-2b-t6001-macos-firstpass.json (label M1Max-T6001-macOS-q4-2B-corrected), qwen38-2b-m2-macos-repro.json (label M2Max-T6021-macOS-q4-2B-corrected-repro — the idle rerun that superseded the load-contaminated first pass). All three share ordered-records digest 301c4fc3…: upstream Metal is deterministic across these hosts.

Cross-OS token identity was never an acceptance bar; the bar is logit-level equivalence plus coherent decoding. Adapter evidence is captured per run (Vulkan loader trace naming the Apple physical device on Linux; mlx device identity on macOS). The backend refuses non-Apple GPUs by default.

Three-laptop parity battery — GPU Qwen3.8-2B (this repo's Honeykrisp stack), ANE whole encoder, Parakeet — against same-SoC macOS denominators. Bar:

=1.00x macOS. No GPU or Parakeet cell meets the bar yet; the M1 Qwen ANE staged cells do (omarchy-ane driver, separate stack: decode 1.49x, TTFT 0.84x, e2e 0.69x, prefill-512 1.223x — see the experiments parity matrix).

m1-host GPU numbers were measured on this repo's main tree 024d4fe60 (records pin dbf704971617fdfc, bit-identical across m1-host and m1max-host). The full matrix with per-cell receipts and unreceipted-value marks lives in joshuaswarren/ane-linux-experiments.

2026-09-25 m1-host addendum (lean lane receipt 2026-09-25-jwm1-parity2-parakeet-lean): the 1588-1598 ms figure pays per-process kernel compile + 628 ms ANE session open every rep, while the macOS denominator amortizes CoreML load across rep10; on the matched in-process boundary the same laptop runs 935.8 ms warm median (hidden content sha 51830b6ffe992568 bit-exact vs the certified pins). Remaining named buckets: TDT per-call host glue (~1.45 ms x 265 calls), encoder submit overhead (293 vs 143 engine), mel 63 ms, q4 GEMV kernel efficiency — a 54.2 GB/s measured read rate disproves the 43 GB/s decode bandwidth wall on this DRAM.

Archival Qwen2.5 tables and older batteries stay in git history / linked receipts — they are not the current recommendation. Current text-generation guidance: docs/serve.md.

Value-tested Mac-usable primitives: 126 / 130 (2026-09-10, docs/coverage.json). Upstream MLX C++ on GPU: 251 / 251; Python: 11,483 / 11,847 (2026-09-11 snapshot in receipts/2026-09-11-upstream-suite/). Standing battery closed 30 / 30 at 8790c463.

Known gaps include ReduceScatter on the Linux ring transport and fast.CustomKernel remaining a Metal subset. Everything else: docs/known-defects.md.

ANE is an internal accelerator for static graph regions, not a user-facing mx.ane device. The wheel ships Parakeet reference encoder paths on ANE where qualified (M1 / M1 Max). mlx-omarchy-parakeet download / transcribe are on PATH after aarch64 install.

Plans and contracts: docs/plans/2026-09-12-coreml-parakeet-ane-plan.md, docs/ane-bundles.md. Driver / libane ABI live in joshuaswarren/omarchy-ane.

The most useful thing an M-series owner can do is submit a redacted hardware report: docs/contribute-data.md. Quick capability capture (no install required for the light path):

git clone https://github.com/joshuaswarren/mlx-omarchy.git
cd mlx-omarchy
python3 scripts/collect_quick.py

Code contributors: start at docs/CONTRIBUTOR-GUIDE.md. Dev machines without Apple GPU set MLX_OMARCHY_ALLOW_NON_APPLE=1 (software Vulkan). GPU kernel changes still need Apple hardware before release.

• docs/roadmap.md
• docs/compatibility.md
• docs/architecture.md
• AGENTS.md — agent contract for this repo

MIT. Prepared MLX source keeps Apple's MIT license and copyright notices. Not affiliated with Apple.