omarchy-mlx

Sponsor this work or help stock the lab: joshuawarren.com/sponsors

MLX on the Apple GPU under Linux, and Core ML models on the Apple Neural Engine.

MLX is Apple's array framework for machine learning on Apple silicon. Upstream it speaks Metal, so it runs on macOS only. omarchy-mlx runs the same import mlx.core as mx code on an Apple silicon Mac running Omarchy Linux. The GPU driver is Mesa's Honeykrisp Vulkan stack. Tensors stay on the GPU. There is no Metal and no CPU fallback.

It is for people who run Linux on an M1 or M2 Mac. You get local chat, serving, and speech to text, with no macOS and no cloud account.

mlx-lm, oMLX, mlx-serve, TensorFold, sushi, llama.cpp (Vulkan) and MCDMA have been tested on an Apple silicon Mac under Linux. The status of each, with the commit or measurement behind it, is in docs/ENGINES.md.

• The mlx Python module with the Vulkan GPU backend. The distribution name is mlx-omarchy; the module stays mlx. Upstream source is fetched at a pinned commit, and this project's code lives in overlay/ and patches/. The tree stays a small patch-set, not a fork.
• MLX Chat, a local web app for chat and comparing options, with a browser UI and a terminal mode. The install registers it in the launcher and sets up a user service that keeps the loaded pair ready across logins.
• mlx-omarchy-serve, a serving CLI with catalog, plan, and serve commands, memory admission, and approve-first model downloads. The same install ships the Laya typed-decision server and the Bonsai2 packed-runtime server.
• Core ML on the Neural Engine. mlx-omarchy-parakeet transcribes audio (download, verify, transcribe). mlx-omarchy-coreml inspects a Core ML package. The aarch64 wheel ships the ANE worker and the pinned Parakeet encoder bundles.
• An opt-in wake word. mlx-omarchy-assistant --wake-word hey_jarvis listens through openWakeWord ONNX models on the CPU (SHA-256-pinned files, downloaded once on first use) and hands the microphone to the chat when it hears the phrase. It is off by default.

Chat models download from Hugging Face the first time you use them. You approve each download first. The default pairs: Everyday is Qwen3.5-9B 4-bit plus the Laya model. Compact is Qwen3-4B-Instruct-2507 4-bit plus Laya. Quality is Qwen3.8-27B 4-bit plus Laya, and it needs a 96 GB machine.

The install accepts every chip in the M1 class and the M2 Max. They share one GPU class and driver path. The Tested rows are the machines this project measures on. On an M3 or M4 Mac, run the tester kit, bash scripts/m3m4_kit.sh (see scripts/m3m4_kit.md), to send us the numbers and ANE state that only real silicon gives.

The ANE is a separate lane from the GPU. It has its own driver and its own tested state. Chip-by-chip ANE status, including what each untested chip needs, lives in the chip coverage table of omarchy-ane.

Not a supported target. With MLX_OMARCHY_ALLOW_NON_APPLE=1 the backend admits a Vulkan 1.3 device that exposes timeline semaphores and a compute queue, for development on a normal Linux box. The flag and the llvmpipe note are in Development override. Driver selection with VK_DRIVER_FILES is in Selection. The non-Apple refusal without the flag stays.

Observed, from PR #111, run record in receipts/2026-10-11-non-apple-gpu-run/README.md:

AMD (RADV) has no qualifying run.

Run one command on an Arch install for an M1-class Mac or an M2 Max. The script picks the latest release, checks the wheel against the release SHA256SUMS, and asks for Python 3.14 on aarch64.

curl -fsSL https://raw.githubusercontent.com/joshuaswarren/omarchy-mlx/main/install.sh | bash

The other ways to install are in docs/INSTALL.md.

See docs/INSTALL.md.

See docs/INSTALL.md.

See docs/INSTALL.md.

After any install path, this works:

mlx-omarchy-info                                                   # GPU and driver state
mlx-omarchy -c "import mlx.core as mx; print(mx.default_device())" # Device(gpu, 0)
mlx-omarchy-chat                                                   # chat in the browser

The first chat run asks you to approve a model download. Then it streams the answer from the Apple GPU.

See docs/USAGE.md.

Same MacBook Pro (M1 Max, 64 GB), same models, same prompts, MLX on both sides with the same mlx-lm (0.32.0), measured 2026-10-09 to 2026-10-11. The Linux column runs the mlx-lm patches the installer applies. Each cell is Linux / macOS and the percentage is Linux as a share of macOS: 100% means equal, higher is faster on Linux.

Decode is the median of 10 greedy generations and prefill is the median of 5 runs, except two Linux cells: gemma-4-E4B is the mean of three runs and Qwen3.6-35B-A3B the mean of two. Those two sets of runs started with a load average above 0.5, so the numbers may be low. The macOS cells are earlier sealed runs.

MLX 4-bit through the installer's mlx-lm patches, tokens per second, Linux / macOS:

Driver build: the two mixture-of-experts rows (gemma-4-26B-A4B and Qwen3.6-35B-A3B) and the gemma-4-E4B row ran on main at e57cb301 with the private Honeykrisp ICD at Mesa git 6543eeb7df7. For the other rows the driver build was not recorded. The full tables, per chip, with the wheel, the driver and the start load of each run, are in docs/BENCHMARKS.md. Differences from macOS and models that did not run are in docs/KNOWN-ISSUES.md.

Decode on the 13-inch M1 changed with the CPU governor with no ANE work running. See Set the CPU governor for local AI.

See docs/DRIVER-AND-KERNEL.md.

See docs/DRIVER-AND-KERNEL.md.

The fastest way to help is a capture from your machine. Owners of the untested chips above are the unblock for those rows. Both collectors print a fully redacted payload. Nothing is sent until you pass --submit.

git clone https://github.com/joshuaswarren/omarchy-mlx.git
cd omarchy-mlx
python3 scripts/collect_quick.py    # quick capture, Linux
python3 scripts/collect_deep.py     # deep capture, Linux and macOS

# add --submit to publish. It already targets the public endpoint; the
# deep run also needs --out FILE:
python3 scripts/collect_deep.py --out mlx-omarchy-deep.tar.gz --submit

On a dual-boot Mac, run collect_deep.py under macOS and under Omarchy, and submit both. The full guide, including what gets collected and how redaction works, is docs/contribute-data.md.

See docs/ANE-LINUX.md.

See docs/ANE-LINUX.md.

See docs/TROUBLESHOOTING.md.

Every bit of support helps keep omarchy-mlx alive and free. If you are able, sponsor on GitHub or send a Lightning donation to joshuaswarren@strike.me to directly fund continued development and new integrations.

If financial support is not an option, you can still make a big difference: star the repo, share it, or recommend it to a colleague. Word of mouth is how most people find omarchy-mlx.

Code contributions start at CONTRIBUTING.md. Hardware captures start at docs/contribute-data.md. Dev machines without an Apple GPU can run the module under software Vulkan. GPU kernel changes still need real hardware before a release. Any other Vulkan GPU (NVIDIA, Intel, AMD): the contributor kit is docs/ANY-GPU.md and the results so far are in docs/TESTED-HARDWARE.md.

See docs/RELEASING.md.

MIT, see LICENSE. Prepared MLX source keeps Apple's MIT license and copyright notices. Other bundled components keep their own notices under LICENSES. Not affiliated with Apple.