Skip to content

docs(mlx): add Apple Silicon execution backends section#28

Merged
jasperf merged 1 commit into
mainfrom
docs/mlx-execution-backends
Jul 2, 2026
Merged

docs(mlx): add Apple Silicon execution backends section#28
jasperf merged 1 commit into
mainfrom
docs/mlx-execution-backends

Conversation

@jasperf

@jasperf jasperf commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds an "Execution Backends on Apple Silicon (Metal / MLX / ANE)" section to docs/MLX-RUNTIME.md, answering a recurring conceptual question: whether there are models that run "on CPU" or better on Mac Silicon, as opposed to the RAM+GPU pattern.

The section clarifies:

  • Unified memory means "RAM vs GPU" isn't a PC-style split — the GPU does the math on weights living in shared RAM.
  • There's no separate class of "CPU model" — it's the same models run through a different backend.
  • Compares the three backends in a table: Metal/GGUF (default, what the "spills X% to CPU" notes describe), MLX (Apple's unified-memory framework, the most optimized practical path), and the Neural Engine via Core ML (not viable for LLM generation today, due to the growing KV-cache of autoregressive decoding).
  • Notes CPU-only inference only makes sense when weights don't fit the GPU ceiling.

Placed right after the Overview since it's foundational framing the rest of the doc builds on.

Changelog

Released as [1.6.2] — 2026-07-02.

Explain that unified memory means there's no PC-style RAM/GPU split and
no separate class of CPU model — the same models run through different
backends. Compare Metal/GGUF, MLX, and the Neural Engine (Core ML), and
note the ANE isn't viable for LLM generation today.

Release as 1.6.2.
@jasperf jasperf merged commit 53c2b2e into main Jul 2, 2026
@jasperf jasperf deleted the docs/mlx-execution-backends branch July 2, 2026 08:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant