OmlX
oMLX is a native macOS LLM inference server built on Apple's MLX framework, managed from the menu bar. It targets Apple Silicon's unified memory architecture to run large language models locally with low latency.
- Paged SSD KV caching: Offloads the KV cache to SSD when VRAM is insufficient. Documented to reduce agent time-to-first-token from 30-90 seconds to under 5 seconds.
- Continuous batching: Serves multiple concurrent inference requests without queuing them serially.
- API compatibility: Exposes an OpenAI- and Anthropic-compatible HTTP API, making it a drop-in local backend for tools that already support those providers.
- Menu bar management: Start, stop, and monitor from the macOS system tray without a terminal.
GitHub: jundot/omlx. Created February 13, 2026. 17,042 stars, Apache-2.0, Python. 611 open issues as of June 2026 -- a high issue count relative to age, suggesting active development and an incomplete feature surface.
Placed in Tools/Assess because the SSD KV caching approach to the TTFT problem on Apple Silicon is technically sound and fills a real gap. Center position reflects that the high open issue count and recent creation date leave meaningful uncertainty about stability. No personal production use to date.
Source: https://github.com/jundot/omlx
