Repository navigation
Releases
v0.0.8
Full Changelog: v0.0.7...v0.0.8
v0.0.7
Highlights
Qwen3.8 vision (image input) — ViT-27B tower (bf16) with mx.compile (-18-25% on large images), stb_image preprocessing + smart_resize, MRoPE position ids with interleaved frequency selector, embed merge + rope_delta decode offsets. CLI --image and server image_url, oracle-verified against mlx-vlm.
Server API surface — GET /v1/models, /v1/tokenize, /v1/detokenize, count_tokens, /v1/embeddings (last-hidden pooling), /metrics (Prometheus text), /v1/completions, responses store + /api/status, minimal /v1/responses compatibility layer, per-request top_p/top_k, tool_choice.
Stability — SSE abort UAF fixes (responses stream + RequestState teardown ordering), segfault on aborted streams, completions OOM leak, loader error-path leaks + double free, incremental streaming detokenizer (no more lost spaces), tool_calls JSON escaping.
Perf investigations (measured; opt-in or closed) — FlashDecoding-style split-KV SDPA for multi-row verify attention; in-place KV slab append (neutral, off); 16K prefill re-investigation (no pathology); MTP acceptance estimator Phase 1 (early-exit refuted); ANE/CoreML FP16 GEMM spike (closed).
Full Changelog: v0.0.6...v0.0.7
v0.0.6
Highlights
- README redesigned around verified flagship paths — Qwen3.8 MTP engine (decode +19.7%/+13.8% over omlx-mtp, 3x faster TTFT) and the DSV4 284B-on-48GB expert-streaming path, with claims trimmed to what is reproducibly measured.
- CI unblocked against mlx-c 0.8.0 — Homebrew's mlx-c 0.8.0 added a
global_scaleparameter tomlx_gather_qmm, breaking the pinned bindings. Bumps mlx-zig to v0.0.4, which dispatches on the imported function arity at comptime and compiles against both mlx-c 0.7.x and 0.8.x.
Full Changelog: v0.0.5...v0.0.6