Skip to content

Releases

v0.0.8

Choose a tag to compare

@github-actions github-actions released this 14 Nov 02:51

Full Changelog: v0.0.7...v0.0.8

v0.0.7

Choose a tag to compare

@zouyee zouyee released this 13 Nov 12:01
e0acb38

Highlights

Qwen3.8 vision (image input) — ViT-27B tower (bf16) with mx.compile (-18-25% on large images), stb_image preprocessing + smart_resize, MRoPE position ids with interleaved frequency selector, embed merge + rope_delta decode offsets. CLI --image and server image_url, oracle-verified against mlx-vlm.

Server API surface — GET /v1/models, /v1/tokenize, /v1/detokenize, count_tokens, /v1/embeddings (last-hidden pooling), /metrics (Prometheus text), /v1/completions, responses store + /api/status, minimal /v1/responses compatibility layer, per-request top_p/top_k, tool_choice.

Stability — SSE abort UAF fixes (responses stream + RequestState teardown ordering), segfault on aborted streams, completions OOM leak, loader error-path leaks + double free, incremental streaming detokenizer (no more lost spaces), tool_calls JSON escaping.

Perf investigations (measured; opt-in or closed) — FlashDecoding-style split-KV SDPA for multi-row verify attention; in-place KV slab append (neutral, off); 16K prefill re-investigation (no pathology); MTP acceptance estimator Phase 1 (early-exit refuted); ANE/CoreML FP16 GEMM spike (closed).

Full Changelog: v0.0.6...v0.0.7

v0.0.6

Choose a tag to compare

@zouyee zouyee released this 13 Nov 11:41
21c183a

Highlights

  • README redesigned around verified flagship paths — Qwen3.8 MTP engine (decode +19.7%/+13.8% over omlx-mtp, 3x faster TTFT) and the DSV4 284B-on-48GB expert-streaming path, with claims trimmed to what is reproducibly measured.
  • CI unblocked against mlx-c 0.8.0 — Homebrew's mlx-c 0.8.0 added a global_scale parameter to mlx_gather_qmm, breaking the pinned bindings. Bumps mlx-zig to v0.0.4, which dispatches on the imported function arity at comptime and compiles against both mlx-c 0.7.x and 0.8.x.

Full Changelog: v0.0.5...v0.0.6