Skip to content

v0.0.7

Choose a tag to compare

@zouyee zouyee released this 13 Nov 12:01
· 92 commits to main since this release
e0acb38

Highlights

Qwen3.8 vision (image input) — ViT-27B tower (bf16) with mx.compile (-18-25% on large images), stb_image preprocessing + smart_resize, MRoPE position ids with interleaved frequency selector, embed merge + rope_delta decode offsets. CLI --image and server image_url, oracle-verified against mlx-vlm.

Server API surface — GET /v1/models, /v1/tokenize, /v1/detokenize, count_tokens, /v1/embeddings (last-hidden pooling), /metrics (Prometheus text), /v1/completions, responses store + /api/status, minimal /v1/responses compatibility layer, per-request top_p/top_k, tool_choice.

Stability — SSE abort UAF fixes (responses stream + RequestState teardown ordering), segfault on aborted streams, completions OOM leak, loader error-path leaks + double free, incremental streaming detokenizer (no more lost spaces), tool_calls JSON escaping.

Perf investigations (measured; opt-in or closed) — FlashDecoding-style split-KV SDPA for multi-row verify attention; in-place KV slab append (neutral, off); 16K prefill re-investigation (no pathology); MTP acceptance estimator Phase 1 (early-exit refuted); ANE/CoreML FP16 GEMM spike (closed).

Full Changelog: v0.0.6...v0.0.7