Repository navigation
v0.0.7
Highlights
Qwen3.8 vision (image input) — ViT-27B tower (bf16) with mx.compile (-18-25% on large images), stb_image preprocessing + smart_resize, MRoPE position ids with interleaved frequency selector, embed merge + rope_delta decode offsets. CLI --image and server image_url, oracle-verified against mlx-vlm.
Server API surface — GET /v1/models, /v1/tokenize, /v1/detokenize, count_tokens, /v1/embeddings (last-hidden pooling), /metrics (Prometheus text), /v1/completions, responses store + /api/status, minimal /v1/responses compatibility layer, per-request top_p/top_k, tool_choice.
Stability — SSE abort UAF fixes (responses stream + RequestState teardown ordering), segfault on aborted streams, completions OOM leak, loader error-path leaks + double free, incremental streaming detokenizer (no more lost spaces), tool_calls JSON escaping.
Perf investigations (measured; opt-in or closed) — FlashDecoding-style split-KV SDPA for multi-row verify attention; in-place KV slab append (neutral, off); 16K prefill re-investigation (no pathology); MTP acceptance estimator Phase 1 (early-exit refuted); ANE/CoreML FP16 GEMM spike (closed).
Full Changelog: v0.0.6...v0.0.7