Skip to content

vMLX 1.6.45

Choose a tag to compare

@jjang-ai jjang-ai released this 29 Aug 05:12
· 176 commits to main since this release

vMLX 1.6.45

Fixed

  • Qwen3.8 Flash Next tool calling. Bundles converter-stamped tool_parser=hermes (JSON-only) while the chat template emits Qwen <function=…><parameter=…> XML, so a share of sampled tool_choice=required calls failed closed with HTTP 400. The stale hermes stamp is now neutralized at every layer — engine registry, app launcher, and a one-time migration for previously saved sessions — so the family qwen parser (which accepts the hermes JSON body and every XML variant) always wins. Explicitly chosen non-hermes parsers and all other families are untouched.
  • Adaptive native-MTP depth policy. The app was silently pinning an explicit depth 1 under the Adaptive setting, and the per-request controller restarted its whole AR-vs-MTP experiment on every request. Adaptive now starts at plain AR for unknown workloads (no first-turn tax), activates MTP immediately only from measured evidence (a validated vmlx_mtp_tuning.json record or an in-session profile that beat its own AR baseline), demotes fast, and re-validates AR verdicts on a bounded interval. Measured: Qwen3.8-Flash-Next-JANG_4M 72.8 tok/s adaptive vs 40.5 AR; Qwen3.8-27B 42.2 vs 27.2 — with cold first turns faster than AR on both.
  • /v1/completions now honors reasoning_effort / enable_thinking, resolving through the bundle's stamped chat contract when unset.
  • vmlx doctor no longer false-fails native-MTP VL bundles.

Added

  • Server-exact prefix-cache fetch provenance in /health and /v1/cache/stats (per-request match kind, restored tokens, RAM-vs-SSD source, internal-continuation labeling).
  • Engine-owned lifecycle load progress surfaced across every panel view, generation-guarded against stale attempts.
  • Per-request native-MTP telemetry now reports the adaptive profile seed and workload key.

Notes

  • Full-precision SSD prefix cache, byte-identical restores proven cold/warm/partial/restart on qwen3_5-hybrid, qwen4_exp, and DeepSeek-V4 native lanes.