vMLX 1.6.25

vMLX 1.6.25

Performance

• DeepSeek V4 Flash long-context prefill is dramatically faster. A new
  indexed-attention prefill kernel (on by default) lifts prompt processing
  from 347 to ~449 tokens/s at 15k context (+29%) and from 199 to ~316
  tokens/s at 40k (+58%). Decode holds 33+ tokens/s sustained at both short
  and long context.
• Aborting and retrying a long prompt is no longer a cold start. The
  engine rewinds the live KV cache to the shared prefix instead of
  re-prefilling from scratch — retry time-to-first-token at 15k context
  dropped from 46s to 2.3s.
• The two-pass answer path reuses the first pass's KV cache instead of
  re-prefilling the whole prompt.
• KV pool quantization keeps cache RAM bounded — under 8 GB even within
  reach of the model's maximum context.
• Extreme context lengths fail honestly instead of crashing. The engine
  now advertises the true hardware ceiling at startup, shrinks prefill chunks
  adaptively under Metal memory pressure, and refuses over-ceiling requests
  with a clean 413 instead of a Metal OOM. The flat request timeout that
  killed healthy very-long prefills is gone.

Fixed

• Prompt-too-long errors now appear as a clear in-chat message instead of a
  raw error dialog, and an oversized message no longer poisons every later
  turn of that conversation (#253).
• Checkpoints with a stripped vision tower load again with
  language_model_only (#251).
• Hybrid SSM models: the deferred state re-derive no longer delays final
  stream completion (#245).
• VLM JANG quantization no longer selects modules that cannot be quantized
  (#246).
• Laguna-S-2.1-JANG_4M loads again (#247).
• A session default max-tokens above the model's cap now clamps with a logged
  warning instead of rejecting every request that doesn't specify its own
  budget.
• Disk (L2) cache: globally evicted payloads no longer leave stale index rows
  that blocked re-storing those blocks, and a mid-session runtime fingerprint
  degradation no longer poisons all L2 lookups.
• Auto-sleep no longer fires in the middle of an active generation, and a
  failed request during setup no longer wedges the composer in a streaming
  state.
• Relative timestamps in the chat transcript update as time passes instead of
  freezing at "Just now".

Built from source commit 81f90bbb3.

Separate signed and Apple-notarized Apple-silicon downloads are available for
macOS Tahoe and Sequoia-compatible systems. Tahoe is the default download.