vMLX 1.6.24

vMLX 1.6.24

Fixed

• Long DeepSeek V4 Flash generations no longer crash the engine. They failed
  with [metal::malloc] Resource limit (499000) exceeded about twelve minutes
  in, returned as an HTTP 500.

  Metal caps the number of live buffers, not just their bytes — allocating
  499,000 one-byte arrays throws with ~0 GB of active memory. The output guard
  modelled only bytes, so it was structurally blind to this and advertised a
  safe cap of 17,575 tokens for a model that dies around 12,000.

  Measured by sampling the live buffer count while decoding:

  model                  layers   buffers per generated token
  Qwen 3.6 27B               64   0.000   (flat across 600 steps)
  DeepSeek V4 Flash          43   ~42     (one per layer per token)
  

  A conventional KV cache grows in 256-token steps and reuses buffers.
  DeepSeek V4 Flash keeps cumulative per-layer compressor and indexer pools, so
  its live count climbs linearly; 499000 / 42 lands at about 11,900 tokens,
  which is exactly why 12,000 succeeded and 14,336 did not. Clearing the MLX
  cache frees none of it, so the ceiling is hard.

  The guard now projects that ceiling alongside the byte ceiling and takes
  whichever binds first. An oversized request is refused up front with a clear
  message; an unspecified one is clamped and logged. Architectures whose
  retention has been measured at zero are deliberately left uncapped, so nothing
  else is affected.

• DeepSeek V4 Flash answers again at small output budgets. The answer
  reserve had a flat 256-token floor, which made every budget under 512
  unsplittable — reasoning consumed all of it and the answer came back empty.
  Measured before: 160, 256 and 400 all produced no content while 512 and above
  answered. The floor is now capped at half the budget, so 128 and up split.
  Every budget at or above 512 is unchanged.

• The chat Max Tokens field no longer sits blank while a budget is silently in
  force. Bundles are not required to declare an output budget; when one does
  not, the engine resolves its own and clamps it to projected headroom. The app
  now shows that resolved number as the placeholder instead of nothing.

Separate signed and Apple-notarized Apple-silicon downloads are available for
macOS Tahoe and Sequoia-compatible systems. Tahoe is the default download.