Skip to content

v0.82.4

Choose a tag to compare

View all tags
@github-actions github-actions released this 19 Jul 02:13
v0.82.4
bc58fb8

Added

  • Ternary Q2_0 models now load (#41)... mainline's Q2_0 type runs Prism ML's Ternary Bonsai GGUFs on Metal instead of failing with "invalid ggml type 42".
  • Manual vision projector selection (#35)... pick a specific mmproj, keep auto-pairing, or turn vision off, from each model card.
  • DFlash speculative decoding (experimental)... a downloaded per-model draft with Off/Auto/Forced modes; Auto only engages on MoE with CPU-offloaded experts, up to +21% generation on an RX 6700 XT.
  • DFlash sizes itself to the GPU... Auto budgets the weights, both KV caches and the VRAM reserve before launch, then offloads draft layers or stays off, and warns if VRAM stays above 95%.

Changed

  • The vision projector no longer downloads with the model... a separate button fetches the mmproj on demand, so text-only users skip the file.
  • On AMD the KV cache picker offers only the fast types (f16/q8_0/q4_0)... the rest have no AMD Flash Attention kernel and ran ~3.7x slower.
  • The DFlash control shows when it is actually running... the bolt lights up with the live draft-acceptance rate only when the engine truly engaged the draft, not just when Auto is picked.
  • Sharing a benchmark uses the benchmark's own config... the share panel publishes the settings shown in Run benchmark, not the global ones.
  • Downloaded models remember their source repo... for on-demand sibling downloads (projector, DFlash draft).

Fixed

  • An unsupported quantization gives a clear reason... a model the engine can't read now says the quant isn't supported instead of "model file damaged or incomplete".
  • Persisted KV caches survive a KV-type change... slot files are kept per KV type, so switching f16↔q8_0 cold-prefills cleanly instead of logging a failed restore.

Full Changelog: v0.82.3...v0.82.4

Assets 5
Loading