v0.81.63

Improved

• MoE auto-sweep leaves VRAM headroom... it measures pp512/tg128, shows live samples and saves only the final recommendation, three ncmoe steps above the tight edge when safe.
• MTP is automatic... it activates only for GGUFs with an MTP head and offloaded MoE experts, avoiding measured regressions on full-GPU models.
• Qwen3.5/3.6 decode uses one less Metal dispatch per GDN layer by fusing SSM_CONV with its following SiLU activation.
• Long-prompt prefill is faster on AMD RDNA for head sizes 64/128/256/512 (up to 54% at pp4096), with no measured decode regression; quantized KV and wave64 keep Flash Attention.

Fixed

• MoE auto-sweep no longer hangs on verbose output and now parses Metal VRAM sizes correctly.
• BF16 decode is covered by the AMD wave64 GPU path instead of falling back to CPU.

Full Changelog: v0.81.62...v0.81.63

v0.81.62

Improved

• Mixed key/value quantized caches now run on the GPU on AMD... a cache like q8_0 keys with f16 values (the recommended trade-off) was quietly falling back to CPU attention, because a same-type check rejected it before the AMD Flash Attention kernel could take it. That kernel handles keys and values independently, so mixed pairs now reach it. Measured on an RX 6700 XT, q8_0/f16 goes from CPU-fallback speed to 56 tokens/s, matching f16/f16. Works on both RDNA (wave32) and GCN/Vega (wave64) cards. Verified against the CPU reference across the key/value type matrix.

Changed

• The MTP toggle's help text now says where it helps... multi-token prediction speeds up generation on MoE models with experts offloaded to the CPU, and can be neutral or a little slower on dense or full-GPU models. The tooltip reflects that, and the toggle still lets you enable it anywhere.

Full Changelog: v0.81.61...v0.81.62

v0.81.61

Improved

• Qwen3.5/3.6 generation speed on AMD GCN/Vega cards (continued)... the short convolution that runs in front of every Gated Delta Net layer was still executing on the CPU on wave64 cards, so each generated token crossed to the CPU and back for every one of those layers. That kernel needs no cross-thread reduction, so it now runs on the GPU with the rest of the layer. This is the piece 0.81.60 missed... the fused delta-net kernel moved to the GPU there, but the convolution beside it did not, which is why generation had not sped up on those cards. Verified numerically against the CPU reference; speed reports from GCN/Vega owners are welcome.

Full Changelog: v0.81.60...v0.81.61

v0.81.60

Improved

• Qwen3.5/3.6 speed on AMD GCN/Vega cards... the Gated Delta Net layers of these models now run their fused GPU kernel on wave64 cards (RX 500 series, Vega, Radeon VII) instead of the step-by-step fallback, which padded every generated token to a 64-token block. Since 0.81.58 these models were correct but slow on those cards; generation should now be several times faster. Verified numerically against the CPU reference; speed reports from GCN/Vega owners are welcome.

Fixed

• The MTP toggle no longer breaks models with a stripped MTP head... many community quantizations remove the multi-token-prediction tensors but keep the metadata entry, and the app could read that as MTP support, making the server abort at startup with the toggle on. Detection now reads the metadata's real value, falling back to the tensor names, so only models that actually ship the head use speculative decoding.

Full Changelog: v0.81.59...v0.81.60

v0.81.59

Added

• GPU Flash Attention for Llama 3.x and gpt-oss on AMD... the AMD Flash Attention kernel now covers head size 64 and attention sinks, the two things these families needed, so their attention runs on the GPU instead of falling back to the CPU. Measured on an RX 6700 XT: gpt-oss-20b goes from 33.5 to 93.2 tokens/s with Flash Attention on (and now beats Flash Attention off, 90.3), Llama-3.2-1B from 72 to 250. Quantized KV caches ride the same kernel: gpt-oss with q4_0 keys and values holds 87 tokens/s. Verified against the CPU reference on 512 attention shapes.

Fixed

• Flash Attention no longer collapses to the CPU on uncovered models... the AMD Flash Attention toggle used to force FA unconditionally, and any model the kernel didn't cover ran its attention on the CPU at ~3× the cost, silently. The toggle now lets the engine decide per model: GPU Flash Attention where the kernel covers it, cleanly disabled where it doesn't, the CPU path never. A quantized KV cache still forces FA on (the engine requires it), and setting Flash Attention to "on" manually keeps the explicit behavior.

Deprecated

• The experimental TurboQuant engine will be retired... new improvements land in the official engine only, and the turbo2/3/4 KV quantization will be studied for integration there. The engine picker marks it, and selecting it shows a notice. It still works in this version.

Full Changelog: v0.81.58...v0.81.59

v0.81.58

Fixed

• Qwen3.5/3.6 models no longer output garbage on AMD GCN/Vega cards... on wave64 GPUs (RX 500 series, Vega, Radeon VII) the Gated Delta Net family produced endless repeated characters instead of text (#1, #25, #21). Two Metal kernels in that op chain assumed 32-lane SIMD groups: the cumulative-sum kernel read its group total from the wrong lane and wrote past its scratch memory, and the triangular solver left half the columns unsolved. Both now follow the hardware's real SIMD width, so these models run fully on the GPU on these cards. Other GPUs are untouched: on RDNA the fixed engine benchmarks identical to the previous release, output verified coherent across the whole model suite.
• The integrated GPU is never picked automatically... on Macs with an Intel iGPU next to discrete cards, macOS could hand the ~1 GB integrated GPU to the engine as the system default (typically when the display is on the internal port), which crashes with any real model. The engine now switches to the largest discrete card and says so in the log, multi-GPU splits count and use discrete cards only, and the VRAM estimator and the image tab's automatic GPU pick skip integrated GPUs. Selecting the iGPU explicitly still works, and iGPU-only Macs are unaffected.

Full Changelog: v0.81.57...v0.81.58

v0.81.57

Added

• Encoder and VAE on a second GPU... on multi-GPU Macs, each image instance can send the text encoder and the VAE to another card, leaving the main one entirely to the diffusion model, so bigger models or larger frames fit. The fit checks, the queue scheduling and the GPU warnings all account for the second card.
• Queued prompts can target an instance... a "Target" picker in the queue composer pins a prompt to one instance (its model, GPU and settings). A targeted prompt waits for its instance without blocking the rest of the queue, and the feed badges every entry with the instance that renders it.
• Per-prompt reference image in the queue... an optional "Image" chooser attaches an img2img source to the prompts you add with it, overriding the rendering instance's own reference for that run only. Pending entries show a small thumbnail of it.
• List or grid results... both the queue feed and the multi-instance canvas can switch between the full-width list and a grid whose columns adapt to the window width. Each view remembers its choice.
• Save all from the queue feed... one button copies every result of the session's gallery into a folder you pick, like the instances canvas already offered.

Fixed

• The queue's prompt box handles long prompts... long text used to run off the right edge (or get cut off with no way to scroll) and didn't re-wrap when the window was resized. The box now wraps at its width, grows up to 8 lines and scrolls beyond that; Cmd+Return adds the prompt to the queue.
• Chat and images on different GPUs no longer warn... the "chat shares a GPU" notice only appears when the chat server could actually land on a card an image instance uses, so chat on one GPU and image instances on the others run together without noise.

Improved

• Results show their prompt and full details... every instance tile and the single-instance canvas now display the prompt that made the image (hover for the full text) plus its real output size, format, seed and timing, and the queue composer keeps the target, seed and image options in one tidy row.

Full Changelog: v0.81.56...v0.81.57

v0.81.56

Fixed

• The no-AVX2 build now really is AVX2-free... the 0.81.54/55 "noavx2" downloads were still compiled with AVX2/FMA/BMI2 (the engine build system silently re-enables them unless each one is turned off explicitly), so on pre-AVX2 Xeons they crashed with the same illegal-instruction error (code 4) they were meant to fix. The legacy variant now pins an SSE4.2 baseline for all three engines (official, turbo and image).

Improved

• The server log identifies the running build... the startup banner now says "no-AVX2 build" on the legacy variant, and an engine killed by an illegal instruction is diagnosed as a CPU-instruction mismatch pointing to the right download, instead of a bare "exited with code 4".

Full Changelog: v0.81.55...v0.81.56

v0.81.55

Added

• GPU Flash Attention on AMD GCN/Vega cards... the AMD flash-attention kernel now has wave64 variants, so on these cards (RX 500 series, Vega, Radeon VII) the attention itself... the last big piece that still ran on the CPU... moves to the GPU, including quantized KV caches. It engages automatically with Flash Attention on. First build with this on real GCN hardware, so reports are very welcome: if anything looks off, turning Flash Attention off returns to the previous behavior.

Fixed

• Legacy-quant models could output garbage on GCN/Vega... on wave64 cards, dense models in the older quantization formats (Q4_0, Q4_1, Q5_0, Q5_1) were dispatched 64 lanes wide while their pipeline was still built 32 wide, corrupting the output in 0.81.54. K-quant models were not affected.

Full Changelog: v0.81.54...v0.81.55

v0.81.54

Added

• Dedicated build for pre-AVX2 Macs... older Mac Pros and Hackintoshes whose Xeons lack AVX2 (which made the normal app crash on launch with an "illegal instruction") now have their own download. It stays on its own update channel, so it never pulls a build that won't run on that CPU.
• More of the model runs on the GPU on AMD GCN/Vega cards... on wave64 cards (RX 500 series, Vega, Radeon VII) the GPU now also handles the legacy quantizations (Q4_0/Q4_1/Q5_0/Q5_1), the group/L2 normalization steps, and the Mixture-of-Experts expert math... all of which previously fell back to the CPU. Together with the existing K-quant path, most of a model's decode now runs on the GPU on these cards. It turns on automatically when a wave64 card is detected (set GGML_METAL_WAVE64_DECODE_DISABLE=1 in Extra arguments to turn it off).

Fixed

• Image generation no longer runs out of memory at high resolutions... the resolution limits now account for the fact that SD1.5/SDXL attention memory grows with the square of the image size, not linearly. Very large frames that the old estimate wrongly allowed (and that could crash the GPU) are now capped per model, so the offered sizes stay within what the card can actually render.

Improved

• Larger image queue previews and a multi-line queue prompt... results in the Queue feed now show a large preview instead of a small thumbnail, and the queue's prompt box grows to several lines so longer prompts are easier to read and edit.

Full Changelog: v0.81.53...v0.81.54