v0.3.0

Overview

llama.cpp 0.3.0 introduces the dots3-note multimodal model (with a new DSA-ISWA KV cache), MTP support for GLM-4.5-Air, and tensor-split (-sm tensor) plus multi-sequence rollback fixes for DeepSeek 4. ggml is bumped to v0.22.0 (meta-backend tensor split, per-op Metal kernels with parallel compilation, non-in-place ggml_clamp), while mtmd gains dots3-note vision/audio, WebP decoding and a Pillow-accurate resize. The server adds a LLAMA_SERVER_SLOTS_N_DIFF debug knob, and the web UI gets tabbed chat navigation.

New models

• Add dots3-note model with a new DSA-ISWA KV cache type (#27060)

Core changes

• DeepSeek 4: add tensor-split mode via -sm tensor (#26490)
• DeepSeek 4: fix rollback with multiple sequences (#26756)
• Fix meta tensor split state propagation for tensor parallel (#27574)
• GLM-4.5-Air: add MTP (multi-token prediction) support (#26534)
• bailingmoe3: support DSpark (#27508)
• mamba2: flatten in/out projections to dispatch GEMM instead of GEMV (#27513)
• Models: use ggml_rope_set_offset in deepseek2/4, dflash, minicpm3 and plm (#27382)
• Grammar: parse \- in char classes as a literal hyphen (#27591)
• Common: add json.h abstraction (#27511) with a clang LTO fix (#27575)
• Common: fit moved out of the server and now takes n_streams into account (#27496)
• Common: fix draft-mtp with embeddings (#27400)
• Arg: remove the -no-cnv CLI option (#27542)

Multi-modality changes

• Support dots3-note vision and audio (#27524)
• Support WebP images via ffmpeg (#27520)
• Fix loading videos with the moov atom at the end of the file (#27596)
• Use a Pillow-accurate resize algorithm and correct resize_algo for all models (#27594)
• Use ggml_rope_set_offset in the CLIP graph (#27521)

Server changes

• Add LLAMA_SERVER_SLOTS_N_DIFF env var to widen the slot debug diff window (#27600)
• Slot fitting logic moved to the common fit, now accounting for n_streams (#27496)
• Adopt the common json.h abstraction (#27511)

UI changes

• Tabbed navigation for chat conversations (#27263)
• Fix keyboard shortcuts for the chat tabs navigation (#27609)

ggml changes

• ggml bumped to v0.22.0 (ggml/1607):
  • This release adds tensor-split support to the multi-backend (meta) backend with improved split-state propagation, reworks the Metal kernels into per-op sources with parallel compilation, and fixes ggml_clamp to be a proper non-in-place op. It also brings new ops
    (POOL_1D, PAD_REFLECT_1D), Q2_K SYCL kernels, MoE bias fusion on OpenCL, and assorted fixes across the CUDA, Metal, SYCL, Vulkan, OpenCL and WebGPU backends.

Assets

Nightly build: b10621

More info

• Releases and versioning of ggml-org projects
• Help tune Metal Flash-attention kernels

Changelog since v0.2.0

c1d0e7a llama.cpp : bump version to 0.3.0 (#27696)
81191af sync : ggml
9388236 ggml : bump version to 0.22.0 (ggml/1607)
eb25b72 grammar : parse - in char classes as literal hyphen (#27591)
814d84b sycl : mark tq2_0 as not supported (#27660)
5ea87dd webgpu : fix handling of infinity values during ARGSORT and TOP_K (#27538)
f280b26 metal : per-device tuned (Q, NE) for flash-attn vec (#26570)
b615f5b metal: per-op source split + parallel compile (#26561)
b3c3b96 misc : read repetition_penalty from generation_config.json (#27659)
7584430 tests : disable DOTS3NOTE arch test for WebGPU (#27654)
71cc86f convert: fix GLM regression in index_tensors (#27655)
a14dba6 ggml : shorten virtual device naming in CUDA and Metal (#27608)
c1c766d webgpu : reorder includes since V that appears in common_decls.tmpl may be defined as K in flash_attn_decls.tmpl if KV_OVERLAP (#27545)
160c6b0 mtmd: video: fix moov atom at the end of file (#27596)
985b149 ci : apply ccache-clear with older/min/dry-run to all ccache jobs (#27602)
6036c63 ggml : fix ggml_clamp (#27644)
a130532 mamba2 : Flatten in/out projections to dispatch GEMM instead of GEMV (#27513)
bf0a29c Deepseek 4: -sm tensor (#26490)
c060ca9 model : support MTP in GLM-4.5-Air (#26534)
ccc8fd2 readme : update links (#27617)
d05f895 fix: Change chat tabs nav shortcuts (#27609)
8d9af25 test : fix multi-GPU server tests (#27614)
4a08fa2 test: move tools/parser to tests (#27548)
56db501 mtmd: use pillow-accurate algo, correct resize_algo for all models (#27594)
95b8e33 ci : add test-llama-archs tensor split for Metal (#27598)
a278dce contrib : recommend waiting for CI before merging (#27603)
e8eed45 server : add LLAMA_SERVER_SLOTS_N_DIFF (#27600)
ba8e0ed common : skip device_info loop if it's not going to be printed (#26692)
b0539c4 DeepseekV4: fix rollback with multi-seq (#26756)
d337192 [Tensor parallel] Fix meta tensor split state propagation (#27574)
8144f31 ui: Chat Conversation Tabbed navigation (#27263)
6657ded vendor : update subprocess.h (#27409)
29ea941 cuda : add POOL_1D support (#27573)
70adb1b common: json.h: fix clang lto (#27575)
3f545be vulkan : added the PAD_REFLECT_1D operation (#26586)
b21e4de mtmd: use ggml_rope_set_offset (#27521)
d9f918d common: add json.h abstraction (#27511)
2fb989b fit: also take into account n_streams (#27496)
9fee29e arg: remove -no-cnv from cli [no ci] (#27542)
e85caa8 ci : Restore ROCm job for Ubuntu (#27399)
2115b73 model : support DSpark for bailingmoe3 (#27508)
54ee5ee mtmd: support dots3-note vision+audio (#27524)
3a653fe ci : add older, min and dry-run options to ccache-clear (#27504)
369e1cd ggml: optimize concat op by replacing per-element memcpy with row-level memcpy (#24575)
2c6b141 common : fix draft-mtp with embeddings (#26352, #27299) (#27400)
8672290 sycl : add Q2_K reordered MMVQ and ESIMD kernels (again) (#27490)
3aeb924 readme : fix server badge alt (#27533)
2100e59 readme : update badges (#27531)
d775b89 mtmd: support webp via ffmpeg (#27520)
3af988f opencl: fold the gpt-oss MoE per-expert bias adds into the epilogue (op/kernel fusion) (#26431)
9a286ac docs: improve Windows build instructions (#27381)
a3b9c23 ci : fix empty release_id in make-release upload step (#27516)
5a32f7b model: add dots3-note (#27060)
873e5d8 model: use ggml_rope_set_offset() (#27382)
d7fa69b ci : run ccache-clear as the last step of release jobs (#27503)