Skip to content

v0.3.2

Choose a tag to compare

@eugenehp eugenehp released this 21 Jun 04:09
· 151 commits to main since this release

Summary

Patch release: latest llama.cpp submodule, EAGLE-3 speculative decoding, multimodal video input for mtmd, and fixes for the reworked upstream mtmd helper API.

Changed

  • llama.cpp: vendored submodule 94a220cd6 → c57607016 (#256). The public llama.h C API is unchanged; the mtmd helper API was reworked (see Fixed).

Added

  • EAGLE-3 speculative decoding (llama-cpp-4): new eagle::Eagle3Session driving upstream COMMON_SPECULATIVE_TYPE_DRAFT_EAGLE3. Pairs a target context with a separate EAGLE-3 draft-model context. The mtp_shim is generalised with a spec_type selector shared by MTP and EAGLE-3 (MtpSession unchanged). Verified end-to-end on Apple M4 Pro (Metal) with Qwen3-8B + RedHatAI/Qwen3-8B-speculator.eagle3 at ~53% draft acceptance.
  • mtmd video input (mtmd feature): mtmd::MtmdVideo (+ MtmdVideoParams, MtmdVideoInfo, MtmdVideoItem) wrapping the new mtmd_helper_video_* API for frame-by-frame decoding via ffmpeg, plus MtmdContext::supports_video() and MtmdContext::marker().
  • examples/eagle (+ README) with scripts/setup-eagle3.sh / scripts/fetch-eagle3.sh to download + convert a target/draft pairing to GGUF.

Fixed

  • mtmd bindings: adapt to the reworked helper API — mtmd_helper_bitmap_init_from_file/_from_buf now take a placeholder flag and return a mtmd_helper_bitmap_wrapper; mtmd_helper_decode_image_chunk takes a post-decode callback. MtmdBitmap::from_file/from_buf no longer leak the video_ctx returned for video inputs.

Crates: llama-cpp-4 0.3.2 · llama-cpp-sys-4 0.3.2