Skip to content

runNburn v0.8.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 02 Aug 12:02
v0.8.0
11a7e29

runNburn 0.8.0 adds end-to-end Gemma 4 vision inference using standard gemma4v projector GGUFs. Local chat and OpenAI-compatible endpoints can process images and retain multimodal KVarN sequence state across follow-up turns without creating model-shaping sidecars or requantizing the language model.

Gemma 4 multimodal inference

  • Load mmap-backed Gemma 4 vision projector tensors from a separate GGUF with --mmproj, including strict metadata, tensor-shape, and tensor-type validation.
  • Execute dynamic-resolution image preprocessing, learned 2D positions, normalized 2D RoPE attention, quick-GeGLU blocks, 3×3 pooling, and BF16 projection.
  • Compile Gemma's <|image|> chat-template placeholder into the required image boundary tokens and unscaled external embedding rows.
  • Run image embedding chunks with non-causal attention while retaining causal boundary-token and text execution.

Stateful chat and API support

  • Accept local PNG/JPEG images in runNburn chat and PNG/JPEG data URLs in Chat Completions and Responses.
  • Preserve the image fingerprint, physical KV rows, logical text positions, and KVarN state for compatible follow-up turns.
  • Keep Gemma decode on linear RoPE positions instead of applying Qwen-specific IMRoPE state.
  • Share RGB image ownership through rnb-core while keeping Qwen and Gemma projector semantics separate.

Acceleration scope

  • CUDA builds accelerate the language-model prefill and decode path, including Gemma image-span non-causal attention.
  • Gemma 4 and Qwen3.6 vision towers currently execute on CPU; this release does not claim end-to-end CUDA vision encoding.
  • CPU-only builds retain the complete multimodal execution path.

Validation

  • Gemma 4 26B-A4B UD-Q4_K_M with the official BF16 projector described the bundled runNburn logo and completed a same-process three-turn context check: Red black → Black red → Black.
  • Gemma model tests: 6 passed. Loader tests: 108 passed. CLI tests: 49 passed. LLM CPU tests: 459 passed, 1 ignored. LLM CUDA tests: 475 passed, 16 ignored. CUDA backend tests: 625 passed, 45 ignored.
  • CPU, CUDA, Vulkan, OpenCL, and MediaTek build-matrix checks passed.
  • No llama.cpp parity or performance claim is made in this release.

Full changelog: v0.7.0...v0.8.0

Assets 20
Loading