Skip to content

runNburn v0.7.0

Choose a tag to compare

@github-actions github-actions released this 02 Aug 10:35

runNburn 0.7.0 adds end-to-end Qwen3.6 vision inference and retains multimodal sequence state across local chat and OpenAI-compatible Responses continuations. Images are encoded directly from the GGUF vision projector, while the original quantized language-model weights remain authoritative.

Multimodal inference

  • Load mmap-backed Qwen3.6 vision projector tensors from a separate GGUF with --mmproj.
  • Decode PNG and JPEG inputs for local runNburn chat --image, /v1/chat/completions, and /v1/responses requests.
  • Execute image and text rows with Qwen3.6 multimodal rotary-position planning, including logical/physical row separation and IMRoPE state.
  • Preserve the existing text-only path when no projector or image is supplied.

Stateful continuation

  • Retain KVarN attention rows, GDN recurrent state, multimodal physical rows, logical IMRoPE position, token history, and image identity after a generated turn.
  • Resume local chat and stored Responses requests from that state and execute only the new text suffix.
  • Clear retained image state together with conversation history on /clear or a system-prompt replacement.
  • Skip multimodal snapshot reuse when the image identity or rendered prefix does not match, rather than silently reusing incompatible state.

Performance

  • Measured on an NVIDIA RTX 3090 with Qwen3.6-35B-A3B-UD-Q4_K_M.gguf, the BF16 Qwen3.6 vision projector, a 607-token image prompt, and a two-token follow-up response.
  • After one explicit A warmup, an ABABAB product-path comparison reduced median follow-up latency from 17710.291 ms for full multimodal history replay to 2140.150 ms for state restore plus the text suffix (-87.92%, 8.28x, same-index 3/3).
  • All six measured follow-ups produced the same Pink Green text after whitespace normalization. The retained-state path included capture of the next-turn snapshot.
  • This is an end-to-end product-path result, not a pure kernel comparison: the compared endpoint framings reported 639 and 643 logical prompt tokens respectively.

API and documentation

  • Document Qwen3.6 multimodal local chat, server projector configuration, and multimodal endpoint support.
  • Add base64 data URL parsing for OpenAI-compatible image content.
  • Keep stored response state process-local and bounded by the configured response-cache budget.

Dependencies

  • Add PNG/JPEG decoding through image, base64 data URL decoding through base64, and SHA-256 image identity through sha2.

Full changelog: v0.6.2...v0.7.0