Skip to content

runNburn v0.9.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 02 Aug 12:19
v0.9.0
b219325

runNburn 0.9.0 accelerates Qwen3.6 image-prompt prefill on Apple Metal while preserving the 0.7.0 multimodal, IMRoPE, and sequence-continuation contracts. The vision path remains hybrid: BF16 linear layers and full attention run on Metal, while smaller graph operations still run on the CPU.

Metal performance

  • Measured on an Apple M5 Pro with 64 GiB unified memory, Qwen3.6-35B-A3B-UD-Q4_K_M.gguf, the BF16 vision projector, and a 768×768 white image producing 2,304 patches and 576 merged image tokens.
  • After explicit A/B warmups, an ABABAB comparison reduced mixed vision-plus-language prefill median from 18241.175 ms with CPU vision and the CPU language-prefill fallback to 5626.101 ms with the default Metal path (-69.16%, 3.24x, same-index 3/3).
  • This is an end-to-end mixed-prefill result, not a claim that an individual kernel is 3.24× faster. The accelerated path combines Metal BF16 linear and online-softmax full-attention primitives with explicit four-axis IMRoPE in the 40-layer language-prefill chain.
  • RNB_METAL_VISION=0 remains available as a diagnostic opt-out.

Correctness

  • Preserve Qwen-owned vision graph and M-RoPE semantics while keeping the existing CPU fallback.
  • BF16 linear, full-attention, and explicit IMRoPE numerical oracles pass. CPU and Metal product chats both produced green pink; a cached Metal continuation produced pink green without re-running image or mixed prefill.
  • A post-merge release smoke on the same M5 Pro completed mixed prefill in 5635.569 ms and preserved the white-image first token The.

Validation

  • Keep the strict Metal NoCopy host-storage contract intact. A CPU-only MoE routing test is now excluded from Metal builds instead of adding an unregistered-weight runtime fallback.
  • The merged Metal LLM suite completed with 488 passed, 5 ignored, and 0 failed. CUDA backend library tests completed with 162 passed, and the CPU/CUDA/Vulkan/OpenCL/Metal/MediaTek build matrix passed.

Full changelog: v0.8.0...v0.9.0

Assets 20
Loading