Skip to content

runNburn v0.6.2

Choose a tag to compare

View all tags
@github-actions github-actions released this 02 Aug 07:27
v0.6.2
3c02c09

runNburn 0.6.2 accelerates the corrected DeepSeek 4 Metal path on Apple Silicon. It moves attention projections and sparse MoE work to Metal and extends the decode Q front so q_a, RMS normalization, and q_b remain device-resident, while preserving CPU fallbacks and the original quantized weights.

Metal execution

  • Batch the DeepSeek 4 attention prefill projections, attention output, and index projection through the existing Metal TensorOps path.
  • Fuse compressor and indexer-compressor Q8_0 projections so they share one input upload, cast, command buffer, wait, and readback.
  • Run supported routed and shared MoE prefill and decode work on Metal, with CPU fallback for unsupported expert counts or layouts.
  • Keep the decode q_a Q5_K projection, RMS normalization, and q_b/index q_b Q8_0 projections in one Metal command buffer. On the tested model this removes 42 CPU Q5_K calls, 42 intermediate allocations, and 336 KiB of host transfer per generated token.
  • Keep the original GGUF quantized weights authoritative; the new paths do not create expanded or requantized weight copies.

Performance

  • On an Apple M5 Pro with 64 GiB unified memory and DeepSeek-V4-Flash-0731-UD-IQ2_M, the combined DeepSeek 4 Metal paths reduced the 21-prompt/4-decode median from 8773.693 ms to 3165.723 ms (-63.92%, same-index 3/3).
  • The device-resident Q front reduced the 21-prompt/100-decode median from 18938.506 ms to 18273.765 ms (-3.51%, same-index 2/3). The 20-token set improved by 1.56%, same-index 3/3.
  • All comparisons used an explicit warmup A/B followed by ABABAB measurements without cooldown. The 100-token Q-front result is a small, variable improvement rather than a strong winner; the path was retained because it also removes independently measured host transfers and allocations.

Correctness

  • Default and Q-front opt-out paths produced the same exact 4-token output hash.
  • All six 20-token A/B runs produced the same exact output hash.
  • The 100-token ignore-EOS runs diverged only in filler generated after the completed answer; product-mode semantic correctness was unchanged.
  • Unsupported quantization, shape, or storage conditions fall back to the previous CPU/Metal path. RNB_METAL_DEEPSEEK4_ATTN_Q_FRONT=0 disables only the new Q-front boundary.

Verification

  • The focused Q5_K-to-RMS-to-Q8_0 Metal test, seven DeepSeek 4 runtime policy tests, and four Metal DeepSeek 4 LLM tests passed.
  • The CPU product check and nightly Metal release build passed, and the release binary reported version 0.6.2.
  • Main CI and all 10 release jobs passed.
  • Release assets contain nine archives and nine matching SHA-256 files.

Full changelog: v0.6.1...v0.6.2

Assets 20
Loading