Skip to content

runNburn v0.4.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 31 Jul 13:10
v0.4.0
d521212

runNburn 0.4.0 makes external speculative decoding automatic across the Gemma 4 family while keeping activation conditional on model support and available VRAM. It also hardens quantized CPU execution, backend memory ownership, GGUF loading, and CUDA cache identity.

CUDA performance

Measured with Gemma 4 Q4_K_M target models and Q8_0 assistant models on RTX 3090 24 GB, using matched on/off ABABAB runs. Every reported run produced token-exact output against the non-MTP path.

  • Automatic Gemma 4 external MTP. Supported E2B, E4B, 12B, 26B-A4B, and 31B models now discover the standard MTP/mtp-*-Q8_0.gguf assistant automatically. The engine selects the draft length from model structure and admits the path only when the required VRAM headroom is available.
  • Dense Gemma 4 generation medians improved by 23.1% (E2B), 24.3% (E4B), 65.5% (12B), and 76.3% (31B) at draft length 3. The 26B-A4B selected-MoE model improved by 11.2% at draft length 1 on RTX 3090 and by 30.6% on RTX 3060 12 GB.
  • Gemma selected-MoE CUDA admission. The Q5_1/Q8_0-down CUDA path now activates from tensor shape, quant type, and measured free-VRAM headroom rather than model names. Force-vs-auto ABABAB overhead was 0.188% on RTX 3090 and 0.193% on RTX 3060, with exact output hashes.
  • Q4 resident cache identity. Mmap-backed GGUF weights now use storage generation, offset, and length as their O(1) identity; temporary slices use a full content fingerprint. This prevents same-length weights with matching edges but different middle blocks from aliasing one CUDA allocation.

Fixed

  • Preserved F16 KV-cache capacity during writes instead of shrinking allocations and invalidating later continuation state.
  • Fixed shared-KV conversion and ignored-EOS correction in Gemma external MTP, restoring token-exact generation.
  • Rejected unsupported GGUF architectures at load time instead of entering an invalid product path.
  • Removed parallel output aliasing from Q4_K, Q5_K, and Q6_K CPU kernels; avoided unaligned quantized block and row references; validated F32 slice casts and dot-product lengths; handled AArch64 NEON attention tails.
  • Retained Metal NoCopy source mappings for the full buffer lifetime, used the host page size for alignment, and enforced carrier shape contracts in release builds.
  • Destroyed Vulkan buffers before freeing their backing memory.
  • Eliminated CUDA test-suite flakes caused by process-wide environment races and sub-ULP atomic accumulation-order drift.

Validation

  • Standard Qwen3.6-35B output remained byte-identical after the CUDA cache changes.
  • The four-model CUDA rerun covered Qwen3.6 35B-A3B, Gemma 4 26B-A4B, Hy3 295B-A21B, and GLM-5.2; Qwen matched its established reference hash, Gemma auto-MTP remained stable, and Hy3/GLM token IDs were exact across repeated runs.
  • The release workflow passed all 10 jobs: Linux CPU/CUDA/Vulkan/OpenCL, macOS Metal, Android CPU/Vulkan/OpenCL/MediaTek, and release publication.

Full changelog: v0.3.0...v0.4.0

Assets 20
Loading