Skip to content

runNburn v0.6.0

Choose a tag to compare

View all tags
@github-actions github-actions released this 02 Aug 04:07
v0.6.0
2f26ac5

runNburn 0.6.0 adds an architecture-aware GGUF inference path for DeepSeek 4 Flash, including its Hyper-Connection residual stream, compressed attention/indexer state, sqrt-softplus MoE routing, and grouped output projection. The release also adds a CUDA grouped routed-expert path while keeping the original quantized GGUF weights authoritative.

Added

  • Load and run deepseek4 GGUF models through a dedicated runtime instead of treating them as Llama-family graphs.
  • Implement learned KV compression, the compressed-key indexer, Hyper-Connection Sinkhorn mixing, hash-routed and score-routed experts, clamped shared/routed SwiGLU, grouped attention output, and GGUF Jinja chat templates.
  • Expose an explicit opaque DeepSeek4Forward IR node so public loader metadata no longer advertises tensor names that the architecture does not contain.

Fixed

  • Correct the source-major Hyper-Connection comb layout used during residual mixing.
  • Clear partially updated compressed-attention state after failed forwards and reject sequence-position desynchronization.
  • Disable the generic backend-only argmax decode path for DeepSeek 4, whose weights are owned by the dedicated runtime.
  • Accept scalar or per-layer clamp metadata, inherit the routed clamp when the shared clamp is omitted, and reject unsupported compression ratios or MoE gating functions.
  • Apply the resolved sparse-MoE CUDA policy to DeepSeek 4. RNB_CUDA_Q2K_Q3K_SPARSE_MOE=0 now keeps routed experts on the CPU path.

CUDA execution

  • Run IQ2_XXS gate/up and IQ3_XXS down routed experts through the selected-expert grouped CUDA path with DeepSeek-compatible clamped SwiGLU and a CPU fallback for unsupported layouts.
  • On an RTX 3090 with DeepSeek-V4-Flash-0731-UD-IQ2_M, a 21-token thinking-off product smoke generated 대한민국의 수도는 **서울**입니다. with sparse CUDA both enabled and explicitly disabled.
  • Earlier development A/B timings were collected before the final Hyper-Connection correctness correction, so this release does not carry them forward as a performance claim.

Verification

  • rnb-loader: 83 tests passed.
  • rnb-llm --features cuda: 473 passed, 20 ignored.
  • CPU, CUDA, Vulkan, OpenCL, and MediaTek build-matrix combinations passed.
  • CI and all 10 release jobs passed.
  • Release assets contain nine archives and nine matching SHA-256 files.

Full changelog: v0.5.1...v0.6.0

Assets 20
Loading