runNburn v0.10.0

runNburn 0.10.0 broadens multimodal GGUF compatibility across the locally validated Qwen3.6 and Gemma 4 families, and keeps compatible MTP drafters active across fresh image prompts and cached follow-up turns. It also fixes speculative and shared-KV snapshot edge cases that could interrupt resumed image conversations.

Multimodal

• Add Qwen3.6 dense 27B support alongside 35B-A3B MoE, including non-MTP and in-model MTP GGUF packages with the Qwen3-VL merger projector.
• Add Gemma 4 unified-projector support across E2B, E4B, 12B, 31B, and 26B-A4B base/QAT variants while preserving each model's projector, shared-KV, PLE, and quantization contracts.
• Preserve image-sequence continuation: cached follow-up turns reuse the saved image fingerprint, KV/SSM state, logical position, and prompt alignment instead of repeating vision and mixed prefill.

MTP

• Route fresh multimodal prefill logits and cached follow-up suffixes into the existing MTP generation loops.
• Preserve Qwen in-model MTP and Gemma sibling Q8_0 external-drafter state in durable sequence snapshots.
• Separate Qwen physical image-KV offsets from logical text RoPE positions during CUDA verification.
• Add Q5_K projection support to the CUDA device verifier for the dense Qwen3.6 27B MTP head without requantizing weights.

Fixed

• Synchronize the multimodal sequence cursor after partial MTP prefix rollback, preventing Qwen3.6 27B MTP snapshot capture from observing a stale physical length.
• Restore sparse KVarN snapshots whose reused Gemma shared-KV layers legitimately store fewer rows than the global logical sequence length.
• Keep Q4_0 QAT weights out of Q4_K-only CUDA chains and execute selected Gemma 4 QAT expert matrices without row-by-row device round trips.

Validation

• An RTX 3090 CUDA product matrix covered 14 locally retained target/projector combinations: four Qwen3.6 packages and ten Gemma 4 base/QAT packages. All 14 completed an image turn plus a cached text follow-up with the expected in-model MTP, external MTP, or target-only path and exited successfully.
• CUDA backend tests completed with 627 passed and 45 ignored. The CUDA-enabled LLM suite completed with 499 passed and 20 ignored. The release CUDA CLI and the full CPU/CUDA/Vulkan/OpenCL/MediaTek build matrix passed.
• These model runs were single-run correctness traces, not controlled performance comparisons. This release makes no new image-MTP speedup claim.
• Release CI completed all 10 packaging/publish jobs, and main CI passed.

Full changelog: v0.9.0...v0.10.0

runNburn v0.9.0

runNburn 0.9.0 accelerates Qwen3.6 image-prompt prefill on Apple Metal while preserving the 0.7.0 multimodal, IMRoPE, and sequence-continuation contracts. The vision path remains hybrid: BF16 linear layers and full attention run on Metal, while smaller graph operations still run on the CPU.

Metal performance

• Measured on an Apple M5 Pro with 64 GiB unified memory, Qwen3.6-35B-A3B-UD-Q4_K_M.gguf, the BF16 vision projector, and a 768×768 white image producing 2,304 patches and 576 merged image tokens.
• After explicit A/B warmups, an ABABAB comparison reduced mixed vision-plus-language prefill median from 18241.175 ms with CPU vision and the CPU language-prefill fallback to 5626.101 ms with the default Metal path (-69.16%, 3.24x, same-index 3/3).
• This is an end-to-end mixed-prefill result, not a claim that an individual kernel is 3.24× faster. The accelerated path combines Metal BF16 linear and online-softmax full-attention primitives with explicit four-axis IMRoPE in the 40-layer language-prefill chain.
• RNB_METAL_VISION=0 remains available as a diagnostic opt-out.

Correctness

• Preserve Qwen-owned vision graph and M-RoPE semantics while keeping the existing CPU fallback.
• BF16 linear, full-attention, and explicit IMRoPE numerical oracles pass. CPU and Metal product chats both produced green pink; a cached Metal continuation produced pink green without re-running image or mixed prefill.
• A post-merge release smoke on the same M5 Pro completed mixed prefill in 5635.569 ms and preserved the white-image first token The.

Validation

• Keep the strict Metal NoCopy host-storage contract intact. A CPU-only MoE routing test is now excluded from Metal builds instead of adding an unregistered-weight runtime fallback.
• The merged Metal LLM suite completed with 488 passed, 5 ignored, and 0 failed. CUDA backend library tests completed with 162 passed, and the CPU/CUDA/Vulkan/OpenCL/Metal/MediaTek build matrix passed.

Full changelog: v0.8.0...v0.9.0

runNburn v0.8.0

runNburn 0.8.0 adds end-to-end Gemma 4 vision inference using standard gemma4v projector GGUFs. Local chat and OpenAI-compatible endpoints can process images and retain multimodal KVarN sequence state across follow-up turns without creating model-shaping sidecars or requantizing the language model.

Gemma 4 multimodal inference

• Load mmap-backed Gemma 4 vision projector tensors from a separate GGUF with --mmproj, including strict metadata, tensor-shape, and tensor-type validation.
• Execute dynamic-resolution image preprocessing, learned 2D positions, normalized 2D RoPE attention, quick-GeGLU blocks, 3×3 pooling, and BF16 projection.
• Compile Gemma's <|image|> chat-template placeholder into the required image boundary tokens and unscaled external embedding rows.
• Run image embedding chunks with non-causal attention while retaining causal boundary-token and text execution.

Stateful chat and API support

• Accept local PNG/JPEG images in runNburn chat and PNG/JPEG data URLs in Chat Completions and Responses.
• Preserve the image fingerprint, physical KV rows, logical text positions, and KVarN state for compatible follow-up turns.
• Keep Gemma decode on linear RoPE positions instead of applying Qwen-specific IMRoPE state.
• Share RGB image ownership through rnb-core while keeping Qwen and Gemma projector semantics separate.

Acceleration scope

• CUDA builds accelerate the language-model prefill and decode path, including Gemma image-span non-causal attention.
• Gemma 4 and Qwen3.6 vision towers currently execute on CPU; this release does not claim end-to-end CUDA vision encoding.
• CPU-only builds retain the complete multimodal execution path.

Validation

• Gemma 4 26B-A4B UD-Q4_K_M with the official BF16 projector described the bundled runNburn logo and completed a same-process three-turn context check: Red black → Black red → Black.
• Gemma model tests: 6 passed. Loader tests: 108 passed. CLI tests: 49 passed. LLM CPU tests: 459 passed, 1 ignored. LLM CUDA tests: 475 passed, 16 ignored. CUDA backend tests: 625 passed, 45 ignored.
• CPU, CUDA, Vulkan, OpenCL, and MediaTek build-matrix checks passed.
• No llama.cpp parity or performance claim is made in this release.

Full changelog: v0.7.0...v0.8.0

runNburn v0.7.0

runNburn 0.7.0 adds end-to-end Qwen3.6 vision inference and retains multimodal sequence state across local chat and OpenAI-compatible Responses continuations. Images are encoded directly from the GGUF vision projector, while the original quantized language-model weights remain authoritative.

Multimodal inference

• Load mmap-backed Qwen3.6 vision projector tensors from a separate GGUF with --mmproj.
• Decode PNG and JPEG inputs for local runNburn chat --image, /v1/chat/completions, and /v1/responses requests.
• Execute image and text rows with Qwen3.6 multimodal rotary-position planning, including logical/physical row separation and IMRoPE state.
• Preserve the existing text-only path when no projector or image is supplied.

Stateful continuation

• Retain KVarN attention rows, GDN recurrent state, multimodal physical rows, logical IMRoPE position, token history, and image identity after a generated turn.
• Resume local chat and stored Responses requests from that state and execute only the new text suffix.
• Clear retained image state together with conversation history on /clear or a system-prompt replacement.
• Skip multimodal snapshot reuse when the image identity or rendered prefix does not match, rather than silently reusing incompatible state.

Performance

• Measured on an NVIDIA RTX 3090 with Qwen3.6-35B-A3B-UD-Q4_K_M.gguf, the BF16 Qwen3.6 vision projector, a 607-token image prompt, and a two-token follow-up response.
• After one explicit A warmup, an ABABAB product-path comparison reduced median follow-up latency from 17710.291 ms for full multimodal history replay to 2140.150 ms for state restore plus the text suffix (-87.92%, 8.28x, same-index 3/3).
• All six measured follow-ups produced the same Pink Green text after whitespace normalization. The retained-state path included capture of the next-turn snapshot.
• This is an end-to-end product-path result, not a pure kernel comparison: the compared endpoint framings reported 639 and 643 logical prompt tokens respectively.

API and documentation

• Document Qwen3.6 multimodal local chat, server projector configuration, and multimodal endpoint support.
• Add base64 data URL parsing for OpenAI-compatible image content.
• Keep stored response state process-local and bounded by the configured response-cache budget.

Dependencies

• Add PNG/JPEG decoding through image, base64 data URL decoding through base64, and SHA-256 image identity through sha2.

Full changelog: v0.6.2...v0.7.0

runNburn v0.6.2

runNburn 0.6.2 accelerates the corrected DeepSeek 4 Metal path on Apple Silicon. It moves attention projections and sparse MoE work to Metal and extends the decode Q front so q_a, RMS normalization, and q_b remain device-resident, while preserving CPU fallbacks and the original quantized weights.

Metal execution

• Batch the DeepSeek 4 attention prefill projections, attention output, and index projection through the existing Metal TensorOps path.
• Fuse compressor and indexer-compressor Q8_0 projections so they share one input upload, cast, command buffer, wait, and readback.
• Run supported routed and shared MoE prefill and decode work on Metal, with CPU fallback for unsupported expert counts or layouts.
• Keep the decode q_a Q5_K projection, RMS normalization, and q_b/index q_b Q8_0 projections in one Metal command buffer. On the tested model this removes 42 CPU Q5_K calls, 42 intermediate allocations, and 336 KiB of host transfer per generated token.
• Keep the original GGUF quantized weights authoritative; the new paths do not create expanded or requantized weight copies.

Performance

• On an Apple M5 Pro with 64 GiB unified memory and DeepSeek-V4-Flash-0731-UD-IQ2_M, the combined DeepSeek 4 Metal paths reduced the 21-prompt/4-decode median from 8773.693 ms to 3165.723 ms (-63.92%, same-index 3/3).
• The device-resident Q front reduced the 21-prompt/100-decode median from 18938.506 ms to 18273.765 ms (-3.51%, same-index 2/3). The 20-token set improved by 1.56%, same-index 3/3.
• All comparisons used an explicit warmup A/B followed by ABABAB measurements without cooldown. The 100-token Q-front result is a small, variable improvement rather than a strong winner; the path was retained because it also removes independently measured host transfers and allocations.

Correctness

• Default and Q-front opt-out paths produced the same exact 4-token output hash.
• All six 20-token A/B runs produced the same exact output hash.
• The 100-token ignore-EOS runs diverged only in filler generated after the completed answer; product-mode semantic correctness was unchanged.
• Unsupported quantization, shape, or storage conditions fall back to the previous CPU/Metal path. RNB_METAL_DEEPSEEK4_ATTN_Q_FRONT=0 disables only the new Q-front boundary.

Verification

• The focused Q5_K-to-RMS-to-Q8_0 Metal test, seven DeepSeek 4 runtime policy tests, and four Metal DeepSeek 4 LLM tests passed.
• The CPU product check and nightly Metal release build passed, and the release binary reported version 0.6.2.
• Main CI and all 10 release jobs passed.
• Release assets contain nine archives and nine matching SHA-256 files.

Full changelog: v0.6.1...v0.6.2

runNburn v0.6.1

runNburn 0.6.1 hardens GGUF parsing and tokenizer construction. It fixes valid-file compatibility, rejects malformed metadata before unsafe allocation or inference, and removes silent tokenizer fallbacks that could produce incorrect token streams.

Fixed

• Decode the standard GGUF uint64, int64, and float64 metadata type IDs correctly, accept only canonical boolean encodings, and use checked integer conversions instead of wrapping negative or oversized values.
• Validate GGUF alignment, header counts, metadata keys, tensor names and dimensions, quantized row block sizes, tensor data bounds, and MTP1 sidecar ranges without panicking.
• Accept valid aligned gaps between tensor payloads instead of requiring every tensor to begin immediately after the previous padded range.
• Reject big-endian GGUF files explicitly because the current mmap-backed tensor runtime consumes little-endian GGML payloads.
• Bound string and array allocations before materialization and reject duplicate metadata keys or tensor names.
• Propagate type errors from optional architecture metadata instead of treating malformed present values as missing.
• Reject empty atomic tokenizer tokens before they can cause a non-advancing tokenization loop and unbounded memory growth.

Tokenizer compatibility

• Preserve GGUF token types, added tokens, tokenizer pre-type metadata, and BOS/EOS/UNK/SEP/PAD identifiers.
• Use explicit unknown, control, user-defined, and added-token metadata for atomic tokenization instead of relying on delimiter-shaped token text.
• Remove generated <tok_N> placeholder vocabularies and the unknown-tokenizer-to-SentencePiece fallback.
• Fail during model loading for missing vocabularies, unsupported tokenizer models, Hugging Face JSON-only tokenizers, malformed merge rules, out-of-range special-token IDs, duplicate tokens, and unsupported add-EOS/add-SEP policies.

Verification

• rnb-loader: 104 tests passed.
• rnb-llm: 464 passed, 5 ignored.
• rnb-backend-cuda --lib: 624 passed, 45 ignored.
• Metal-excluded workspace: 2039 passed, 65 ignored.
• Qwen3.6 and Gemma4 GGUF metadata loading passed, and a Gemma4 E2B product chat smoke completed tokenizer construction, model loading, and one-token generation.
• CI and all 10 release jobs passed.
• Release assets contain nine archives and nine matching SHA-256 files.

Full changelog: v0.6.0...v0.6.1

runNburn v0.6.0

runNburn 0.6.0 adds an architecture-aware GGUF inference path for DeepSeek 4 Flash, including its Hyper-Connection residual stream, compressed attention/indexer state, sqrt-softplus MoE routing, and grouped output projection. The release also adds a CUDA grouped routed-expert path while keeping the original quantized GGUF weights authoritative.

Added

• Load and run deepseek4 GGUF models through a dedicated runtime instead of treating them as Llama-family graphs.
• Implement learned KV compression, the compressed-key indexer, Hyper-Connection Sinkhorn mixing, hash-routed and score-routed experts, clamped shared/routed SwiGLU, grouped attention output, and GGUF Jinja chat templates.
• Expose an explicit opaque DeepSeek4Forward IR node so public loader metadata no longer advertises tensor names that the architecture does not contain.

Fixed

• Correct the source-major Hyper-Connection comb layout used during residual mixing.
• Clear partially updated compressed-attention state after failed forwards and reject sequence-position desynchronization.
• Disable the generic backend-only argmax decode path for DeepSeek 4, whose weights are owned by the dedicated runtime.
• Accept scalar or per-layer clamp metadata, inherit the routed clamp when the shared clamp is omitted, and reject unsupported compression ratios or MoE gating functions.
• Apply the resolved sparse-MoE CUDA policy to DeepSeek 4. RNB_CUDA_Q2K_Q3K_SPARSE_MOE=0 now keeps routed experts on the CPU path.

CUDA execution

• Run IQ2_XXS gate/up and IQ3_XXS down routed experts through the selected-expert grouped CUDA path with DeepSeek-compatible clamped SwiGLU and a CPU fallback for unsupported layouts.
• On an RTX 3090 with DeepSeek-V4-Flash-0731-UD-IQ2_M, a 21-token thinking-off product smoke generated 대한민국의 수도는 **서울**입니다. with sparse CUDA both enabled and explicitly disabled.
• Earlier development A/B timings were collected before the final Hyper-Connection correctness correction, so this release does not carry them forward as a performance claim.

Verification

• rnb-loader: 83 tests passed.
• rnb-llm --features cuda: 473 passed, 20 ignored.
• CPU, CUDA, Vulkan, OpenCL, and MediaTek build-matrix combinations passed.
• CI and all 10 release jobs passed.
• Release assets contain nine archives and nine matching SHA-256 files.

Full changelog: v0.5.1...v0.6.0

runNburn v0.5.1

runNburn 0.5.1 is a compatibility patch for the v0.5.0 release. It fixes two model paths discovered while running the standard four-model suite on Apple M5 Pro without changing model formats or CUDA kernels.

Fixes

• Retain model-owned F32 weight storage used by Metal NoCopy buffers. This fixes the storage-lease panic in Qwen3.6 in-model MTP after F16 or quantized weights are converted to F32. Registration is compiled only when the Metal feature is enabled.
• Reject unsupported AArch64 Q2_K prequantized Q8K dispatch. Q2_K now uses the existing F32 or fused NEON path instead of panicking, while the existing Q8 fast paths for Q4_0, Q4_1, Q5_0, Q5_1, Q8_0, and Q8_1 remain intact.

Verification

• Workspace test suite: 1,659 passed, 205 ignored.
• Host build matrix: CPU, CUDA, Vulkan, OpenCL, MediaTek, and Metal combinations passed.
• Release product checks and the v0.5.1 binary version smoke passed.
• Qwen3.6 Metal MTP and Hy3 Q2_K reproductions completed successfully; the Apple M5 Pro standard four-model suite completed with stable correctness hashes or token IDs.

runNburn v0.5.0

runNburn 0.5.0 more than doubles Gemma 4 26B-A4B end-to-end throughput by removing row-by-row CUDA transfers, keeping selected expert work on device, reusing quantized down weights across rows, and parallelizing large KVarN compaction batches. The release keeps mmap-backed GGUF weights and the existing quantized formats intact; it does not introduce expanded weight caches or model sidecars.

CUDA performance

The official release rerun used an RTX 3090 24 GB, Gemma 4 26B-A4B Q4_K_M with the external Q8_0 assistant selected automatically at k=1, a 1,115-token prompt, and 100 generated tokens. Values are medians after one warmup and three measured runs against the latest v0.4.0 baseline on the same device and workload:

• Prefill increased from 38.760 to 102.210 tok/s (+163.70%).
• Decode increased from 14.170 to 17.426 tok/s (+22.98%).
• End-to-end generation increased from 2.788 to 6.011 tok/s (+115.59%).
• All three measured runs used auto-MTP k=1, produced the same 100-token output hash, and accepted 47/53 drafted tokens (88.7%). A separate 10-token Korean correctness run produced the established exact hash and the correct answer, Seoul.

The release-baseline comparison above is the official product regression gate, not an interleaved attribution experiment. Each implementation step was also measured independently with warmup plus ABABAB on the same RTX 3090 and 1,115-token Gemma workload:

• Batched expert activations reduced prefill median by 8.12%, same-index 3/3.
• Batched expert normalization and residual operations reduced it by 16.30%, same-index 3/3.
• Chaining gate/up, GELU, and quantized down projection on device reduced it by 8.16%, same-index 3/3.
• Device-resident expert group gather, FFN, and weighted scatter reduced it by 14.04%, same-index 3/3.
• Four-row Q5_1/Q8_0 down-weight reuse reduced it by 17.15%, same-index 3/3.
• Ordered parallel KVarN block compaction reduced it by 22.09%, same-index 3/3.

Validation

• The official four-model regression suite passed on Qwen3.6 35B-A3B, Gemma 4 26B-A4B, Hy3 295B-A21B, and GLM-5.2. Qwen end-to-end throughput was within +0.35% of the latest baseline; Hy3 was within -1.68%; GLM showed no regression, but its 58.18% run variance is too high to claim an improvement.
• Qwen matched its established 100-token reference hash. Hy3 and GLM produced exact token IDs across all three measured runs.
• The CUDA backend library suite passed 623 tests with 45 explicitly ignored tests. The CPU, CUDA, Vulkan, OpenCL, and MediaTek build matrix passed.
• The release workflow passed all 10 jobs and published 9 platform archives with 9 matching SHA-256 files.

Full changelog: v0.4.0...v0.5.0

runNburn v0.4.0

runNburn 0.4.0 makes external speculative decoding automatic across the Gemma 4 family while keeping activation conditional on model support and available VRAM. It also hardens quantized CPU execution, backend memory ownership, GGUF loading, and CUDA cache identity.

CUDA performance

Measured with Gemma 4 Q4_K_M target models and Q8_0 assistant models on RTX 3090 24 GB, using matched on/off ABABAB runs. Every reported run produced token-exact output against the non-MTP path.

• Automatic Gemma 4 external MTP. Supported E2B, E4B, 12B, 26B-A4B, and 31B models now discover the standard MTP/mtp-*-Q8_0.gguf assistant automatically. The engine selects the draft length from model structure and admits the path only when the required VRAM headroom is available.
• Dense Gemma 4 generation medians improved by 23.1% (E2B), 24.3% (E4B), 65.5% (12B), and 76.3% (31B) at draft length 3. The 26B-A4B selected-MoE model improved by 11.2% at draft length 1 on RTX 3090 and by 30.6% on RTX 3060 12 GB.
• Gemma selected-MoE CUDA admission. The Q5_1/Q8_0-down CUDA path now activates from tensor shape, quant type, and measured free-VRAM headroom rather than model names. Force-vs-auto ABABAB overhead was 0.188% on RTX 3090 and 0.193% on RTX 3060, with exact output hashes.
• Q4 resident cache identity. Mmap-backed GGUF weights now use storage generation, offset, and length as their O(1) identity; temporary slices use a full content fingerprint. This prevents same-length weights with matching edges but different middle blocks from aliasing one CUDA allocation.

Fixed

• Preserved F16 KV-cache capacity during writes instead of shrinking allocations and invalidating later continuation state.
• Fixed shared-KV conversion and ignored-EOS correction in Gemma external MTP, restoring token-exact generation.
• Rejected unsupported GGUF architectures at load time instead of entering an invalid product path.
• Removed parallel output aliasing from Q4_K, Q5_K, and Q6_K CPU kernels; avoided unaligned quantized block and row references; validated F32 slice casts and dot-product lengths; handled AArch64 NEON attention tails.
• Retained Metal NoCopy source mappings for the full buffer lifetime, used the host page size for alignment, and enforced carrier shape contracts in release builds.
• Destroyed Vulkan buffers before freeing their backing memory.
• Eliminated CUDA test-suite flakes caused by process-wide environment races and sub-ULP atomic accumulation-order drift.

Validation

• Standard Qwen3.6-35B output remained byte-identical after the CUDA cache changes.
• The four-model CUDA rerun covered Qwen3.6 35B-A3B, Gemma 4 26B-A4B, Hy3 295B-A21B, and GLM-5.2; Qwen matched its established reference hash, Gemma auto-MTP remained stable, and Hy3/GLM token IDs were exact across repeated runs.
• The release workflow passed all 10 jobs: Linux CPU/CUDA/Vulkan/OpenCL, macOS Metal, Android CPU/Vulkan/OpenCL/MediaTek, and release publication.

Full changelog: v0.3.0...v0.4.0