BeeLlama v0.4.0 + Nanbeige4.2 (Looped Transformer) — Win CUDA sm_120, KVarN

BeeLlama v0.4.0 + Nanbeige4.2 (Looped Transformer)

Adds nanbeige architecture support (the Looped Transformer used by Nanbeige4.2) on top of current BeeLlama main — so this build has the full feature set: KVarN variance-normalized KV-cache, low-bit KV (q2/q3), DFlash speculation, MTP, and more.

Nanbeige4.2 reuses its 22 physical layers num_loops=2 → 44 effective depth. Ported using the native n_layer/n_layer_all API and verified token-for-token identical against the authors' Nanbeige/llama.cpp@nanbeige42 reference (greedy). Pure graph/arch code — no new ggml kernels.

Branch nanbeige-arch · upstream PR #100.

Companion GGUF quants

Andgihat/Nanbeige4.2-3B-GGUF — imatrix + KLD-guided mixed-precision quants.

Platform

• Windows x64, CUDA 12.8, built for Blackwell sm_120 (RTX 50-series), native (no PTX JIT).

Long context with KVarN

Nanbeige4.2 is native 256K, but the looped arch keeps KV for all 44 logical layers (2× a normal 22-layer model). KVarN makes the full window fit on 16 GB:

llama-server.exe -m Nanbeige4.2-3B-UD-Q4_K_XL.gguf -ngl 99 -c 262144 -ctk kvarn3 -ctv kvarn3 -fa on --jinja

Included

Runtime DLLs (ggml-*, llama.dll, llama-common.dll, mtmd.dll) + tools: llama-server, llama-cli, llama-quantize, llama-imatrix, llama-perplexity, llama-completion, llama-bench, llama-tokenize, llama-gguf-split, llama-mtmd-cli.

License

MIT (inherited from llama.cpp / BeeLlama). Nanbeige4.2 weights are Apache 2.0, © the original authors.