Skip to content

model: add Nanbeige4.2 (Looped Transformer) support#100

Open
Andgihat wants to merge 1 commit into
Anbeeld:mainfrom
Andgihat:nanbeige-arch
Open

model: add Nanbeige4.2 (Looped Transformer) support#100
Andgihat wants to merge 1 commit into
Anbeeld:mainfrom
Andgihat:nanbeige-arch

Conversation

@Andgihat

Copy link
Copy Markdown

Adds support for the Nanbeige4.2 architecture (Nanbeige/Nanbeige4.2-3B, Apache-2.0) — a Looped Transformer: the physical decoder layers are reused num_loops times, so effective depth grows without adding parameters (22 physical layers × num_loops=2 → 44 effective).

Rebased onto current main, so the diff is a single clean commit.

What's included

  • New arch LLM_ARCH_NANBEIGE + KV keys nanbeige.num_loops / nanbeige.skip_loop_final_norm.
  • Graph builder src/models/nanbeige.cpp: standard RMSNorm + GQA + SwiGLU decoder with the loop unrolled over the logical (n_layer_all) layers. Physical weights are shared across loops via pointer copies (layers[i + j*n_phys] = layers[i]); each logical slot keeps its own KV index. An optional loop-boundary output_norm is applied between passes unless skip_loop_final_norm is set.
  • gguf-py constants + writer keys and a conversion/nanbeige.py converter.
  • Pure graph/arch code — no new ggml kernels.

load_arch_hparams expands the logical layer count via the existing n_layer (physical) / n_layer_all (logical) fields and replicates the per-layer arrays (n_head, n_head_kv, n_ff, is_swa_impl, is_recr_impl) across the loop copies, so the graph loop, KV cache and out-ids all size correctly with no changes to the base hparams API.

Verification

Built for Windows/CUDA 12.8 (sm_120) on top of current main and compared token-for-token against the authors' reference branch Nanbeige/llama.cpp@nanbeige42 on the same BF16 GGUF with greedy decoding — outputs are identical. Runs correctly with KVarN KV-cache (e.g. -ctk kvarn3 -ctv kvarn3) and long context.

GGUF quants (imatrix + KLD-guided mixed precision) are available at Andgihat/Nanbeige4.2-3B-GGUF.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant