GGUF Bridge

GGUF Bridge

Consumer-side checkpoint access in src/moe/checkpoint.rs, constants in gguf.rs, dequant in dequant.rs.

Supported tensor types

GGML type	ID	Use
F32	0	Routing tensors (required rank-2 F32)
F16	1	Preferred attn synapse path
Q8_0	8	Embedding dequant
Q5_K	13	Embedding dequant
IQ3_S	21	Detected; rejected for token embeddings with clear error

Wire / constants

• Magic GGUF, version 3
• Value-type IDs for KV metadata parsing (u8…f64, string, array)

Mapped access

MappedGgufCheckpoint (mmap via memmap2):

• Tensor directory + metadata
• f32_tensor / embedding extraction with dequant for supported quants
• Layout parse is frozen for enhancements pending engram-parser extraction (#8)

Dequant (dequant.rs)

Helper	Notes
dequantize_row_q8_0	34-byte blocks, width % 32 == 0
dequantize_row_q5_k	176-byte blocks, width % 256 == 0
f16_to_f32	via half crate
tensor_row_size	Q8_0 / Q5_K only

Routing orientation

Gate matmul supports both [hidden, experts] and [experts, hidden] F32 layouts (routing_weight_index).

Related

• Parser Boundary
• MoE Router

---

Last updated: July 29, 2026 Updated by: Grok Build: Grok 4.5 Package tip reference: 37c5a21 (main)