GGUF VRAM Calculator

Calculate VRAM requirements for GGUF models based on the GGUF specification.

Features

• Parses GGUF model files to extract metadata
• Calculates VRAM usage for different model architectures
• Supports multiple architectures (LLaMA, MPT, GPT-NeoX, GPT-J, GPT-2, BLOOM, FALCON, Mamba, RWKV)
• Handles different quantization schemes
• Provides VRAM estimates for various context lengths

Usage

# Calculate VRAM with default context length (2048)
vram-calculator model.gguf

# Calculate VRAM with specific context length
vram-calculator model.gguf 4096

# Calculate VRAM with specific context length and KV cache type
vram-calculator model.gguf 4096 fp8

Supported Quantization Schemes

• F32 (4 bytes per weight)
• F16 (2 bytes per weight)
• Q4_0 (0.5 bytes per weight)
• Q4_1 (0.5625 bytes per weight)
• Q5_0 (0.625 bytes per weight)
• Q5_1 (0.6875 bytes per weight)
• Q8_0 (1 byte per weight)
• Q8_1 (1.125 bytes per weight)
• Q2_K (0.25 bytes per weight)
• Q3_K (0.375 bytes per weight)
• Q4_K (0.5 bytes per weight)
• Q5_K (0.625 bytes per weight)
• Q6_K (0.75 bytes per weight)
• Q8_K (1 byte per weight)
• IQ2_XXS (0.25 bytes per weight)
• IQ2_XS (0.3125 bytes per weight)
• IQ3_XXS (0.375 bytes per weight)
• IQ1_S (0.5 bytes per weight)
• IQ4_NL (0.5 bytes per weight)
• IQ3_S (0.375 bytes per weight)
• IQ2_S (0.3125 bytes per weight)
• IQ4_XS (0.375 bytes per weight)
• I8 (1 byte per weight)
• I16 (2 bytes per weight)
• I32 (4 bytes per weight)
• I64 (8 bytes per weight)
• F64 (8 bytes per weight)
• IQ1_M (0.5 bytes per weight)
• BF16 (2 bytes per weight)

Example Output

=== Model Information ===
Architecture: qwen3moe
Name: Qwen3-Coder-30B-A3B-Instruct
Author: 
Version: 
Base Name: Qwen3-Coder-30B-A3B-Instruct
Finetune: Instruct
Size Label: 30B-A3B
Quantized By: Unsloth
File Type: 7
File Size: 30.25 GB
Alignment: 32
Context Length: 262144
Embedding Length: 2048
Block Count: 48
Feed Forward Length: 5472
Head Count: 32
Head Count KV: 4
Expert Count: 128
Expert Used Count: 8
RoPE Dimension Count: 0
Parsed Size - Expert Count: 0, Parameter Count: 30B-A3B

=== VRAM Calculation ===
Estimated VRAM Usage: 37.49 GB
Context Length: 92000 tokens
KV Cache Type: fp16

=== VRAM Usage for Common Context Lengths ===
Context 512: 33.30 GB
Context 768: 33.31 GB
Context 1024: 33.33 GB
Context 2048: 33.37 GB
Context 4096: 33.47 GB
Context 8192: 33.65 GB

Building

go build -o vram-calculator main.go

License

Apache 2.0, see LICENSE.md