Home

BitForge Wiki

Forge smaller, run faster, measure honestly.

BitForge is an Inference Foundry initiative exploring the theory and practice of LLM quantization—compressing floating-point weights to lower-bit representations and measuring what that costs in quality, throughput, and memory.

What BitForge provides

Area	Description
Python library	Quantization wrappers (GPTQ, AWQ, PTQ), calibration loaders, and evaluation metrics
Experiments	Reproducible scripts for perplexity, latency, QLoRA merge error, and outlier detection
Interactive lab	Browser dashboard in site/ for exploring quantization trade-offs

Quick links

• Getting Started — install, first commands, hardware notes
• Architecture — repository layout and module map
• Quantization Theory — methods, granularity, calibration
• Experiments — running and extending benchmark scripts
• CLI Reference — bitforge quantize and bitforge evaluate
• Runtime Integration — llama.cpp, vLLM, HuggingFace, Super-Ollama
• Interactive Lab — local web dashboard
• Contributing — how to help and where to coordinate

Research questions

BitForge is built around questions we want to answer with open, reproducible measurements:

1. At what bit-width does a given model class start to degrade measurably on reasoning vs. factual recall tasks?
2. Does calibration data distribution matter more for smaller or larger models?
3. Can a lightweight automated benchmark detect quantization regressions fast enough to fit in a CI loop?

Status

The library and experiments are early-stage. Core modules define interfaces and skeletal implementations; benchmarks may return placeholder values until full pipelines are wired up. See Architecture for what is implemented today vs. planned.

Related projects

• Super-Ollama — consumes quantized GGUF artifacts at inference time
• Inference Foundry org — sibling repos and org-wide docs