Architecture

Architecture

Overview of the BitForge repository layout, Python package structure, and implementation status.

Repository layout

BitForge/
├── bitforge/           # Core Python library
│   ├── cli.py          # Click CLI entry point
│   ├── core/           # Quantization engines
│   ├── calibration/    # Calibration dataset loaders
│   └── evaluation/     # Metrics and benchmark suites
├── experiments/        # Standalone experiment scripts
├── site/               # Interactive web dashboard
├── pyproject.toml      # Package metadata and dependencies
└── requirements.txt    # Pinned dependency list

Python package

bitforge.core

Module	Class	Purpose
quantizers.py	WeightQuantizer, QuantizationConfig	Base PTQ: scale/zero-point calculation, quantize/dequantize
gptq.py	GPTQQuantizer	Hessian-aware post-training quantization
awq.py	AWQQuantizer	Activation-aware weight scaling before quantization
gguf.py	GGUFConverter	Hugging Face → GGUF conversion for llama.cpp runtimes

QuantizationConfig controls:

• bits — target bit width (default 4)
• group_size — per-group quantization block size (default 128)
• symmetric — symmetric vs. asymmetric quantization
• use_double_quant — nested quantization of scales (GGUF-style)

bitforge.calibration

Module	Class	Purpose
dataset.py	CalibrationDataLoader	Load Wikitext-2, C4, or custom datasets as fixed-length token blocks

Calibration data drives GPTQ Hessian estimation and AWQ activation-scale measurement. Dataset choice affects final perplexity—see Quantization Theory.

bitforge.evaluation

Module	Symbols	Purpose
metrics.py	perplexity, memory_footprint, latency, detect_outliers	Core evaluation primitives
benchmarks.py	QuantizationBenchmarkSuite	Cross-bit-width benchmark orchestration

QuantizationBenchmarkSuite runs a model across bit widths and collects perplexity, VRAM, and tokens/sec into a pandas DataFrame.

bitforge.cli

Click-based CLI registered as the bitforge console script in pyproject.toml.

Experiments

Numbered scripts in experiments/ are self-contained entry points:

Script	Focus
01_perplexity_benchmarks.py	FP16 → INT8 → INT4 → INT2 perplexity sweep
02_latency_memory.py	TTFT, tokens/sec, peak VRAM
03_qlora_merge.py	Rounding error when merging LoRA into 4-bit base
04_outlier_detection.py	Activation spike detection and SmoothQuant simulation

Each script imports from bitforge and can be run directly with python experiments/NN_*.py.

Interactive site

site/index.html and site/app.js implement a client-side dashboard with:

• Quantization taxonomy reference
• Bit-width vs. quality/speed simulator (Chart.js)
• Outlier detection lab
• KaTeX-rendered formulas

No backend or build toolchain—static files only.

Implementation status

BitForge is scaffolded for extension. As of v0.1.0:

Component	Status
WeightQuantizer	Basic scale/zero-point logic implemented
GPTQQuantizer / AWQQuantizer	Interface + skeletal methods; full calibration loop TBD
GGUFConverter	Metadata validation stub; conversion TBD
CLI quantize / evaluate	Command structure in place; logs placeholder results
Experiment scripts	Runnable; some output is mock data until pipelines connect
site/ dashboard	Fully client-side, functional UI

Contributors extending a module should match existing dataclass/config patterns and add experiment coverage where behavior changes. See Contributing.

Data flow

Hugging Face model
       │
       ▼
CalibrationDataLoader ──► GPTQ / AWQ / PTQ quantizer
       │
       ▼
Quantized weights ──► GGUFConverter (optional)
       │
       ▼
evaluate / QuantizationBenchmarkSuite ──► metrics & reports

For runtime-specific export paths, see Runtime Integration.