Skip to content

Quantization Lab

Jump to bottom
Manit Arora edited this page Oct 8, 2026 · 2 revisions

Quantization Lab

Overview

The AXIOM Quantization Lab analyzes a model against a selected agent workload and the target device.

Agent profiles

  1. Coding
  2. Reasoning
  3. Research
  4. General Assistant
  5. Automation
  6. Math
  7. Writing
  8. Multilingual

Workflow

Questionnaire
     ↓
Model selection
     ↓
System inspection
     ↓
Memory analysis
     ↓
Quantization recommendation
     ↓
Runtime bundle
     ↓
Real benchmark

Throughput

The default target is 10 tok/s.

AXIOM does not report 10 tok/s as achieved unless a real benchmark verifies the result on the target hardware.

Runtime bundle

AXIOM preserves:

  • Model weights
  • Tokenizer files
  • Configuration required for inference

AXIOM omits:

  • Documentation
  • Tests
  • Source-only files
  • Other repository artifacts that are not needed by the inference runtime

Deleting learned knowledge from neural network weights is not equivalent to removing files from a repository. Weight-level compression or capability removal requires a dedicated model transformation workflow.

CLI

axiom optimize profiles

axiom optimize run \
  --model Qwen/Qwen3-8B \
  --profile 1 \
  --target-tps 10

AXIOM AI source repository · Issues · MIT license

Clone this wiki locally