MicroAnt

Small models, heavy lifting.

A reproducible ruler for what quantization does to a small model's capability — not its speed, and not its perplexity.

---

LocalScore already tells you how fast your device runs a model. MicroAnt asks the question after that one: is the output still usable?

"Don't go below Q4 for agent tasks." "Reasoning falls off a cliff at low bit-widths." Everyone repeats these. Nobody publishes a ruler you can re-run. MicroAnt is that ruler: it takes one model, runs the same capability pack across quantization tiers and across the llama.cpp/GGUF and MLX ecosystems, and reports where the cases die — with a bootstrap confidence interval on every number and an explicit refusal to compare things that aren't comparable.

It is deliberately not a leaderboard. Speed appears only as a cost column.

Status

v0.1.0. MicroAnt is a family-scoped evaluation workflow, not a fixed-model campaign. The product boundary is: choose a model family, select the available chains / tiers for that family, then run profile → prescribe → confirm → compare to produce a reproducible report. The registry entries in this repo are reference families used to prove that workflow, not the definition of the product itself.

Four commands are built — profile, prescribe, confirm, compare. profile runs a pack, checks the equivalence guards, and writes a report. prescribe picks the lowest scaffold rung (L0 bare → L3 self-check retry) that clears a quality bar, per cell, on the dev half. confirm closes the loop: it re-measures only the rung prescribe chose, against the held-out test half, and a rung counts as held only if the test CI's lower bound clears the bar — the point estimate alone is not enough. compare answers the one question whose two arms live in different runs: does scaffolding close a parameter-count gap?

The workflow is proven on one real family: qwen3. The committed reports are qwen3-1.7b's profile and prescribe, both structured@0.7.0 over the 250-case pack — and both are currently the MLX-only half of the run (the six mlx-community tiers 3bit…bf16). The GGUF ladder was evicted from local disk and is being re-fetched over a slow link; until it lands, the committed reports carry the MLX arm alone, and say so. gemma4-e2b, the confirm run, and the GGUF-side tables have been pruned from reports/ and rejoin it when a run republishes them. qwen3-8b remains the next reference target for same-family compare and the v2 Pi-agent capability-floor check.

✅ Built	profile, prescribe, confirm, compare, pack init, equivalence guards, layered scoring, bootstrap CIs, charts, multimodal (VLM) memory accounting, conversion-chain grouping, the full scaffold ladder (L1 grammar · L2 few-shot · L3 self-check retry, both ecosystems), the full quantization ladder (6 GGUF tiers · 5–6 MLX tiers), the multi-turn depth axis with length controls + the two decay curves, the cost column, the dual-ring radar (bare vs scaffolded), dev/test split + test-set usage ledger, within-chain failure attribution (§8 v1.4), a worked BYOD example
🚧 Not built	the figure atlas site (v1.5)
✅ Workflow proven on	the qwen3 family (qwen3-1.7b profile + prescribe, structured@0.7.0, 250-case pack)
📤 Committed reports today	qwen3-1.7b profile + prescribe, MLX arm only (the GGUF ladder is re-downloading); confirm and gemma4-e2b reports pruned until republished
⏳ Reference-family work still open	the GGUF arm of qwen3-1.7b; qwen3-8b baseline runs for same-family compare and the v2 Pi-agent floor check; gemma4-e2b deferred as second-family validation
Registry entries today	three reference entries: qwen3-1.7b · qwen3-8b (the same-family pair) · gemma4-e2b (multimodal VLM)
Packs shipped	one: structured (250 cases — sized by power analysis, not by appetite), plus a worked BYOD example

The design's narrative is coach, not doctor: profile → prescribe → confirm. All three commands are built and proven; the committed artifacts currently cover profile and prescribe (confirm's report was pruned with the rest and returns when it is re-run).

Install

Python ≥ 3.11. The logic layer and reports run anywhere; running actual models needs Apple Silicon.

git clone https://github.com/Code-byte404/microant
cd microant
pip install -e ".[dev]"        # logic, scoring, reports, charts
pip install -e ".[inference]"  # llama-cpp-python + mlx-lm — Apple Silicon only
pip install -e ".[spike]"      # requests — only needed for the v2 Pi-agent spike (scripts/spike/pi_agent/)
/opt/miniconda3/envs/microant/bin/python -m pytest   # 637 passed, 14 deselected

The passing tests need no GPU and no downloaded weights. That is the layer worth trusting: the parser, the scorer, the grammar builder, and the report are pure functions over fixtures.

The suite is fully green on a fresh clone. It was long red by two — a pack-staleness and a schema-staleness detector both firing on gemma4-e2b's stale committed prescribe — and those cleared when the gemma4-e2b reports were pruned from reports/; the golden tests now parametrize over the one published model, qwen3-1.7b. The detectors were doing their job while they were red, and they rejoin gemma4-e2b in the same change that republishes its report.

Run

microant family qwen3
# choose one concrete member from the family list, then run it:
microant profile \
  --model qwen3-1.7b \
  --backends gguf,mlx \
  --tiers Q4_K_M,f16/4bit,bf16 \
  --pack packs/structured \
  --out reports

--tiers is <gguf tiers>/<mlx tiers>. The command writes reports/profile-<model>.json, reports/profile-<model>.md, and a damage + decay PNG per conversion chain.

MicroAnt is family-first at the selection layer: start from a family query (for example qwen3 or gemma4), then choose the concrete member whose ladders / baselines / tiers you want to measure. Reports are still per concrete member, because the actual runnable artifacts live at that level.

A full ladder is a dozen model loads and tens of thousands of generations, so profile and prescribe checkpoint each (backend, tier) cell the moment it finishes. Re-run the same command after a crash and the finished cells are reused rather than regenerated — the run picks up where it stopped. Reuse is refused, out loud, if the pack's sha256 or the software stack has changed since the checkpoint was written: those cells measure something else, and splicing them into a table that prints one env header is the confound the whole harness exists to avoid.

A committed example lives in reports/profile-qwen3-1.7b.md — currently the MLX arm of the run (six mlx-community tiers). It measures each chain only against its own 16-bit baseline, never against another chain's; the GGUF arm and a second conversion chain rejoin the report when the GGUF ladder finishes downloading. A multimodal example (gemma4-e2b, whose bpw counts the language tower only, since both runtimes drop the vision/audio weights) rejoins reports/ when that deferred second family is re-run.

UI

A local Streamlit app — a thin shell over the same CLI (it assembles a microant … command, runs it, and reads the artifacts; it reimplements no measurement logic):

pip install -e ".[ui]"
streamlit run src/microant/ui/app.py

Three modes:

• Run — pick a family, then a concrete member, then a stage (profile / prescribe / confirm / compare), backends, and quant tiers; the exact microant … command is shown before you launch, and per-cell progress is read straight from the checkpoint file the CLI already writes. A ✓/⬇ marker flags which tiers are already downloaded.
• Models — search HuggingFace (type qwen, filter GGUF/MLX), see each repo's quant files with sizes, and download just the one you need. Results are badged profilable ✓ when the repo backs a registered chain, or needs a registry entry otherwise — downloads run through hf and are tailed live, with free disk shown, since downloads are the real constraint.
• Browse reports — each report's rendered Markdown, its damage/decay/radar PNGs, and the raw JSON.

Set MICROANT_UI_LOCAL=0 to expose only the read-only report browser (for hosting); MICROANT_REPORTS=<dir> points it at a different reports directory.

What a report says

What follows is real output — Qwen3-1.7B on an M2 Pro, 250 cases, greedy decoding, the MLX arm.

Where cases die

Scoring is layered: parse_ok → schema_valid → content_correct. Every metric carries a bootstrap 95% CI over the case set. Every table is rendered twice, under a strict and a lenient parser — and the difference between the two tiers is itself a finding here, not a footnote.

At the 16-bit baseline the model scores 0.76 [0.71, 0.82] content_correct under the lenient parser and 0.52 under the strict one. That 0.24 gap is the parser's, not the model's: strict demands the entire reply be pure JSON, so it rejects the <tool_call> envelope every function_call case emits — and fails all 80 / 80 of those cases at every tier, baselines included. So strict does not report a stricter estimate of capability; it reports capability minus a whole task it cannot read. Worse, that blindness corrupts strict's quantization Δ in both directions — it masks real losses (a case wrong-after-quantizing that strict scored 0 before and after → Δ 0) and books phantom gains (quantization breaks the wrapper off a still-correct answer, which strict then reads). Read the lenient rows for the model; read strict for what a bare-JSON pipeline gets — and never read a strict Δ on a task strict cannot parse (§8 v1.10).

Quantization damage, within one ecosystem

Abridged from the report (mlx-community, lenient tier, baseline bf16) — the full table also carries weights, parse_ok, schema_valid, and the Δ CI width, under both parser tiers:

| tier | bpw   | memory  | content_correct   | Δ content_correct    | credible     |
|------|-------|---------|-------------------|----------------------|--------------|
| bf16 | 16.00 | 3.44 GB | 0.76 [0.71, 0.82] | baseline             | —            |
| 3bit |  3.73 | 0.75 GB | 0.50 [0.44, 0.56] | -0.26 [-0.32, -0.20] | yes          |
| 4bit |  4.50 | 0.97 GB | 0.78 [0.73, 0.83] | +0.02 [-0.01, +0.05] | inconclusive |
| 8bit |  8.50 | 1.83 GB | 0.76 [0.71, 0.82] | +0.00 [+0.00, +0.00] | inconclusive |

3bit is the only established damage on the MLX ladder — a −0.26 lenient drop whose CI clears 0 by a wide margin. From 4bit up the tiers are baseline-indistinguishable: their Δ CIs straddle 0 (inconclusive = underpowered-or-null, never "harmless"), and 8bit prints +0.00 [+0.00, +0.00] — exact agreement with bf16 on all 250 cases, which the report labels as exact agreement on this pack, not infinite precision. The sub-4-bit story continues on the GGUF ladder (Q2_K, Q3_K_M), which is why that arm is being re-fetched; it is not in this committed table yet.

prescribe — which scaffold rung to deploy

microant family qwen3
# choose one concrete member from the family list, then run it:
microant prescribe \
  --model qwen3-1.7b \
  --backends gguf,mlx \
  --tiers f16,Q4_K_M/bf16,4bit \
  --pack packs/structured \
  --threshold 0.9 \
  --out reports

Run the installed console script (microant prescribe ...), not python -m microant.cli — the CLI module has no __main__ guard, so invoking it that way silently does nothing.

What it answers: the lowest scaffold rung — L0 (bare model) or L1 (grammar-constrained decoding) — that clears a quality bar, per (backend, tier) cell. --threshold (default 0.9) is never a hidden constant; it is recorded in the report header (threshold: 0.90) alongside the metric and decision tier, so a prescription can always be traced back to the bar it was measured against.

Decision tier is lenient (metric content_correct), and the report says why: the strict parser requires the entire reply to be pure JSON, so it returns None for the <tool_call> envelope every function_call case emits — capping any cell at 16/24 under strict and making a 0.9 bar unreachable for reasons that have nothing to do with the scaffold. prescribe is answering "which scaffold should I deploy," and a real deployment parses the envelope, so lenient is the honest deployable-capability measure. Strict is still computed and reported alongside, never hidden.

Grammar coverage (constrained_ok / total) is the honest denominator behind any L1 number: a case the grammar builder can't express, that the backend can't compile, or that truncated is not counted as constrained. The L0-vs-L1 delta is computed only on the constrained_ok subset, so an uncovered case can never dilute the measurement.

L1 ships for both ecosystems (scaffold.L1.backends == ("gguf", "mlx")): a GBNF grammar on the llama.cpp side, an outlines-core logits processor on the MLX side. The tool-call envelope the grammar wraps is declared per model, read off that model's own chat template — never hardcoded in the builder — so a model is never masked into another family's wire format just to make it parse. A model whose envelope isn't declared has its function_call cases counted build_unexpressible, and coverage drops honestly.

The committed run is reports/prescribe-qwen3-1.7b.md — the four MLX representative tiers, on the 128-case dev half. It was measured at threshold 0.80, not the 0.9 default. A pre-registered 0.9 run prescribed none on every tier (the model's lenient ceiling is 0.76); the bar was then lowered to 0.80 as a deployer's product decision, so this table is read as post-hoc, not as clearing the pre-registered line. The GGUF tiers (Q2_K…f16) rejoin the table when that ladder finishes downloading.

qwen3-1.7b, MLX, threshold 0.80 — three of four cells clear the bar:

cell	bpw	lenient content_correct, L0	lenient content_correct, L1	prescription
mlx/3bit	3.73	0.48 [0.40, 0.57]	0.64 [0.55, 0.73]	none (ceiling below the bar)
mlx/4bit	4.50	0.79 [0.72, 0.86]	0.86 [0.80, 0.91]	L1 (grammar) — tentative
mlx/5bit	5.87	0.80 [0.73, 0.87]	0.87 [0.80, 0.92]	L0 (bare) — tentative
mlx/bf16	16.00	0.78 [0.70, 0.85]	0.84 [0.77, 0.90]	L1 (grammar) — tentative

Read the prescriptions off the lowest sufficient rung: 5bit clears 0.80 bare, so it needs no scaffold; 4bit and bf16 need the L1 grammar to cross; 3bit cannot be carried over at all (its best rung tops out at 0.65). Every "clear" here is tentative — the point estimate is over the bar but the CI lower bound is not — so the report labels it so rather than rounding up.

Grammar buys about +0.06–+0.08 per cell, and that is enough exactly twice. L1 lifts 4bit 0.79 → 0.86 and bf16 0.78 → 0.84 — over the 0.80 bar — but the same lift leaves 3bit at 0.64, short. L2 (few-shot) and L3 (self-check retry) are no better than L1 on this model — often slightly worse — so the ladder's useful rung is L1 and nothing above it. And what the grammar repairs is format-layer refusals (a case that answered in prose instead of emitting a tool call), never wrong values: a grammar cannot fix a hallucinated email character or a dropped timestamp by construction. So headline experiment §11.3 stays essentially negative — constraining the output buys back envelope compliance, not capability.

Two things the L1 numbers do not say for themselves, so the report says them:

• Tool selection IS measured under L1, not forced. The grammar unions all the tools a case declares and never only the expected one — a real anti-cheat property, covered by unit tests. Every function_call case in structured declares all three registered tools (get_weather, convert_currency, search_web) — the committed run records tools_per_function_call_case: [3] — so the union is a genuine three-way branch and choosing among them stays the model's job under L1. The committed report carries the per-run selection result; the residual function_call failures it shows are refusals to call (rescued by the grammar), not mis-selections.
• The two backends' L1 grammars are not the same constraint. GBNF permits whitespace around the envelope; the outlines-core regex does not. That is a whitespace difference, never a content one, and it never lands inside a comparison — every L0-vs-L1 delta is computed within a single cell. (The committed table is MLX-only for now, so only the outlines-core side appears in it.)

What binds a number

MicroAnt's quality numbers are bound to a four-tuple: (runtime, kernel backend, quantization file, sampling config). Inside that tuple the results are reproducible token for token. Across kernel backends they do not transfer.

That last sentence is a measurement, not a caveat. Same GGUF file, same greedy sampling, temperature 0: llama.cpp's CPU and Metal kernels produce identical output on only 2 of 10 prompts. Float reduction order differs, near-tied logits flip at the argmax, and the sequences diverge — sometimes at the first character. See docs/spikes/llamacpp-spike.md.

So the community's shorthand that "temperature 0 means reproducible" holds only within one kernel path. A Mac running Metal and a phone running llama.cpp on NEON will not agree, and no amount of seed-fixing changes that. This is exactly the kind of widely-assumed claim MicroAnt exists to go measure.

Two rules that make the numbers mean something

1. Lenient parser, strict scorer. The parser rescues anything rescuable; the scorer judges without mercy. Keeping them separate is what lets a failure be attributed to the right layer. Both parser tiers are always reported, so you can see which conclusions survive the parser and which are artifacts of it.

2. Ecosystems are never compared by tier name. GGUF's Q4_K_M is mixed precision — 169 q4_K tensors and 29 q6_K tensors, measuring 5.05 bits per weight. MLX's 4bit is uniform 4-bit at 4.50 bpw. They do not quantize the same tensors. Putting them side by side because both say "4" is a category error, and no rendered line of a MicroAnt report may contain two ecosystems' tier names — there is a test that scans the markdown to enforce it.

MicroAnt v1.2 is within-framework only: there is no cross-ecosystem comparison at all. Each chain is reported only against its own 16-bit baseline; two chains are never subtracted from each other or ranked against one another.

Bits-per-weight is measured per artifact, never taken from a declaration. The two ecosystems don't even agree on how many weights the model has: GGUF's conversion chain materializes Qwen3's tied lm_head as a separate tensor (2,031,739,904 weights) while MLX shares it (1,720,574,976). The difference, 311,164,928, is exactly vocab 151,936 × hidden 2,048. A conversion chain that duplicates a tensor pays for it on the memory axis, which is correct.

Four guards run before any metric is produced: tokenizer_equivalence, template_fidelity, gguf_template_drift, tensor_count_parity. A model that fails them is excluded with the reason on record.

Capability packs

A pack is a versioned, content-hashed bundle of cases plus the JSON schemas and matching rules that define a right answer. It is not merely a dataset, because a dataset doesn't carry its own grading rules — and the grading rules are where the honesty lives.

MicroAnt's published numbers are answers about MicroAnt's pack. Your question is different — what does Q4 do to your task — and only your data can answer it. microant pack init scaffolds a pack over it; docs/byod.md walks the whole flow, and examples/support-triage/ is a complete worked pack in a different domain, loaded and scored in CI so it cannot rot into an example that no longer works. Nothing you author leaves the machine.

Each case declares, per argument, how that argument is compared:

rule	for	behavior
exact	identifiers, enums	character-exact
number, iso_minute	amounts, timestamps	normalized, then exact
set	unordered collections	order-insensitive
equiv	declared synonyms	case-by-case list
text	names, titles, places	casefold + collapse whitespace, then exact — word order still matters
semantic	paraphrasable free text	Jaccard overlap of content tokens above a threshold

The set of rule kinds is closed; an unknown rule fails at load time. Silently degrading an unknown rule to exact was a real bug once.

semantic is a lexical proxy, and the report says so, with its three disclosed failure modes. It can accept a wrong answer that drops a discriminating token. Against a short gold string it can accept a single substituted word — swapping winner for loser scores 0.714, identical to a genuine paraphrase, so no threshold separates them. And lacking a stemmer, it can reject a correct paraphrase that merely pluralizes a word. Structured fields — emails, dates, amounts, identifiers — are never scored this way; one hallucinated character stays a failure.

The reproducibility contract

to_markdown(run) is a pure function. Feeding it the committed profile-qwen3-1.7b.json regenerates the committed profile-qwen3-1.7b.md byte for byte, and a test enforces it. Every number in the report is therefore re-derivable from the run JSON, which ships in the repo.

That is not a property of one favoured report. Both committed runs — qwen3-1.7b's profile and prescribe — ship their JSON and are held to the same byte-exact projection by parametrized golden tests. A published claim with no committed JSON behind it is a claim this repo cannot keep, so it does not get published; that is exactly why confirm and the gemma4-e2b reports were removed from reports/ rather than left as .md claims with no re-runnable JSON beside them.

Charts are the one carve-out: they are regenerable derived views, not byte-pinned artifacts, because PNG bytes differ across machines (freetype rasterization). Every number a figure draws also appears in the .md and the .json. A separate test checks that no report ever links a chart it didn't ship.

Every report header records chip, OS, the exact llama_cpp / mlx_lm / transformers / numpy versions, the model repo, and the pack's version and sha256. Sampling is greedy at temperature 0. There is no seed variance at temperature 0, so confidence intervals come from bootstrap resampling over the case set — never from re-running with different seeds.

Scope

Covered: the llama.cpp/GGUF and MLX ecosystems on Apple Silicon, reported per kernel backend.

Not covered: CoreML/ANE, QNN, ExecuTorch, MediaPipe/LiteRT. Their quantization formats and kernels differ enough that nothing here has any force for those deployment paths. They are long-term backend candidates, not implied coverage.

A boundary worth naming: MicroAnt renders every prompt from the source repo's chat template, for both backends, so that a template difference can never be confounded with quantization damage. A stock llama.cpp deployment uses the template embedded in the GGUF instead — and for this model those two differ (the gguf_template_drift guard records it). These numbers describe the model. They do not describe an out-of-the-box llama-server.

Prior art

MicroAnt is a measuring stick, and it tries hard not to claim other people's ground.

• Speed is not ours. LocalScore (Mozilla Builders) already answers "how fast does my hardware run this." MicroAnt publishes no speed leaderboard; tok/s is a cost column beside a quality column.
• "Quantization hurts quality" is not our insight. It is community consensus. Our claim is narrower: those claims currently have no reproducible ruler.
• We do not compete with BFCL or τ-bench. They rank how strong a model is. We measure how much of one model survives being quantized and converted, and what repairs it.
• No "first to discover X" claims. Not in this README, not in the reports.

License

MIT — see LICENSE.