Skip to content

v0.4.3 — how far, not just which way

Latest

Choose a tag to compare

@rohanvinaik rohanvinaik released this 29 Jul 01:29

Corpus and engine improvements. The network.db asset is updated — 11,178 models
re-extracted through the certifier, plus depth navigation and a corrected PageRank
transform in the query engine.

Existing v0.4.2 installs keep working; the v0.4.2 asset stays live. Re-download only if
you want the cleaner corpus.

The engine can now say how far, not just which way

A bank position is [SIGN][DEPTH], but navigate_models only ever carried the sign — so
efficiency=+1 admitted a 13B model and a 400B one on identical terms. "The most capable
reasoning model, compute no concern" returned 31B, 120B, 14B, 12B, 32B, 26B. Not
disordered — unfiltered.

min_depth and max_depth add the missing half, as a set-intersection filter over
candidates rather than a scoring term:

navigate_models(require_anchors=["reasoning"], efficiency=1,
                min_depth={"EFFICIENCY": 2})       # at least two steps out
min_depth candidates sizes returned
none 7,608 31B, 120B, 14B, 12B, 32B, 26B
{"EFFICIENCY": 2} 2,443 26B, 120B, 32B, 30B, 32B, 31B
{"EFFICIENCY": 3} 691 120B, 235B, 162B, 80B, 60B, 120B

max_depth is the ceiling, and applies at direction 0 as a band around the zero state.
Navigation filters; the scalar ranks. Depth deliberately does not enter the score —
folding scale into the ternary alignment factor would collapse two questions the
architecture keeps apart.

PageRank was being flattened by its own normalisation

pr_frac = pagerank / max(pagerank) is wrong for power-law data: on this corpus the
median equals the minimum, so dividing by the maximum pinned 96.9% of models below
0.01
— under 0.2% of final score. Six of the seven scoring factors are filters bounded
at 1.0 and cannot order anything; this was the only term that could, and it was silent.

Replaced with a rank percentile over the candidate set. Ties collapse to one percentile,
so the mass sitting at the baseline lands at 0.0 rather than being spread across the
bottom half. A 1-download experiment no longer outranks Phi-4-mini.

Corpus: 11,178 models re-extracted

Targeted rather than exhaustive — models that were demonstrably wrong were selected:
1,620 claiming a capability their pipeline_tag contradicts, 1,141 whose two independent
size estimators dissent, 418 carrying contradictory size anchors.

The anchor count falls 8.7% and that is the fix, not the cost. On a 400-model sample
of re-extracted "Instruct" models, base-model went 198 → 0 — an instruct model is a
fine-tune, and tagging it as a base model was always wrong. instruction-following held
(398 → 397) and chat rose (238 → 334). The certifier is removing false claims, not
thinning real ones.

Also now captured: tags, architectures, safetensors_total, safetensors_dtypes,
quantization_config, and license — which had been empty corpus-wide because HF's
ModelInfo has no .license attribute and every ingest path read
getattr(info, "license", "").

Anchor aliases work

aliases.py shipped in v0.4.0 with 48 seeded aliases and was never wired to anything:

v0.4.2 v0.4.3
require_anchors=["gguf"] 0 results resolves
["moe"] 0 results resolves
["mlx"] 0 results resolves

An unresolvable mention passes through unchanged, so resolution can only add matches —
it never widens a query by discarding a constraint it could not read. Unknown anchors are
now reported as unknown_anchors rather than returning a silent empty set.

The refinement loop no longer dead-ends

ranking_degraded asked for prefer_anchors while returning options: [], so a caller
driving the loop mechanically had nothing to merge. It now offers window-drawn anchors.
New refine.scope_unfiltered reports when a query supplied no require_anchors — the
only parameter that narrows the candidate set — since without one the window is the top
slice of an unfiltered corpus, not the best of a considered field.

A measuring instrument

python -m model_atlas.evaluation scores canonical questions against the real corpus,
with expectations as named predicates over recorded facts rather than expected model IDs,
so they survive corpus churn. It exists because 968 passing tests coexisted with a query
returning two 256M vision-language models for "a code review bot" — every other check
tests mechanism, none could say whether the answers were good.

96.5% → 97.2% across this release.

Honest limits

Extraction provenance is mixed: 22% of models went through the current certifier, 78%
carry their original extraction. The asset is replayable — script and input lists are
in-repo — but not deterministic, being the product of a particular pattern of HF
rate-limit failures. A complete pass would give a reproducible asset and is the right
basis for v0.5.0.

The ranking is better but still compressed, and there is no usage signal: PageRank
measures lineage ancestry, not popularity (log-log r = 0.259 against downloads;
all-MiniLM-L6-v2 has 249M downloads and ranks 333rd). Two eval cases still fail.

Install

git clone https://github.com/rohanvinaik/ModelAtlas.git && cd ModelAtlas && uv sync
mkdir -p ~/.cache/model-atlas
curl -L --fail -o ~/.cache/model-atlas/network.db \
  https://github.com/rohanvinaik/ModelAtlas/releases/download/v0.4.3/network.db

network.db (attached) — 50,973 models, 192 anchors, 48 aliases, 506,963 model-anchor
links, 406,111 positions. integrity_check: ok.

Full changelog: v0.4.2...v0.4.3