freshdocs 0.5.0

0.4.0 loaded only the documentation a model cannot know, judged against a training cutoff copied from a vendor page. 0.5.0 measures the cutoff instead.

Measure, do not guess

freshdocs models --probe            # prints a 20-library recall question
freshdocs models --record <model-id> '<your JSON answer>'

Each answer is checked against the package registry: a real version yields its publication date, an invented one is recorded as a hallucination and does not count. The cutoff is the median of the verified dates, so one lucky late answer cannot pull it forward, and the per-library dates are kept, because coverage is uneven. An MCP client does the same in two calls to freshdocs_probe; freshdocs_gap returns the per-library verdict as JSON.

Precedence is now --cutoff > measured > models --set > shipped table.

What 16 models actually know

data/model_cutoffs.json ships measured cutoffs; data/README.md has the analysis. Three findings:

• Every measured cutoff is well behind the release date. Models shipped in 2026 recall library versions from late 2023 to mid 2025. A 2026 library is a training gap for all of them.
• Prompt wording moved the same model by a year. "Prefer an old version if unsure" pushed claude-fable-5-1 from 2025-05 to 2024-05 and minimax-m3 from 2024-11 to 2024-03, with zero hallucinations either way. Real knowledge discarded, nothing gained. The recall wording is now default; the registry check catches invention.
• Context changes recall. The same model id answered 2025-06-11 in a long session and 2025-05-16 as a fresh subagent. A measured cutoff describes a model in a context, which is why the safety margin stays and --cutoff overrides.

Retrieval fixes found while testing

• The budget is split per library by verdict. A library the model has never seen is guaranteed its share and listed first; a library that merely scores higher on the query words can no longer take every slot.
• A covered library's hit is a pointer (header and source URL, no prose). The model knows that release; the prose would repeat its training.
• Search walks a precise-to-loose plan and tops up rather than restarting, with a middle tier that requires any two query terms.
• CHANGELOG and MIGRATION rank below guide pages for usage questions; a question about upgrading lifts the penalty.

Four-library project, probed model: no --model 2430 tokens · one gap 1700 · all covered 240.

Install

uv tool install git+https://github.com/Supersynergy/freshdocs@v0.5.0
freshdocs models --import "$(python -c 'import freshdocs,os;print(os.path.dirname(freshdocs.__file__))')/../../data/model_cutoffs.json" 2>/dev/null || true