v0.9.36: llmfit doctor, honest fit verdicts, and a self-updating model catalog — a stability milestone

[AlexsJones]

v0.9.36 is out, and it's a bigger release than the version number suggests — this one is all about making llmfit's answers trustworthy and making the project easier to maintain and contribute to. Here's what landed.

🩺 llmfit doctor — hardware bugs become one-command reports

Hardware detection has been our biggest source of recurring bugs (multi-GPU boxes, ROCm quirks, Intel Arc, APUs…). Each fix used to depend on whatever partial output a reporter happened to paste — and fixes for one GPU topology sometimes regressed another.

llmfit doctor ends that cycle. It captures the raw output of every tool detection relies on (nvidia-smi, rocm-smi, sysfs, lspci, system_profiler, WMI, vulkaninfo, npu-smi) plus what llmfit concluded, as one paste-ready report. The new bug-report template asks for it, and each report can be dropped verbatim into our regression test suite — two real user systems from #638 are already fixtures. Detection bugs now stay fixed.

And the fixes themselves:

• Multi-GPU AMD: a 32 GB Instinct MI50 that reports the generic name "AMD Radeon Graphics" was being silently dropped as an "integrated GPU". The discrete/integrated check is now VRAM-aware, and generic ROCm names get GFX-version disambiguation. (Detect multiple GPUs #638 — thanks @keyz182 and @cb88 for the raw output that made this diagnosable!)
• Intel GPUs: everything Intel showed 0.0 GB VRAM because the sysfs file we read is amdgpu-only. Integrated Arc (Lunar/Meteor Lake) is now a proper unified-memory device with the full RAM pool; discrete Arc gets real VRAM from Vulkan. Recommendations on Intel laptops flip from CPU-only to GPU mode. (Intel Arc Pro B70 not recognized #609)

📏 Honest fit verdicts

Two long-requested changes to make the score mean what you think it means:

• Usable context ([feature] Approximate usable context #621): a "Perfect fit" that leaves you 8k of context out of a 262k window is not a perfect fit for real work. The Ctx column now shows 262k→14k — native window → what actually fits on your machine after weights and KV cache — with a warning colour when it drops below 4k. Sorting by Ctx uses the achievable number, and it's in the JSON output as usable_context. Credit to @MrMarble for the design.
• Use-case-aware quality (Feature Request: Weighted “Best Model” Score per Use Case (e.g., Coding Agent) #150): scoring now uses a curated per-family benchmark table (coding / reasoning / chat), so a genuinely strong coding model outranks a bigger generalist for --use-case coding — parameter count no longer dominates. The table ships in the repo and corrections are welcome PRs. Thanks @mvanhorn for the proposal this is built on.

📦 The model catalog keeps itself fresh

• Weekly automated refresh: a scheduled workflow re-scrapes the catalog every Monday, validates it against a JSON Schema, and opens a PR. No more waiting for a release to see new models. Huge thanks to @Romeo-mz for building this (and the new issue templates!).
• Add your own models locally (Suggestion: Give end users a quick way to add models to the db #451, How to update the models list with a brew install? #296): drop a custom_models.json into llmfit's data directory (or point LLMFIT_CUSTOM_MODELS at a file) to add or override any model — no rebuild, no release, works with brew/scoop installs. See the "Adding your own models locally" section in the README.
• Plus ~11 MB of duplicated catalog data removed from the repo.

🤝 Contribution pipeline

We also cleared the review backlog — long-stalled PRs have been merged or closed with proper explanations, and several ideas from older PRs (Windows paths, sharded downloads, Ollama pulls, MoE scoring) had already shaped what's in main; that credit is now recorded on each PR. If your PR sat quiet for a while: sorry, and it won't be the pattern going forward — the new templates, weekly automation, and regression-fixture flow are exactly about keeping the loop fast.

Get it

brew upgrade llmfit        # or: scoop update llmfit
uv tool install -U llmfit  # or grab a binary from the release page

Full changelog: https://github.com/AlexsJones/llmfit/releases/tag/v0.9.36

If llmfit misdetects your hardware, run llmfit doctor and open an issue with the output — it'll likely become a regression test within days. That's the stability loop we're most excited about. 🚀