feat(model): curate the built-in catalog by hardware target#84
Conversation
juhovainio
left a comment
There was a problem hiding this comment.
Overall the design is clean — embedding a JSON catalog, parsing it once with OnceLock, and reusing the existing recipe-index schema is a good approach. A few findings below, ranging from a medium bug to a doc nit.
|
Two follow-up suggestions for this PR or a future iteration: 1. Move platform definitions into
"platforms": [
{
"label": "Strix Halo (Lemonade / llama.cpp)",
"engines": ["lemonade", "llama.cpp", "llamacpp"],
"gfx_families": ["gfx120X-all"]
},
{
"label": "MI300X (vLLM)",
"engines": ["vllm"],
"gfx_families": ["gfx906", "gfx908", "gfx90a", "gfx90a-all"]
}
]
2. Highlight the current platform in The gfx target detection already exists ( This makes it immediately clear which section is relevant for the user's hardware without them having to know which GPU maps to which platform. The detection is a fast sysfs read on Linux / WMI probe on Windows, same cost as |
`rocm model` listed every built-in recipe flat, including tiny smoke/test paths, and did not say what the list was for. Turn it into a short curated list of current popular open-weight models, each sized to a quant that fits a single GPU (multi-GPU serving is not supported). - Author the built-in list as an embedded JSON file using the same schema as a signed recipe index, so the offline default and hosted indexes share one format and the catalog is easy to curate. A unit test parses and validates it so a malformed catalog cannot ship. - Feature current popular models under canonical Hugging Face ids with the quant that fits one GPU: Qwen3.6-35B-A3B and Gemma 4 26B-A4B as Q4_K_M GGUF on Strix Halo (served via Lemonade's owner/repo:variant form), and Qwen3.6-27B and Gemma 4 31B as BF16 on one MI300X via vLLM. - `rocm model` shows only the featured list for the built-in catalog, grouped by hardware target, with a note that any compatible Hugging Face model can be served. A configured recipe index is shown in full and named in the header. The default assistant, smoke/test, and superseded recipes stay resolvable for `rocm serve` and appear under `--verbose`. Signed-off-by: Roman Inflianskas <Roman.Inflianskas@amd.com>
- Add `featured` boolean field to ModelRecipeRecord; mark the four curated models in model_catalog.json. model_recipe_featured() now reads the field directly, eliminating the hardcoded FEATURED_MODEL_IDS Rust constant that had to be kept in sync by hand. - Add `platforms` array to ModelRecipeIndexDocument with label, engines, and gfx_families per platform. MODEL_CATALOG_PLATFORMS constant and the engine-name match arm are removed; model_recipe_target_platform_label() looks up the recipe's engine against the catalog's platform list instead. Signed indexes that omit platforms fall back to the built-in definitions. - rocm model highlights the matching platform with "← your GPU" by probing detect_host_gfx_target() + normalize_therock_family() and matching against each platform's gfx_families list. - Footer "These are recommendations..." is now only emitted for the built-in catalog, not for operator-configured SignedIndex registries. - Remove dangling gpt-oss-120b manual_alternative entries (no recipe in the catalog resolves that id). - Fix docstring: catalog validation is test-time, not build-time. Signed-off-by: Juho Vainio <juho.vainio@amd.com>
Signed-off-by: Juho Vainio <juho.vainio@amd.com>
Signed-off-by: Juho Vainio <juho.vainio@amd.com>
Add three hardware platform entries to model_catalog.json replacing the previous two (Strix Halo / MI300X): - AMD Ryzen AI — Strix Halo (Lemonade / llama.cpp): gfx1151 - AMD Radeon — RX 7000/9000, Radeon PRO (Lemonade / llama.cpp): gfx110X-all, gfx120X-all - AMD Instinct — MI300X, MI350X, MI355X (vLLM): gfx906, gfx908, gfx90a, gfx94X-dcgpu, gfx950-dcgpu Lemonade recipes now appear under both Strix Halo and Radeon by matching per-platform engine lists directly rather than assigning each recipe to only the first matching platform. Signed-off-by: Juho Vainio <juho.vainio@amd.com>
2b9f761 to
3b841e4
Compare
What
rocm modeldumped every built-in recipe as a flat list — including tiny smoke/test paths — and didn't say what the list was for. This turns it into a short, curated catalog of current popular open-weight models, grouped by the hardware path each targets, with each entry sized to a quantization that fits a single GPU (multi-GPU serving isn't supported).How
model_catalog.json) using the same schema as a signed recipe index, so the offline default and hosted indexes share one format and the catalog is easy to curate. A unit test parses and validates it, so a malformed catalog can't ship.owner/repo:variantGGUF form thatrocm serveneeds (via the Lemonade canonical-name serving path); MI300X entries serve at BF16.rocm modelshows only the curated featured list for the built-in catalog, grouped by hardware target, plus a note that any compatible Hugging Face model can be served. A configured recipe index is shown in full and named in the header.rocm serveand are listed underrocm model --verbose(annotated as hidden) — nothing is removed from resolution.Testing
cargo test -p rocm-core(incl. catalog parse/validate + featured/platform helpers) andcargo test -p rocm --bin rocm modelpass.cargo clippy/cargo fmtclean.