Bug Description
When a model name exists in both custom_providers and Hermes' built-in CURATED_MODELS registry, the built-in mapping silently wins. This causes /model <name> to route to the wrong provider, wrong base_url, and wrong API key — 401 auth failure, then fallback to an unintended model.
Root Cause
hermes_cli/models.py hardcodes gpt-5.5 under openai-api:
# models.py line 247-255
"openai-api": [
"gpt-5.6-sol",
...
"gpt-5.5", # ← hardcoded
"gpt-5.5-pro", # ← hardcoded
...
],
But the user has a custom_providers entry:
custom_providers:
- name: lmuai
base_url: https://api.lmuai.com/v1
api_key: sk-xxx
When the user runs /model gpt-5.5, Hermes:
- Looks up
gpt-5.5 in CURATED_MODELS → matches openai-api
- Sets
model.provider = openai-api
- Does NOT check custom_providers for a matching model
- Uses stale/wrong
model.base_url (e.g. api.deepseek.com/v1 from previous provider)
- Sends
gpt-5.5 to DeepSeek with DeepSeek's API key → 401
- Falls back to first
fallback_providers entry (gemini)
Evidence from agent.log
01:03:45 - API call failed:
provider=openai-api
base_url=https://api.deepseek.com/v1 ← wrong!
model=gpt-5.5
HTTP 401: Authentication Fails
01:03:45 - Fallback activated: gpt-5.5 → gemini-2.5-pro (google)
Workaround
The user must explicitly specify the custom provider: /model lmuai/gpt-5.5. Without the prefix, the built-in registry wins every time.
Expected Behavior
When resolving a model name, custom_providers entries should be consulted BEFORE or at equal priority to the built-in CURATED_MODELS. If a model name appears in both, the custom_provider's base_url/api_key should take precedence — the user explicitly configured it.
Fix Suggestion
In the model resolution path (likely hermes_cli/models.py or hermes_cli/runtime_provider.py), check custom_providers before falling through to the built-in CURATED_MODELS dict. If a custom_provider has a matching model (either via API probe or explicit config), that provider's base_url and api_key should be used.
Environment
- OS: WSL2 (Ubuntu) on Windows 11
- Hermes version: 0.18.0
- Python: 3.11
- Config: 4 providers (deepseek, google, lmuai custom, qwen custom)
Related
Bug Description
When a model name exists in both
custom_providersand Hermes' built-inCURATED_MODELSregistry, the built-in mapping silently wins. This causes/model <name>to route to the wrong provider, wrong base_url, and wrong API key — 401 auth failure, then fallback to an unintended model.Root Cause
hermes_cli/models.pyhardcodesgpt-5.5underopenai-api:But the user has a
custom_providersentry:When the user runs
/model gpt-5.5, Hermes:gpt-5.5in CURATED_MODELS → matchesopenai-apimodel.provider = openai-apimodel.base_url(e.g.api.deepseek.com/v1from previous provider)gpt-5.5to DeepSeek with DeepSeek's API key → 401fallback_providersentry (gemini)Evidence from agent.log
Workaround
The user must explicitly specify the custom provider:
/model lmuai/gpt-5.5. Without the prefix, the built-in registry wins every time.Expected Behavior
When resolving a model name,
custom_providersentries should be consulted BEFORE or at equal priority to the built-in CURATED_MODELS. If a model name appears in both, the custom_provider's base_url/api_key should take precedence — the user explicitly configured it.Fix Suggestion
In the model resolution path (likely
hermes_cli/models.pyorhermes_cli/runtime_provider.py), checkcustom_providersbefore falling through to the built-inCURATED_MODELSdict. If a custom_provider has a matching model (either via API probe or explicit config), that provider'sbase_urlandapi_keyshould be used.Environment
Related