Skip to content

[Bug]: custom_providers model names overridden by hardcoded CURATED_MODELS, causing wrong base_url on /model switch #62240

Description

@jason-lsx

Bug Description

When a model name exists in both custom_providers and Hermes' built-in CURATED_MODELS registry, the built-in mapping silently wins. This causes /model <name> to route to the wrong provider, wrong base_url, and wrong API key — 401 auth failure, then fallback to an unintended model.

Root Cause

hermes_cli/models.py hardcodes gpt-5.5 under openai-api:

# models.py line 247-255
"openai-api": [
    "gpt-5.6-sol",
    ...
    "gpt-5.5",       # ← hardcoded
    "gpt-5.5-pro",    # ← hardcoded
    ...
],

But the user has a custom_providers entry:

custom_providers:
  - name: lmuai
    base_url: https://api.lmuai.com/v1
    api_key: sk-xxx

When the user runs /model gpt-5.5, Hermes:

  1. Looks up gpt-5.5 in CURATED_MODELS → matches openai-api
  2. Sets model.provider = openai-api
  3. Does NOT check custom_providers for a matching model
  4. Uses stale/wrong model.base_url (e.g. api.deepseek.com/v1 from previous provider)
  5. Sends gpt-5.5 to DeepSeek with DeepSeek's API key → 401
  6. Falls back to first fallback_providers entry (gemini)

Evidence from agent.log

01:03:45 - API call failed:
  provider=openai-api
  base_url=https://api.deepseek.com/v1    ← wrong!
  model=gpt-5.5
  HTTP 401: Authentication Fails

01:03:45 - Fallback activated: gpt-5.5 → gemini-2.5-pro (google)

Workaround

The user must explicitly specify the custom provider: /model lmuai/gpt-5.5. Without the prefix, the built-in registry wins every time.

Expected Behavior

When resolving a model name, custom_providers entries should be consulted BEFORE or at equal priority to the built-in CURATED_MODELS. If a model name appears in both, the custom_provider's base_url/api_key should take precedence — the user explicitly configured it.

Fix Suggestion

In the model resolution path (likely hermes_cli/models.py or hermes_cli/runtime_provider.py), check custom_providers before falling through to the built-in CURATED_MODELS dict. If a custom_provider has a matching model (either via API probe or explicit config), that provider's base_url and api_key should be used.

Environment

  • OS: WSL2 (Ubuntu) on Windows 11
  • Hermes version: 0.18.0
  • Python: 3.11
  • Config: 4 providers (deepseek, google, lmuai custom, qwen custom)

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    P2Medium — degraded but workaround existsarea/configConfig system, migrations, profilescomp/cliCLI entry point, hermes_cli/, setup wizardtype/bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions