Skip to content

[Feature]: Add OrcaRouter as a built-in provider #27965

Description

@zhenjunchen-png

Problem or Use Case

I'd like to use Hermes Agent with OrcaRouter (https://www.orcarouter.ai) — an
OpenAI-compatible meta-router that exposes 150+ upstream models behind a single
key and an adaptive routing strategy (LinUCB contextual bandit on per-request
features such as prompt length and code/math density). Today the only way to
reach OrcaRouter from Hermes is the generic "custom OpenAI-compatible endpoint"
path, which means users have to remember the base URL, miss the curated model
picker, and don't get the per-vendor reasoning protocol that OrcaRouter expects
(Anthropic's thinking block vs. OpenAI's top-level reasoning_effort, etc.).

Disclosure: I'm an engineer on the OrcaRouter team.

Proposed Solution

Add OrcaRouter alongside OpenRouter and NovitaAI as an API-key provider, using
the fast-path plugin layout under plugins/model-providers/orcarouter/:

  • Plugin profile (OrcaRouterProfile subclass of ProviderProfile) with
    base_url=https://api.orcarouter.ai/v1, env vars
    ORCAROUTER_API_KEY / ORCAROUTER_BASE_URL, alias orca.
  • fetch_models() against /v1/models (Bearer auth) with a module-level cache,
    mirroring the OpenRouter plugin's caching pattern.
  • build_api_kwargs_extras() that dispatches reasoning config to the correct
    upstream-native field:
    • anthropic/* → top-level thinking={"type":"enabled","budget_tokens": N}
      (budget capped strictly below max_tokens)
    • deepseek/*reasoner* / deepseek/*r1* → no reasoning field (these models
      reason by default and reject reasoning_effort)
    • everything else (OpenAI, Gemini, Grok, Qwen, Kimi …) → top-level
      reasoning_effort: "minimal|low|medium|high"
  • Default HTTP-Referer / X-Title attribution headers so OrcaRouter's console
    can see Hermes traffic.
  • Curated fallback_models for when the live /v1/models fetch fails: starts
    with orcarouter/auto (the adaptive router) plus a flagship from each major
    upstream family.

Because of the auto-extension blocks in hermes_cli/auth.py:459-491 and
hermes_cli/models.py:962-979, no edits to auth.py, models.py,
runtime_provider.py, or main.py are required — the plugin alone wires the
provider into PROVIDER_REGISTRY, CANONICAL_PROVIDERS, the model picker,
hermes model, hermes setup, and the --provider CLI flag.

Docs touched:

  • README.md — add OrcaRouter to the provider sentence next to OpenRouter
  • website/docs/integrations/providers.md — new row in the setup table
  • website/docs/reference/environment-variables.md — add ORCAROUTER_API_KEY
    and ORCAROUTER_BASE_URL
  • .env.example — new commented block

Alternatives Considered

  1. Custom endpoint (OPENAI_BASE_URL=https://api.orcarouter.ai/v1) — works
    today, but no model picker, no per-vendor reasoning dispatch, no attribution
    headers, and the user has to know the base URL.
  2. A user plugin in ~/.hermes/plugins/model-providers/ — also works
    today, but every user has to install it themselves; it's the same code as
    bundling it.
  3. No reasoning dispatch (rely on Hermes' default OpenRouter-style nested
    reasoning block) — would break for anthropic/* and deepseek/*reasoner
    models on OrcaRouter, because OrcaRouter forwards the per-vendor native
    field shape, not the OpenRouter wrapper.

Feature Type

Configuration option (new built-in provider)

Scope

Small (single new plugin directory ≈ 130 lines, plus 4 small docs edits and a
focused 34-test file)

Contribution

  • I'd like to implement this myself and submit a PR

A branch is ready locally with the implementation, unit + behavior tests
(34 / 34 passing, lint clean, Windows-footgun clean), and the docs touches
listed above. I'll open the PR once this issue gets a thumbs-up so the
discussion isn't duplicated.

Metadata

Metadata

Assignees

No one assigned

    Labels

    P3Low — cosmetic, nice to havecomp/pluginsPlugin system and bundled pluginssweeper:not-plannedSweeper: closed per standing maintainer policy (design direction)type/featureNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions