Skip to content

[feature] components-v2: allow the same upstream model across multiple targets/endpoints (duplicate registration hard-fails the gateway) #51

Description

@shakhal

Problem

In a components-v2 profile config, two targets that point at the same upstream model (e.g. the same model served through two endpoints with different API keys) cause serve to abort at load with SwitchyardDuplicateRegistrationError, even when the targets and profiles have distinct ids. This makes it impossible to serve one upstream model via two credentials in a single gateway — the only workaround today is running a separate gateway process per key.
?

Reproduction

profiles.json — two endpoints (same upstream, different keys), two distinctly-named targets and profiles, both pointing at the same model:

{
  "endpoints": {
    "team_a": { "base_url": "https://api.openai.com/v1", "api_key": "sk-team-a" },
    "team_b": { "base_url": "https://api.openai.com/v1", "api_key": "sk-team-b" }
  },
  "targets": {
    "gpt-5.4__a": { "endpoint": "team_a", "model": "gpt-5.4", "format": "responses" },
    "gpt-5.4__b": { "endpoint": "team_b", "model": "gpt-5.4", "format": "responses" }
  },
  "profiles": {
    "team_a__gpt-5.4": { "type": "passthrough", "target": "gpt-5.4__a" },
    "team_b__gpt-5.4": { "type": "passthrough", "target": "gpt-5.4__b" }
  }
}
$ switchyard serve --config profiles.json --port 4000
INFO - Switchyard components-v2 profile config loaded from profiles.json
Traceback (most recent call last):
  ...
  File ".../switchyard_rust/server.py", line 17, in run_profile_server
    _load_native().run_profile_server(config_path, host, port, backlog, dry_run)
_switchyard_rust.SwitchyardDuplicateRegistrationError: model "gpt-5.4" is already registered

Proposed solution

When multiple targets declare the same upstream model, don't abort. The namespaced profile/target ids already disambiguate for routing — only the bare-model alias is ambiguous. Any of these would unblock it:

  1. Skip the bare-model alias when it's ambiguous (register only the unique profile/target ids). Simplest; selection by profile id keeps working.
  2. Register the bare alias once to a designated default (first target, or an explicit "default": true on a target).
  3. Gate the bare-model alias behind an opt-in flag (per-target or global), defaulting to off.

Option 1 seems least surprising.

Scope notes

  • Is this a new role/component in the chain (RequestProcessor / LLMBackend / ResponseProcessor / ResponseTranslator), or an extension of an existing one? No
  • Does it touch a public API listed in switchyard/__init__.py.__all__? No
  • Backward-compatibility concerns? No

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Fields

    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions