[Feature Request] Native "LiteLLM-style" Key Management: Multi-Key Pooling & Automatic Failover

[ThomasOnTraining]

Concept Overview

To make OpenClaude the most resilient open-source coding agent, I suggest implementing a native Key Management & Load Balancing system inspired by LiteLLM.

The goal is to allow users to define a "Pool" of API keys for the same provider/model. If one key hits a Rate Limit (429) or a Provider Error (5xx), OpenClaude should automatically switch to the next available key without interrupting the agent's current task or losing conversation context.

Why this is a game-changer?

1. Uninterrupted Workflows: Large-scale refactoring often hits rate limits. Native failover ensures the agent keeps working while other keys are on "cooldown".
2. Zero-Configuration Proxy: Users won't need to run external Docker containers or Python proxies (like LiteLLM) just to manage their keys.
3. Cost Optimization: It allows users to distribute load across multiple "Free Tier" or "Low-Tier" accounts/keys from providers like OpenRouter or DeepSeek.

Proposed Architecture

Instead of a single string for API keys, the configuration could support an Object or Array:

{
  "provider": "openrouter",
  "model": "anthropic/claude-3.5-sonnet",
  "key_pool": {
    "strategy": "round-robin", // Options: round-robin, random, priority
    "keys": [
      "sk-or-v1-key_01...",
      "sk-or-v1-key_02...",
      "sk-or-v1-key_10..."
    ],
    "retry_on_429": true,
    "cooldown_time": 60 // Seconds to wait before retrying a failed key
  }
}

[Vasanthdev2004]

This is a good feature idea.

The maintainer tradeoff here is that native key-pooling/failover sounds great from a resilience perspective, but it also adds a lot of complexity around:

• provider-specific retry semantics
• error classification (429 vs auth failure vs model-not-found)
• cooldown bookkeeping
• deterministic routing / observability
• making sure failover does not silently hide billing or account issues

So I do think it is valuable, but I would probably frame it as a staged feature instead of one big jump:

1. support multiple credentials for a provider
2. add explicit retry/failover only for safe transient classes (429, some 5xx)
3. surface which key/profile is active so behavior is debuggable
4. only then add balancing strategies like round-robin/priority

If someone picks this up, a minimal first version with:

• ordered fallback keys
• retry-on-429 only
• visible logging/status output

would already be useful without overcomplicating the config too early.