Built-in CLIProxyAPI: pick where the proxy runs, switch providers to it, and have Limits and recovery just work

[saphid]

The problem

Many users already run CLIProxyAPI next to T3 Code. The FAQ points people to it for account pools. The current integration is read only: a usage hub under Usage → Limits. Routing a provider through the proxy is a manual recipe: environment variables on each instance, on each machine, with the proxy installed and logged in by hand.

That gap causes a steady stream of reports. These are the open issues, PRs, and discussions I found:

#	Symptom
#11873	Codex limits show -32600 warnings for proxied instances
#23054	A hub-routed Codex instance gets one Limits tile per environment
#26574, #23316	Hub reset times and cooldowns never reach thread recovery or snooze
#23057, #23435	Claude monthly spend disappears when a hub is configured
#24680, #24687	Plugin-backed hub accounts are missing
#22421	Claude cooldown deadlines from the hub are ignored
#21802	Grok limits through the hub
#25579, #25385, #21887, #18895, #22218	Hub URL, account visibility, cross-environment hubs, identity

Most of these share one cause. T3 does not know that an instance is routed through a proxy, so it probes the provider directly, cannot match the proxy's accounts to the instance, and cannot use the proxy's reset times.

Proposal in one paragraph

Add one setting, Proxy host: Off, one of your T3 environments, or an external machine. Each provider instance gets one switch, Use proxy. Everything else lives in the existing provider view: the proxy's accounts appear under the provider they belong to, with add, disable, and remove. T3 then knows which instances are proxied, so Limits, recovery, model lists, and status work the same as for a local login. Apart from choosing a host and flipping the switch, the proxy is invisible.

Direction check first

docs/user/faq.md says T3 Code is not the place to manage a pile of accounts, because an auto-switcher risks TOS bans. This proposal does not add account switching to T3. CLIProxyAPI still owns credentials and routing, and T3 only hosts it, points providers at it, and reads it. But it does make a pool one click away instead of an afternoon's setup, and that is the maintainers' call. I'd like a yes or no on direction before any code. If the answer is no, phases 1 and 4 alone still fix the bug list above for people who run a proxy themselves.

Design

1. Where the proxy runs

proxyHost is one of:

• Off. Today's behaviour.
• A T3 environment. That server downloads a pinned CLIProxyAPI release and supervises it. It reuses the cloudflared pattern: per-platform version and SHA-256 pin, consent before download, never replacing a user's own binary, restart backoff (packages/shared/src/relayClient.ts, apps/server/src/cloud/ManagedEndpointRuntime.ts). It binds loopback only, and T3 writes its config.yaml and auth directory under T3 home.
• External machine. You run stock CLIProxyAPI anywhere. It is one Go binary of about 25 MB, also available through Docker, Homebrew, and an installer script. T3 needs nothing installed there. The dialog gives a copyable install command and a config snippet with generated keys, and you paste back the URL and management key. This covers a small box dedicated to the job, without a T3 server on it.

Reachability for a T3 host. T3 Connect deliberately exposes only T3's own loopback origin (docs/internals/t3-connect.md, "Managed tunnels expose only a validated loopback HTTP origin"). So the proxy is reached through T3's HTTP server at an allowlisted /proxy/* route, the same approach as DeviceHubProxy.ts.

• The route allows /v1/messages, /v1/responses (HTTP and WebSocket), /v1/chat/completions, /v1/models, and /backend-api/codex/*. It does not allow /v0/management/*.
• Requests are authenticated with the proxy's client key, not a T3 session.
• One URL shape then works over LAN, tailnet, and Connect.
• Consumers on the host machine call loopback directly, with no extra hop.

2. Keys and which environment holds what

Following the #21887 review concern, the management key, which controls every pooled account, lives on one environment only: the T3 host, or the environment you pick to manage an external proxy. Every other environment gets a per-environment client key:

• The host mints the key when you enable the proxy for that environment.
• The client carries it once into that environment's secret store, like pairing.
• Turning the environment off revokes its key on the host.

Settings stay per environment. Enabling the proxy on several environments is an explicit bulk edit with per-destination results and Retry failed saves, as in the model prices dialog (apps/web/src/components/usage/usagePriceTargets.ts), and it follows the "All environments" rule in docs/internals/overview.md.

Each consuming environment stores an ordered list of candidate proxy URLs, taken from the host's learned routes. The server probes them in order and uses the first that answers, the same way client routes work in docs/internals/remote.md.

3. Use proxy on a provider instance

The instance config gets proxy: boolean. Routing is applied by the driver at provider spawn only, not through instance.environment. Today the instance environment also feeds integrated terminals (apps/server/src/terminal/Manager.ts) and usage scanning, and proxy credentials should not leak into a user's shell.

Driver	How it routes	v1
Claude	ANTHROPIC_BASE_URL, ANTHROPIC_AUTH_TOKEN, empty ANTHROPIC_API_KEY. Status already labels this "Custom API endpoint" (ClaudeProvider.ts).	Yes
Codex	Keep the built-in openai provider ID, set its base URL to the proxy, and pass the client key as the API key. Codex stores the provider ID in rollouts and needs it unchanged to resume (CodexManagedRuntime.ts), so a new model_provider would break resume whenever someone toggles the switch. Needs a spike.	Yes
OpenCode	Generated OPENCODE_CONFIG_CONTENT with Anthropic- and OpenAI-compatible providers pointing at the proxy	Yes
Grok	GROK_CLI_CHAT_PROXY_BASE_URL plus the client key. CLIProxyAPI has xAI OAuth.	Phase 4
Pi	Pi's own endpoint and key variables	Phase 4, if Pi's custom-endpoint config allows it
Cursor	CLIProxyAPI has no Cursor upstream	Not supported; switch hidden
Antigravity	The driver strips Google auth variables and re-adds them from its own fields; CLIProxyAPI has Antigravity OAuth	Not in v1
Muse, fx, ACP registry	Backend is chosen outside T3	Not supported; switch hidden

When an instance is proxied:

• Status shows "via proxy on " instead of a login state. The native login check and native limits probe are skipped. This fixes [Bug]: Usage Limits shows Codex -32600 warnings for custom proxy providers without OpenAI auth #11873.
• Models come from the proxy's /v1/models, filtered to that driver's channel, plus customModels.
• If the proxy is unreachable, the instance is unavailable with that reason. It never falls back silently to a direct login, because a silent fallback spends a different account than the user chose.

4. Accounts in the provider view

On the managing environment, a ProxyAccounts service wraps CLIProxyAPI's management API. As a service, it serves the UI and MCP tools alike, and scheduled tasks can reach it later.

• List. Accounts appear under the provider they belong to: Claude accounts under Claude, Codex under Codex. Each shows email, plan, status, cooldown deadline, and disabled.
• Disable/enable and remove, each with its reverse.
• Add account. It uses .../oauth/auth-url?provider= and then polls the status endpoint.
  • Device flows (Codex device login, xAI, Kimi) work from any client, including mobile.
  • Redirect flows (Claude, Codex browser, Antigravity) redirect to localhost:<port> on the browser's machine. On desktop, the Electron shell briefly listens on that port and forwards the callback to the server. On web and mobile, you paste back the URL your browser lands on. CLIProxyAPI's own headless login already works this way.

5. Usage and limits

• The proxy becomes the usage source automatically. The manual Add hub dialog stays for read-only hubs you don't route through. Existing usageLimitSources entries keep working and need no migration.
• A proxied instance reports routedVia: <proxy>. Limits then shows the pool for that instance once, not one unknown tile per environment. This fixes [Bug]: Usage → Limits shows one Codex tile per environment for a hub-routed Codex instance, though rateLimits/read already returns the matching ChatGPT account id #23054.
• Recovery. When a proxied turn ends with usage_limit and no reset time, the recovery worker reads the earliest next_retry_after for that provider and model from the proxy. This fixes [Bug]: CLIProxy reset times are available in Usage but missing from thread recovery; queued messages block snoozing #26574 and [Bug]: CLIProxyAPI account-pool cooldowns never reach usage-limit auto-resume #23316, and covers fix(server): recover Claude proxy cooldowns from management deadlines #22421. Today Codex reset times come only from the native rate-limit snapshot (codexUsageLimits.ts), which a proxied instance never has.
• Reset credits. These keep using the hub's api-call passthrough (apps/server/src/usage/cliproxyApi.ts).

6. Defaults that limit the TOS concern

T3 writes CLIProxyAPI's routing.session-affinity: true. A thread then stays on one account instead of hopping between accounts each request, and that also keeps prompt caching working. Otherwise T3 keeps CLIProxyAPI's defaults and exposes no routing-strategy UI.

Surfaces

• Clients.
  • Web and desktop: everything.
  • Desktop adds the OAuth callback catcher.
  • Mobile: status, the account list, and device-flow login. Mobile does not edit provider instances today, so the Use proxy switch is not on mobile in v1.
• Contracts. proxyHost and per-environment proxy connection settings in ServerSettings, proxy on the instance config, and routedVia on provider status. Secrets use secretSetting.
• Agents. Account list and status as MCP tools in a later phase. No write tools without explicit approval.
• Reverse states.
  • Use proxy off: the instance returns to its native login.
  • Proxy host off: the managed process stops and client keys are revoked. Proxied instances show "proxy off" until someone turns their switch off.
  • Removing an account: removed on the proxy.
• Connection modes. LAN, tailnet, and Connect all use the /proxy/* route. External proxies use their own URL.
• Docs.
  • Replace the hub section of docs/user/usage.md and the proxy recipe in providers-claude.md.
  • Update the FAQ answer if direction changes.
  • One internals note on the key split and the /proxy/* route.

Phases (one PR each)

1. Route through an external proxy. proxyHost: external, per-environment client key, Use proxy for Claude, Codex, and OpenCode, spawn-only injection, proxied status and models, and the proxy as the automatic usage source. Fixes [Bug]: Usage Limits shows Codex -32600 warnings for custom proxy providers without OpenAI auth #11873. Includes the Codex resume spike.
2. Accounts in the provider view. ProxyAccounts service, list, disable and remove, device and paste-back login, desktop callback catcher.
3. T3-hosted proxy. Pinned download and supervision, /proxy/* route, key minting, candidate URLs, and the Proxy host dropdown with environment fan-out.
4. Limits and recovery. routedVia pooling ([Bug]: Usage → Limits shows one Codex tile per environment for a hub-routed Codex instance, though rateLimits/read already returns the matching ChatGPT account id #23054), hub reset times in recovery ([Bug]: CLIProxy reset times are available in Usage but missing from thread recovery; queued messages block snoozing #26574, [Bug]: CLIProxyAPI account-pool cooldowns never reach usage-limit auto-resume #23316, fix(server): recover Claude proxy cooldowns from management deadlines #22421), Grok and Pi routing.
5. Standalone setup helper. Copyable install command and config generator for an external machine, and docs.

Phase 1 stands alone and helps everyone who already runs a proxy.

Open questions

1. Direction. Is a T3-hosted proxy (phase 3) acceptable given the FAQ? Or should T3 only route to and read a proxy you run yourself (phases 1, 2, 4, 5)?
2. Codex resume. Does pointing the built-in openai provider at the proxy keep rollouts resumable when you toggle the switch, or do we need a stable t3_proxy provider ID and a rule that toggling applies only to new threads?
3. /proxy/* over Connect. This carries model traffic, including WebSockets, through the T3 server, and it widens what a Connect hostname serves, though only behind the proxy's own key. Acceptable, or tailnet and LAN only?
4. Where routing lives. Is a boolean on the instance right, or should a proxied provider be its own instance, for example "Claude (proxy)" next to "Claude"? A separate instance avoids resume questions, but it is the extra machinery this design tries to avoid.

I'm happy to build phase 1 once direction is settled.