feat(ai): add Claude Code Max/Pro subscription pass-through to LiteLLM - #1467
Merged
Conversation
Registers `claude-code-subscription`, the one model CR in this repo for which the proxy holds no credential. A `claude` CLI logged in to a personal Claude Max/Pro subscription sends its own OAuth token per request; LiteLLM forwards it to Anthropic so tokens bill that person's flat-rate plan, and the cluster gains per-request token/latency/virtual-key attribution for traffic that previously bypassed the proxy entirely. Deliberately WITHOUT `general_settings.forward_client_headers_to_llm_api`. The upstream tutorial calls that flag required; on the pinned v1.98.0 image it is both unnecessary and unsafely scoped: - It never forwards `Authorization`. Its only effect is `_get_forwardable_headers`, an allow-list of `x-*` (minus `x-stainless*`) plus `anthropic-beta`. - The OAuth token travels a separate, ungated path: `add_provider_specific_headers_to_request` is called unconditionally at litellm_pre_call_utils.py:1704, so the feature works with the flag off. - The flag is genuinely proxy-global, so setting it would forward every client `x-*` header to every backend (xai, zai, openrouter, local llama.cpp) for no benefit here. Live measurements on the running proxy, with the flag absent from the rendered config, i.e. describing production as it already is: - Virtual-key governance is not bypassable: the `demo` key is denied `claude-sonnet-5` with 403 identically with and without a client `Authorization` header, and its allowed model still returns 200 identically either way. - Pre-existing finding: a client `sk-ant-oat` token DOES override the shared deployment key on Anthropic-family models the caller is already entitled to. A fake token sent to `claude-sonnet-5` made Anthropic answer "OAuth access token is invalid.", proving our ANTHROPIC_API_KEY was never sent. Bounded to that prefix, to anthropic/bedrock/vertex_ai, and to callers authenticating via `x-litellm-api-key`; it cannot bypass entitlement. It also briefly cools down that model group for all consumers. Two footguns closed, both verified in-pod rather than assumed: - The CR carries a non-secret placeholder `apiKey`. Omitting it does not make the model credential-less: `get_api_key` falls back to `os.environ/ANTHROPIC_API_KEY`, which the pod holds, so a caller who forgot the OAuth header would silently bill the household's metered account. The placeholder forces a clean 401 and zero spend instead. - Explicit $0 `info.extra` prices, a deliberate exception to this directory's no-hand-written-prices rule and the same rule underneath: prices must match the invoice, and this invoice is flat-rate, so the metered cost map is what would lie. Zero is honoured rather than read as unset (`use_custom_pricing_for_model` tests `is not None`). Consequence: maxBudget cannot constrain this model, so grant it only with rpmLimit/tpmLimit. Additive: no existing model, virtual key, allow-list, fallback chain or generalSettings value is modified, and the new model is deliberately absent from every fallback chain. Mechanism, full evidence tables and the client runbook (ANTHROPIC_BASE_URL / ANTHROPIC_MODEL / ANTHROPIC_CUSTOM_HEADERS, getting a virtual key, and the one-time interactive OAuth login): docs/ai-system/litellm/claude-code-subscription.md
…in AGENTS.md Adds item (9) to the litellm entry: a client-supplied sk-ant-oat Authorization header replaces the deployment apiKey on Anthropic-family models unconditionally, forward_client_headers_to_llm_api does not gate it and is deliberately unset, and the two reusable traps that finding exposed (a keyless model silently falling back to os.environ/ANTHROPIC_API_KEY, and explicit $0 pricing being honoured but making maxBudget inert).
Captain decision 2026-08-27, the entitlement half of the pass-through model. Registration is not entitlement, so without this the model was callable by name but reachable by nobody. Scoped to `claude-code-subscription` and nothing else: it names neither `auto` nor the metered `claude-sonnet-5`/`claude-opus-5`, so it cannot become a second door into budgeted cloud models. The only key in app/virtualkeys/ with NO maxBudget, deliberately. Its one model is priced at an explicit $0 because a flat-rate subscription must not be priced off LiteLLM's metered cost map, so recorded spend is always $0.00 and any budget would be silently inert - a cap that can never trip reads as protection that does not exist. rpmLimit/tpmLimit are therefore the entire guardrail and neither should be dropped. Sized as one interactive CLI user at the same order of magnitude as the opencode workspace key (8 rpm / 200000 tpm), nudged to 10 / 250000 because Claude Code's agentic loop spends requests per tool call rather than per human turn. These are a runaway guardrail, not a throughput target: the real ceiling is Anthropic's own rate limiting against the caller's personal subscription, which this proxy neither sees nor can raise. Pre-flight checks against the live cluster, both clear: - Server-side dry-run accepted by the CRD and the operator's failurePolicy Fail webhook, for the key and its PushSecret. - No `claude-code-subscription` alias exists in the proxy's Postgres, so the operator will mint cleanly rather than hitting the globally-unique-alias trap that stalled the O1 cutover. flux-local: 227 passed.
Contributor
--- kubernetes/apps/base/ai/litellm/app Kustomization: ai/litellm LiteLLMModel: ai/claude-code-subscription
+++ kubernetes/apps/base/ai/litellm/app Kustomization: ai/litellm LiteLLMModel: ai/claude-code-subscription
@@ -0,0 +1,21 @@
+---
+apiVersion: litellm.home-operations.com/v1alpha1
+kind: LiteLLMModel
+metadata:
+ labels:
+ app.kubernetes.io/name: litellm
+ kustomize.toolkit.fluxcd.io/name: litellm
+ kustomize.toolkit.fluxcd.io/namespace: ai
+ name: claude-code-subscription
+ namespace: ai
+spec:
+ info:
+ extra:
+ input_cost_per_token: 0
+ output_cost_per_token: 0
+ modelName: claude-code-subscription
+ params:
+ apiKey: sk-ant-oat-PLACEHOLDER-CLIENT-SENDS-ITS-OWN-TOKEN
+ model: anthropic/claude-sonnet-5
+ proxyRef: litellm
+
--- kubernetes/apps/base/ai/litellm/app Kustomization: ai/litellm LiteLLMVirtualKey: ai/claude-code-subscription
+++ kubernetes/apps/base/ai/litellm/app Kustomization: ai/litellm LiteLLMVirtualKey: ai/claude-code-subscription
@@ -0,0 +1,20 @@
+---
+apiVersion: litellm.home-operations.com/v1alpha1
+kind: LiteLLMVirtualKey
+metadata:
+ labels:
+ app.kubernetes.io/name: litellm
+ kustomize.toolkit.fluxcd.io/name: litellm
+ kustomize.toolkit.fluxcd.io/namespace: ai
+ name: claude-code-subscription
+ namespace: ai
+spec:
+ keyAlias: claude-code-subscription
+ models:
+ - claude-code-subscription
+ proxyRef: litellm
+ rpmLimit: 10
+ secretKey: key
+ secretName: litellm-key-claude-code-subscription
+ tpmLimit: 250000
+
--- kubernetes/apps/base/ai/litellm/app Kustomization: ai/litellm PushSecret: ai/litellm-key-claude-code-subscription
+++ kubernetes/apps/base/ai/litellm/app Kustomization: ai/litellm PushSecret: ai/litellm-key-claude-code-subscription
@@ -0,0 +1,25 @@
+---
+apiVersion: external-secrets.io/v1alpha1
+kind: PushSecret
+metadata:
+ labels:
+ app.kubernetes.io/name: litellm
+ kustomize.toolkit.fluxcd.io/name: litellm
+ kustomize.toolkit.fluxcd.io/namespace: ai
+ name: litellm-key-claude-code-subscription
+ namespace: ai
+spec:
+ data:
+ - match:
+ remoteRef:
+ property: key
+ remoteKey: litellm-consumer-claude-code-subscription
+ secretKey: key
+ refreshInterval: 5m
+ secretStoreRefs:
+ - kind: ClusterSecretStore
+ name: onepassword
+ selector:
+ secret:
+ name: litellm-key-claude-code-subscription
+
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Intent
Wire LiteLLM (kubernetes/apps/base/ai/litellm/, operator-shaped: LiteLLMProxy + LiteLLMModel + LiteLLMVirtualKey CRs from the home-operations litellm-operator) to support the Claude Code Max/Pro SUBSCRIPTION pass-through pattern from https://docs.litellm.ai/docs/tutorials/claude_code_max_subscription. Captain request 2026-08-27: 'use the claude code subscription claude-code cli and get more insight.'
MECHANISM, which is unlike every other model in this repo: the client is a
claudeCLI on a person's workstation, logged in via Claude Code's own subscription OAuth. It sends its OWN Authorization: Bearer sk-ant-oat... header on every request. LiteLLM holds NO shared credential for this model and forwards the client's token upstream; tokens bill that person's flat-rate plan. The cluster's payoff is observability (per-request tokens/latency, attributable per virtual key) for traffic that previously bypassed the proxy. There is deliberately NO 1Password item and NO ExternalSecret change for this model - that absence is correct, not an omission.DELIBERATE DECISION 1 (captain-approved, supersedes the original instruction): do NOT set general_settings.forward_client_headers_to_llm_api. The task originally said to add it because the upstream tutorial calls it 'Required: forwards OAuth token to Anthropic'. Research against our pinned image ghcr.io/berriai/litellm-non_root:v1.98.0 found that claim wrong on both halves: (a) it never forwards Authorization - its only effect is _get_forwardable_headers, an allow-list of x-* (minus x-stainless*) plus anthropic-beta; (b) the OAuth token instead rides a separate UNGATED path, add_provider_specific_headers_to_request, called unconditionally at litellm_pre_call_utils.py:1704, so pass-through works with the flag off; and (c) the flag is genuinely proxy-global, so setting it would forward every client x-* header to every backend (xai, zai, openrouter, local llama.cpp) for zero benefit. The task's own escape clause required stopping rather than shipping an unsafely-scoped flag; the captain reviewed the evidence and approved shipping without it. A narrower per-model form (litellm_settings.model_group_settings.forward_client_headers_to_llm_api, a list of model names) exists and is documented as the option if it is ever needed. Do not 'fix' this by adding the flag.
DELIBERATE DECISION 2 (captain-approved): the new model ships WITH its own LiteLLMVirtualKey, scoped to that one model, carrying rpmLimit/tpmLimit only and deliberately NO maxBudget - unique among the keys in app/virtualkeys/. A budget there would be silently inert because the model is priced at $0, and a cap that can never trip reads as protection that does not exist. Sized as one interactive CLI user, same order of magnitude as the existing opencode workspace key (8 rpm / 200000 tpm), set to 10 / 250000 because Claude Code's agentic loop spends requests per tool call rather than per human turn.
DELIBERATE DECISION 3: explicit $0 info.extra prices on the model, a documented exception to that directory's standing 'no hand-written prices on cloud models' rule - and the same rule underneath, since the rule is 'declared prices must match the invoice' and this invoice is a flat-rate subscription with no per-token line, so LiteLLM's metered cost map is what would lie and would inflate a D4 budget with dollars nobody is charged. Verified in the running pod that explicit 0 is honoured rather than read as unset (use_custom_pricing_for_model tests
is not None, not truthiness) and that omitting the keys is what falls back to the metered map.DELIBERATE DECISION 4: the model CR carries a non-secret placeholder apiKey string 'sk-ant-oat-PLACEHOLDER-CLIENT-SENDS-ITS-OWN-TOKEN'. This is NOT a leaked credential and authenticates nothing. It is load-bearing: omitting apiKey does NOT make the model credential-less, because AnthropicModelInfo.get_api_key is
api_key or get_secret_str('ANTHROPIC_API_KEY')and the proxy pod holds ANTHROPIC_API_KEY via envFrom - verified live in-pod - so a caller who forgot the OAuth header would SILENTLY bill the household's metered account. The placeholder keeps api_key non-None (blocking that fallback) while carrying the sk-ant-oat prefix so the request takes the OAuth branch and fails as a clean Anthropic 401 'OAuth access token is invalid.' with zero spend. A real client token overrides it.MODEL ID: resolved against Anthropic's real current direct catalog rather than copying the tutorial's stale examples (it shows claude-sonnet-4-20250514 / claude-3-5-*). Uses anthropic/claude-sonnet-5, the same dash-form id family as the working sibling claude-sonnet-5 CR.
SECURITY REVIEW, done before shipping and verified live against the running proxy with the flag absent from the rendered config (so these describe production as it already is): virtual-key governance is NOT bypassable by a client header - the demo key is denied claude-sonnet-5 with 403 identically with and without a fake Authorization header, and its allowed model returns 200 identically either way, on both /v1/chat/completions and /v1/messages. Separately confirmed a PRE-EXISTING behaviour that this PR neither introduces nor changes: a client sk-ant-oat token already overrides the shared ANTHROPIC_API_KEY on Anthropic-family models a caller is entitled to (Anthropic answered 'OAuth access token is invalid.' for a fake token sent to claude-sonnet-5, proving our real key was never sent), and it briefly cooled that model group down for all consumers. It is bounded (that prefix only, anthropic/bedrock/vertex_ai only, only when authenticating via x-litellm-api-key) and cannot bypass allow-lists. That finding is reported to the captain OUTSIDE this task and is explicitly NOT something this PR should fix - do not expand scope to address it.
SCOPE: strictly additive. One LiteLLMModel CR, one LiteLLMVirtualKey (+ its PushSecret, matching the sibling pattern), a new runbook doc, README updates and one AGENTS.md knowledge entry. No existing model CR, virtual key, allow-list, fallback chain, ExternalSecret or generalSettings value is modified. The new model is deliberately absent from every fallback chain, because a config-declared fallback bypasses the calling key's allow-list and pointing one at a model that requires a client-supplied token would fail every caller who does not send one.
Documentation deliverable includes the client-side runbook: ANTHROPIC_BASE_URL, ANTHROPIC_MODEL, ANTHROPIC_CUSTOM_HEADERS carrying the virtual key as x-litellm-api-key (never as Authorization, which is reserved for the subscription token), how to get the virtual key, and the one-time interactive OAuth login - which is stated plainly as a manual per-person step whose headless verification is out of scope, same as any other manual credential step in this repo.
Local validation already run: flux-local 'task flux:test:all' 227 passed; kustomize builds clean; server-side dry-run of both new CRs accepted by the live CRD and the operator's failurePolicy:Fail webhook; no key-alias collision in the proxy's Postgres; pre-commit hooks including the secret detector pass.
What Changed
claude-code-subscriptionLiteLLMModel(Anthropicclaude-sonnet-5via client OAuth pass-through, placeholderapiKey, explicit$0pricing, no cluster credential) and a matching single-modelLiteLLMVirtualKey+PushSecretwith rpm/tpm limits only.forward_client_headers_to_llm_api, pricing/budget exceptions, security findings, and client runbook indocs/ai-system/litellm/claude-code-subscription.md, with README andAGENTS.mdcross-links.scripts/ci/litellm-claude-code-subscription-test.pyto lock the CR shape and documented decisions.Risk Assessment
✅ Low: Strictly additive LiteLLM model and virtual-key wiring that matches every captain-approved constraint (placeholder apiKey, $0 prices, rpm/tpm-only key, no header-forward flag, no fallback membership) and follows existing sibling CR patterns without touching live shared config.
Testing
Added and ran a focused operator-render + kustomize + runbook contract test, re-ran the existing fallback-chain suite for regression, and executed the same subscription test inside the pinned litellm-non_root:v1.98.0 image to prove OAuth placeholder routing, ANTHROPIC_API_KEY fallback blocking, and explicit $0 custom pricing (metered sonnet 0.007 vs zeroed 0.0). No live cluster or interactive Claude OAuth session was available; those remain out of scope per the intent. All targeted checks passed.
Evidence: Operator file-mode render of subscription model + virtual key (no forward flag, not in fallbacks)
subscription_model: anthropic/claude-sonnet-5 + sk-ant-oat-PLACEHOLDER + $0 prices; forward_client_headers_to_llm_api_set: false; fallbacks only chat-ha/auto→claude-sonnet-5; virtualkey rpm=10 tpm=250000 no maxBudgetEvidence: Host subscription contract test transcript (40 passed)
Evidence: LiteLLM v1.98.0 runtime proofs transcript (47 passed)
use_custom_pricing zero: top/metadata/messages=True; completion_cost metered=0.007 zeroed=0.0; is_anthropic_oauth_key(placeholder)=True; get_api_key(None) falls back to env; get_api_key(placeholder) does notEvidence: Runtime proof machine-readable results JSON
Evidence: Fallback-chain regression transcript (55 passed)
Pipeline
Updates from git push no-mistakes
✅ **intent** - passed
✅ No issues found.
✅ **Rebase** - passed
✅ No issues found.
✅ **Review** - passed
✅ No issues found.
✅ **Test** - passed
✅ No issues found.
python3 scripts/ci/litellm-claude-code-subscription-test.py(host: operator render, kustomize consumer, virtual key, ExternalSecret absence, runbook contract; 40/40)podman run --entrypoint python3 ghcr.io/berriai/litellm-non_root:v1.98.0 scripts/ci/litellm-claude-code-subscription-test.pywithREQUIRE_LITELLM_RUNTIME=1(OAuth prefix, get_api_key env fallback trap, explicit $0 custom pricing vs metered sonnet cost 0.007→0.0, Router model_info zeros; 47/47)python3 scripts/ci/litellm-fallback-chain-test.py(regression: fallback chains still only chat-ha/auto→claude-sonnet-5; 55/55)kubectl kustomize kubernetes/apps/base/ai/litellm/app(emits LiteLLMModel/VirtualKey/PushSecret for claude-code-subscription)✅ **Document** - passed
✅ No issues found.
✅ **Lint** - passed
✅ No issues found.
✅ **Push** - passed
✅ No issues found.