Skip to content

feat(litellm): raise claude-code-subscription rate limits - #1503

Merged
Aviator-Coding merged 3 commits into
mainfrom
fm/homeops-litellm-claude-sub-rpm
Aug 31, 2026
Merged

feat(litellm): raise claude-code-subscription rate limits#1503
Aviator-Coding merged 3 commits into
mainfrom
fm/homeops-litellm-claude-sub-rpm

Conversation

@Aviator-Coding

Copy link
Copy Markdown
Owner

Intent

Raise the claude-code-subscription LiteLLM virtual key's rate limits in kubernetes/apps/base/ai/litellm/app/virtualkeys/claude-code-subscription.yaml. Captain reported client-side timeouts on the Claude Code subscription pass-through key; verified via LiteLLM proxy logs (litellm-99ccbcc74-dj4jp, last 24h) showing ~556 429s with 'Limit type: requests. Current limit: 10' and ~171 429s with 'Limit type: tokens. Current limit: 250000' (250000 is unique to this key's tpmLimit). Change: rpmLimit 10 -> 300 (captain-specified number). tpmLimit 250000 -> 7500000 (captain-approved after escalating concrete evidence that the token limit was also being hit; 7500000 preserves the same tokens-per-request headroom now that rpm is 300, since Claude Code requests can carry 50-200k token contexts). Rewrite the sizing comment to explain both numbers with the measured evidence (both 429 counts), replacing the stale 'one interactive CLI user, same order of magnitude as opencode.yaml' rationale which no longer applies. Preserve intact: the surrounding safety-reasoning comment explaining why this key deliberately has NO maxBudget (its model is priced at $0 so any budget would be silently inert) and that rate limits are therefore the whole guardrail - do not remove either limit, do not add a maxBudget. Do not widen the key's models list (stays scoped to claude-code-subscription only). Do not touch any other virtual key, the model catalog, fallback chains, the ai-pr-review key, or the proxy's own configuration. State plainly in the PR that this is a runaway-agent guardrail, not a throughput target: the real ceiling is Anthropic's own rate limiting against the captain's personal subscription, which this proxy neither sees nor can raise - if timeouts persist after this, the cause is upstream of the proxy and this change will not have fixed it. Also state plainly that the LiteLLMVirtualKey is server-side state reconciled by litellm-operator, so the new limits only take effect on the live key after this merges and the operator reconciles - verifying that the 429s actually stop is a post-merge check, not something confirmed in this PR.

What Changed

  • Raised the claude-code-subscription LiteLLM virtual key from rpmLimit: 10 / tpmLimit: 250000 to 300 / 7500000, keeping the model allow-list scoped to claude-code-subscription only and leaving maxBudget unset (rate limits remain the sole guardrail on the $0-priced pass-through model).
  • Rewrote the CR sizing comment and matching docs/CI expected values to record the measured proxy 429 evidence (~556 request-limit and ~171 token-limit in 24h) and the proportional tokens-per-request headroom at the new rpm.
  • These numbers are a runaway-agent guardrail, not a throughput target: the real ceiling is Anthropic’s own rate limiting on the captain’s personal subscription, which this proxy neither sees nor can raise. The LiteLLMVirtualKey is server-side state reconciled by litellm-operator, so the new limits take effect only after merge + operator reconcile; confirming the 429s stop is a post-merge check.

Risk Assessment

✅ Low: Single-file intentional rpm/tpm raise with preserved no-budget guardrail, scoped model list, and no other keys or proxy config touched.

Testing

Updated the stale CI pin for the new limits, then ran the focused subscription test plus a kustomize-emitted CR semantic proof; all checks passed showing 300 rpm / 7500000 tpm, preserved 25000 tokens-per-request headroom, no maxBudget, single-model scope, siblings untouched, and the rewritten evidence-backed sizing comment. Live 429 cessation remains intentionally post-merge after litellm-operator reconcile.

Evidence: kustomize-emitted CR rate-limit semantics + isolation
=== kustomize-emitted LiteLLMVirtualKey (consumer surface) ===
name:        claude-code-subscription
keyAlias:    claude-code-subscription
models:      ['claude-code-subscription']
rpmLimit:    300  (was 10, expected 300)
tpmLimit:    7500000  (was 250000, expected 7500000)
maxBudget:   ABSENT (deliberate)
spec keys:   ['keyAlias', 'models', 'proxyRef', 'rpmLimit', 'secretKey', 'secretName', 'tpmLimit']

=== tokens-per-request headroom ===
old: 250000/10 = 25000 tokens/request
new: 7500000/300 = 25000 tokens/request
ratio preserved: True (both 25000; covers 50-200k context via multi-request headroom math at session scale)

=== sibling virtual keys untouched (isolation) ===
  ai-pr-review                 rpm=10       tpm=200000     maxBudget=5.00     models=['pr-review-local']
  claude-code-subscription     rpm=300      tpm=7500000    maxBudget=<absent> models=['claude-code-subscription'] << CHANGED TARGET
  demo                         rpm=2        tpm=2000       maxBudget=0.05     models=['qwen3.6-35b-a3b']
  ha-demo                      rpm=10       tpm=60000      maxBudget=1.0      models=['chat-ha']
  opencode                     rpm=8        tpm=200000     maxBudget=10.00    models=['auto']
  repo-wiki                    rpm=12       tpm=200000     maxBudget=30.00    models=['chat-local']
  router-demo                  rpm=6        tpm=60000      maxBudget=0.5      models=['auto']

other holders of claude-code-subscription: []
keys without maxBudget:    ['claude-code-subscription']

=== operator-facing sizing/safety comment contract ===
  [OK] contains 'THE ONLY KEY HERE WITH NO `maxBudget`'
  [OK] contains 'Rate limits are therefore the whole guardrail'
  [OK] contains 'runaway-agent guardrail, not a'
  [OK] contains 'throughput target'
  [OK] contains "Anthropic's own rate limiting"
  [OK] contains '~556'
  [OK] contains '~171'
  [OK] contains '7500000'

=== change isolation (litellm tree vs base commit) ===
committed paths under litellm: ['kubernetes/apps/base/ai/litellm/app/virtualkeys/claude-code-subscription.yaml']
worktree status:
 M scripts/ci/litellm-claude-code-subscription-test.py


ALL SEMANTIC CHECKS PASSED

NOTE (post-merge only, per intent): LiteLLMVirtualKey is server-side state
reconciled by litellm-operator. New limits take effect on the live key only
after merge + operator reconcile. Verifying 429s stop is a post-merge check.
If timeouts persist after that, cause is upstream Anthropic rate limiting,
which this proxy neither sees nor can raise.
Evidence: litellm-claude-code-subscription-test.py full run (40/40)
APP_DIR=/Users/coder/.no-mistakes/worktrees/07f90326a876/01M1AM9G34X7ME5WMH2P964YC7/kubernetes/apps/base/ai/litellm/app
rendered model_list=35 names_has_subscription=True
[PASS] claude_code_subscription_model_present_in_operator_render — n_models=35 has=True
[PASS] upstream_model_id_is_anthropic_claude_sonnet_5_dash_form — model='anthropic/claude-sonnet-5'
[PASS] api_key_is_present_non_env_placeholder — api_key='sk-ant-oat-PLACEHOLDER-CLIENT-SENDS-ITS-OWN-TOKEN'
[PASS] api_key_carries_sk_ant_oat_prefix_for_oauth_branch — prefix_ok=True
[PASS] sibling_claude_sonnet_5_still_uses_shared_env_key — sonnet_api_key='os.environ/ANTHROPIC_API_KEY'
[PASS] model_info_declares_explicit_zero_token_prices — model_info={'input_cost_per_token': 0, 'output_cost_per_token': 0}
[PASS] zero_prices_are_numeric_zero_not_none — types=(int,int)
[PASS] general_settings_does_not_enable_forward_client_headers — general_settings.forward=None
[PASS] litellm_settings_has_no_model_group_forward_headers_list — model_group_settings.forward=None
[PASS] proxy_cr_generalSettings_omits_forward_client_headers — keys=['maximum_spend_logs_retention_period', 'store_model_in_db', 'store_prompts_in_spend_logs']
[PASS] proxy_cr_litellmSettings_omits_global_forward_flag — keys=['cache', 'cache_params', 'callbacks']
[PASS] fallback_maps_parse — avail={'chat-ha': ['claude-sonnet-5'], 'auto': ['claude-sonnet-5']} ctx={'chat-ha': ['claude-sonnet-5'], 'auto': ['claude-sonnet-5']}
[PASS] claude_code_subscription_not_a_fallback_primary — in_avail=False in_ctx=False
[PASS] claude_code_subscription_not_a_fallback_target — avail_targets={'chat-ha': ['claude-sonnet-5'], 'auto': ['claude-sonnet-5']} ctx_targets={'chat-ha': ['claude-sonnet-5'], 'auto': ['claude-sonnet-5']}
[PASS] virtualkey_cr_present — keys=['ai-pr-review', 'claude-code-subscription', 'demo', 'ha-demo', 'opencode', 'repo-wiki', 'router-demo']
[PASS] virtualkey_scoped_only_to_subscription_model — models=['claude-code-subscription']
[PASS] virtualkey_key_alias_matches_name — keyAlias='claude-code-subscription'
[PASS] virtualkey_has_no_maxBudget — spec_keys=['keyAlias', 'models', 'proxyRef', 'rpmLimit', 'secretKey', 'secretName', 'tpmLimit']
[PASS] virtualkey_rpm_tpm_match_captain_sizing — rpm=300 tpm=7500000
[PASS] only_subscription_key_lacks_maxBudget — no_budget_keys=['claude-code-subscription']
[PASS] no_other_virtualkey_holds_subscription_model — other_holders=[]
[PASS] sibling_opencode_key_still_has_maxBudget_and_lower_rpm — opencode={'maxBudget': '10.00', 'rpmLimit': 8, 'tpmLimit': 200000, 'models': ['auto']}
[PASS] pushsecret_companion_present — n=1
[PASS] pushsecret_targets_litellm_consumer_item — remote={'remoteKey': 'litellm-consumer-claude-code-subscription', 'property': 'key'} selector={'secret': {'name': 'litellm-key-claude-code-subscription'}}
[PASS] externalsecret_file_still_single_shared_secret — n=1
[PASS] externalsecret_does_not_reference_subscription_model_or_placeholder — es_name=litellm
[PASS] externalsecret_still_pulls_shared_ai_keys_item — extracts=['litellm', 'cloudnative-pg', 'ai-keys', 'litellm-sso']
[PASS] kubectl_kustomize_app_succeeds — bytes=31303
[PASS] kustomize_emits_subscription_model_key_pushsecret — missing=[]
[PASS] kustomize_emitted_model_keeps_placeholder_and_zero_prices — params={'apiKey': 'sk-ant-oat-PLACEHOLDER-CLIENT-SENDS-ITS-OWN-TOKEN', 'model': 'anthropic/claude-sonnet-5'} extra={'input_cost_per_token': 0, 'output_cost_per_token': 0}
[PASS] kustomize_emitted_key_has_limits_no_budget — spec={'keyAlias': 'claude-code-subscription', 'models': ['claude-code-subscription'], 'proxyRef': 'litellm', 'rpmLimit': 300, 'secretKey': 'key', 'secretName': 'litellm-key-claude-code-subscription', 'tpmLimit': 7500000}
[PASS] runbook_doc_exists — bytes=14262
[PASS] runbook_documents_client_env_contract — missing=[]
[PASS] runbook_model_value_is_claude_code_subscription — model assignment present
[PASS] runbook_forbids_putting_virtual_key_in_authorization — authorization guidance present
[PASS] runbook_states_oauth_login_is_manual_per_person — oauth login section present
[PASS] runbook_states_headless_oauth_out_of_scope — headless scope note present
[PASS] runbook_records_forward_client_headers_deliberately_off — flag decision recorded
[PASS] app_readme_links_subscription_pass_through_section — readme_bytes=21262
[PASS] litellm_runtime_available_for_subscription_proofs — litellm not installed here - semantic CR proofs still ran; re-run under ghcr.io/berriai/litellm-non_root:v1.98.0 for runtime proofs

=== SUMMARY ===
passed=40 failed=0 total=40
wrote /var/folders/yr/h20mxtv56yj1c1pt9tr1_kbc0000gn/T/no-mistakes-evidence/01M1AM9G34X7ME5WMH2P964YC7/claude-code-subscription-test-results.json
Evidence: structured test results JSON
{
  "results": [
    {
      "name": "claude_code_subscription_model_present_in_operator_render",
      "ok": true,
      "detail": "n_models=35 has=True"
    },
    {
      "name": "upstream_model_id_is_anthropic_claude_sonnet_5_dash_form",
      "ok": true,
      "detail": "model='anthropic/claude-sonnet-5'"
    },
    {
      "name": "api_key_is_present_non_env_placeholder",
      "ok": true,
      "detail": "api_key='sk-ant-oat-PLACEHOLDER-CLIENT-SENDS-ITS-OWN-TOKEN'"
    },
    {
      "name": "api_key_carries_sk_ant_oat_prefix_for_oauth_branch",
      "ok": true,
      "detail": "prefix_ok=True"
    },
    {
      "name": "sibling_claude_sonnet_5_still_uses_shared_env_key",
      "ok": true,
      "detail": "sonnet_api_key='os.environ/ANTHROPIC_API_KEY'"
    },
    {
      "name": "model_info_declares_explicit_zero_token_prices",
      "ok": true,
      "detail": "model_info={'input_cost_per_token': 0, 'output_cost_per_token': 0}"
    },
    {
      "name": "zero_prices_are_numeric_zero_not_none",
      "ok": true,
      "detail": "types=(int,int)"
    },
    {
      "name": "general_settings_does_not_enable_forward_client_headers",
      "ok": true,
      "detail": "general_settings.forward=None"
    },
    {
      "name": "litellm_settings_has_no_model_group_forward_headers_list",
      "ok": true,
      "detail": "model_group_settings.forward=None"
    },
    {
      "name": "proxy_cr_generalSettings_omits_forward_client_headers",
      "ok": true,
      "detail": "keys=['maximum_spend_logs_retention_period', 'store_model_in_db', 'store_prompts_in_spend_logs']"
    },
    {
      "name": "proxy_cr_litellmSettings_omits_global_forward_flag",
      "ok": true,
      "detail": "keys=['cache', 'cache_params', 'callbacks']"
    },
    {
      "name": "fallback_maps_parse",
      "ok": true,
      "detail": "avail={'chat-ha': ['claude-sonnet-5'], 'auto': ['claude-sonnet-5']} ctx={'chat-ha': ['claude-sonnet-5'], 'auto': ['claude-sonnet-5']}"
    },
    {
      "name": "claude_code_subscription_not_a_fallback_primary",
      "ok": true,
      "detail": "in_avail=False in_ctx=False"
    },
    {
      "name": "claude_code_subscription_not_a_fallback_target",
      "ok": true,
      "detail": "avail_targets={'chat-ha': ['claude-sonnet-5'], 'auto': ['claude-sonnet-5']} ctx_targets={'chat-ha': ['claude-sonnet-5'], 'auto': ['claude-sonnet-5']}"
    },
    {
      "name": "virtualkey_cr_present",
      "ok": true,
      "detail": "keys=['ai-pr-review', 'claude-code-subscription', 'demo', 'ha-demo', 'opencode', 'repo-wiki', 'router-demo']"
    },
    {
      "name": "virtualkey_scoped_only_to_subscription_model",
      "ok": true,
      "detail": "models=['claude-code-subscription']"
    },
    {
      "name": "virtualkey_key_alias_matches_name",
      "ok": true,
      "detail": "keyAlias='claude-code-subscription'"
    },
    {
      "name": "virtualkey_has_no_maxBudget",
      "ok": true,
      "detail": "spec_keys=['keyAlias', 'models', 'proxyRef', 'rpmLimit', 'secretKey', 'secretName', 'tpmLimit']"
    },
    {
      "name": "virtualkey_rpm_tpm_match_captain_sizing",
      "ok": true,
      "detail": "rpm=300 tpm=7500000"
    },
    {
      "name": "only_subscription_key_lacks_maxBudget",
      "ok": true,
      "detail": "no_budget_keys=['claude-code-subscription']"
    },
    {
      "name": "no_other_virtualkey_holds_subscription_model",
      "ok": true,
      "detail": "other_holders=[]"
    },
    {
      "name": "sibling_opencode_key_still_has_maxBudget_and_lower_rpm",
      "ok": true,
      "detail": "opencode={'maxBudget': '10.00', 'rpmLimit': 8, 'tpmLimit': 200000, 'models': ['auto']}"
    },
    {
      "name": "pushsecret_companion_present",
      "ok": true,
      "detail": "n=1"
    },
    {
      "name": "pushsecret_targets_litellm_consumer_item",
      "ok": true,
      "detail": "remote={'remoteKey': 'litellm-consumer-claude-code-subscription', 'property': 'key'} selector={'secret': {'name': 'litellm-key-claude-code-subscription'}}"
    },
    {
      "name": "externalsecret_file_still_single_shared_secret",
      "ok": true,
      "detail": "n=1"
    },
    {
      "name": "externalsecret_does_not_reference_subscription_model_or_placeholder",
      "ok": true,
      "detail": "es_name=litellm"
    },
    {
      "name": "externalsecret_still_pulls_shared_ai_keys_item",
      "ok": true,
      "detail": "extracts=['litellm', 'cloudnative-pg', 'ai-keys', 'litellm-sso']"
    },
    {
      "name": "kubectl_kustomize_app_succeeds",
      "ok": true,
      "detail": "bytes=31303"
    },
    {
      "name": "kustomize_emits_subscription_model_key_pushsecret",
      "ok": true,
      "detail": "missing=[]"
    },
    {
      "name": "kustomize_emitted_model_keeps_placeholder_and_zero_prices",
      "ok": true,
      "detail": "params={'apiKey': 'sk-ant-oat-PLACEHOLDER-CLIENT-SENDS-ITS-OWN-TOKEN', 'model': 'anthropic/claude-sonnet-5'} extra={'input_cost_per_token': 0, 'output_cost_per_token': 0}"
    },
    {
      "name": "kustomize_emitted_key_has_limits_no_budget",
      "ok": true,
      "detail": "spec={'keyAlias': 'claude-code-subscription', 'models': ['claude-code-subscription'], 'proxyRef': 'litellm', 'rpmLimit': 300, 'secretKey': 'key', 'secretName': 'litellm-key-claude-code-subscription', 'tpmLimit': 7500000}"
    },
    {
      "name": "runbook_doc_exists",
      "ok": true,
      "detail": "bytes=14262"
    },
    {
      "name": "runbook_documents_client_env_contract",
      "ok": true,
      "detail": "missing=[]"
    },
    {
      "name": "runbook_model_value_is_claude_code_subscription",
      "ok": true,
      "detail": "model assignment present"
    },
    {
      "name": "runbook_forbids_putting_virtual_key_in_authorization",
      "ok": true,
      "detail": "authorization guidance present"
    },
    {
      "name": "runbook_states_oauth_login_is_manual_per_person",
      "ok": true,
      "detail": "oauth login section present"
    },
    {
      "name": "runbook_states_headless_oauth_out_of_scope",
      "ok": true,
      "detail": "headless scope note present"
    },
    {
      "name": "runbook_records_forward_client_headers_deliberately_off",
      "ok": true,
      "detail": "flag decision recorded"
    },
    {
      "name": "app_readme_links_subscription_pass_through_section",
      "ok": true,
      "detail": "readme_bytes=21262"
    },
    {
      "name": "litellm_runtime_available_for_subscription_proofs",
      "ok": true,
      "detail": "litellm not installed here - semantic CR proofs still ran; re-run under ghcr.io/berriai/litellm-non_root:v1.98.0 for runtime proofs"
    }
  ],
  "failed": [],
  "model_names": [
    "auto",
    "chat-ha",
    "chat-local",
    "claude-code-subscription",
    "claude-fable-5",
    "claude-opus-4-8",
    "claude-opus-5",
    "claude-sonnet-5",
    "deepseek-v4-flash",
    "deepseek-v4-flash-0731",
    "deepseek-v4-pro",
    "deepseek-v4-pro-0813",
    "gemini-3.1-pro",
    "gemini-3.5-flash-lite",
    "gemini-3.6-flash",
    "gemini-3.7-flash",
    "gemma-4-31b",
    "glm-5.2",
    "glm-5.3",
    "gpt-5.6-luna",
    "gpt-5.6-sol",
    "gpt-5.6-terra",
    "gpt-oss-20b",
    "grok-4.5",
    "grok-4.6",
    "kimi-k3",
    "minimax-m3",
    "muse-glimmer-30b",
    "nemotron-3.5-lightning",
    "pr-review-local",
    "qwen3.6-27b",
    "qwen3.6-35b-a3b",
    "qwen3.6-35b-a3b-classifier",
    "qwen3.8-2.4t-a95b",
    "qwen3.8-27b"
  ],
  "subscription_entry": {
    "model_name": "claude-code-subscription",
    "litellm_params": {
      "model": "anthropic/claude-sonnet-5",
      "api_key": "sk-ant-oat-PLACEHOLDER-CLIENT-SENDS-ITS-OWN-TOKEN"
    },
    "model_info": {
      "input_cost_per_token": 0,
      "output_cost_per_token": 0
    }
  },
  "general_settings": {
    "store_model_in_db": false,
    "store_prompts_in_spend_logs": true,
    "maximum_spend_logs_retention_period": "30d"
  },
  "router_settings_fallbacks": [
    {
      "chat-ha": [
        "claude-sonnet-5"
      ]
    },
    {
      "auto": [
        "claude-sonnet-5"
      ]
    }
  ],
  "router_settings_context_window_fallbacks": [
    {
      "chat-ha": [
        "claude-sonnet-5"
      ]
    },
    {
      "auto": [
        "claude-sonnet-5"
      ]
    }
  ]
}
Evidence: kustomize-emitted LiteLLMVirtualKey YAML

apiVersion: litellm.home-operations.com/v1alpha1 kind: LiteLLMVirtualKey metadata: name: claude-code-subscription spec: keyAlias: claude-code-subscription models: [claude-code-subscription] rpmLimit: 300 tpmLimit: 7500000 (maxBudget absent)

apiVersion: litellm.home-operations.com/v1alpha1
kind: LiteLLMVirtualKey
metadata:
  name: claude-code-subscription
  namespace: ai
spec:
  keyAlias: claude-code-subscription
  models:
  - claude-code-subscription
  proxyRef: litellm
  rpmLimit: 300
  secretKey: key
  secretName: litellm-key-claude-code-subscription
  tpmLimit: 7500000

Pipeline

Updates from git push no-mistakes

✅ **intent** - passed

✅ No issues found.

✅ **Rebase** - passed

✅ No issues found.

✅ **Review** - passed

✅ No issues found.

✅ **Test** - passed

✅ No issues found.

  • Updated EXPECTED_RPM/EXPECTED_TPM in scripts/ci/litellm-claude-code-subscription-test.py from 10/250000 to 300/7500000 to match the captain-approved CR values
  • EVIDENCE_OUT=... python3 scripts/ci/litellm-claude-code-subscription-test.py (40/40 pass, including rpm/tpm sizing, no maxBudget, models scope, sibling isolation, kustomize emit)
  • kubectl kustomize kubernetes/apps/base/ai/litellm/app consumer parse asserting emitted LiteLLMVirtualKey rpmLimit=300 tpmLimit=7500000, models=['claude-code-subscription'], maxBudget absent
  • tokens-per-request headroom check: 250000/10 == 7500000/300 == 25000
  • Sibling virtual-key inventory isolation (ai-pr-review/demo/ha-demo/opencode/repo-wiki/router-demo unchanged)
  • Operator comment contract: safety/no-maxBudget guardrail text preserved; sizing comment includes ~556/~171 evidence and 300/7500000
  • git diff isolation: only kubernetes/apps/base/ai/litellm/app/virtualkeys/claude-code-subscription.yaml under litellm vs base
✅ **Document** - passed

✅ No issues found.

✅ **Lint** - passed

✅ No issues found.

✅ **Push** - passed

✅ No issues found.

Measured 429s in LiteLLM logs at the old 10 rpm limit presented to the
client as timeouts. Rewrites the sizing comment to reflect the real,
current rationale instead of the stale "one interactive CLI user"
justification.
Rpm-only fix left roughly a quarter of the observed 429s in place: logs
show ~171 token-limit 429s alongside ~556 request-limit 429s in 24h.
7500000 keeps the same tokens-per-request headroom now that rpm is 300,
since Claude Code requests can carry 50-200k token contexts.
@mortyops

mortyops Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor
--- LiteLLMVirtualKey ai/claude-code-subscription
+++ LiteLLMVirtualKey ai/claude-code-subscription
@@ -12,8 +12,8 @@
   models:
   - claude-code-subscription
   proxyRef: litellm
-  rpmLimit: 10
+  rpmLimit: 300
   secretKey: key
   secretName: litellm-key-claude-code-subscription
-  tpmLimit: 250000
+  tpmLimit: 7500000
 

@github-actions

Copy link
Copy Markdown

Automated recommendation: APPROVE

Analysis engine: pr-review-local@http://litellm.ai.svc.cluster.local:4000/v1 (openai)

Recommendation

Approve. This PR correctly raises the claude-code-subscription LiteLLM virtual key rate limits based on measured 429 evidence, updates the corresponding documentation and CI test expectations, and preserves the critical "no maxBudget" guardrail design.

Change-by-Change Findings

1. kubernetes/apps/base/ai/litellm/app/virtualkeys/claude-code-subscription.yaml

  • Rate Limit Increase: rpmLimit raised from 10 to 300 and tpmLimit from 250000 to 7500000. This is proportional (300x increase in RPM matches 30x increase in TPM, preserving the ~25k tokens-per-request headroom). The sizing comment is rewritten to reflect the measured evidence (~556 request-limit 429s and ~171 token-limit 429s in 24h) rather than the previous heuristic.
  • Guardrail Preservation: The maxBudget field remains unset, which is correct for this $0-priced model. The comment explicitly states that rate limits are the "whole guardrail" and neither should be dropped. This aligns with the AI stack standards.
  • Scope: The models list remains scoped to claude-code-subscription only. No fallback chains or other keys are touched.

2. docs/ai-system/litellm/claude-code-subscription.md

  • Documentation Sync: The doc is updated to reflect the new limits (300 / 7500000) and the date of the change (2026-08-30). It also clarifies that the LiteLLMVirtualKey is server-side state reconciled by litellm-operator, so changes take effect after merge + reconcile. This manages expectations about post-merge verification.

3. scripts/ci/litellm-claude-code-subscription-test.py

  • Test Expectations: EXPECTED_RPM and EXPECTED_TPM are updated to 300 and 7500000 respectively. The comment explains the rationale (measured 429s, proportional headroom). The file mode change to 755 is appropriate for a CI script.

Standards Compliance

  • AI Stack Standards: The PR adheres to the kubernetes/apps/base/ai/ standards. It does not add fallbacks that bypass allow-lists, does not name the HTTPRoute after the proxy CR, and preserves the $0-priced key's design (no maxBudget, rate limits as sole guardrail).
  • No Secrets: No secrets are introduced or modified. The key is managed by ExternalSecret/1Password.
  • Flux/Kustomize: No ${...} variables are introduced that could cause strict envsubst failures. The CR is a standard LiteLLMVirtualKey.
  • Commits: The PR title follows the type(scope): description convention (feat(litellm): ...).

Tool Harness Findings

  • Tool harness was disabled.

Unknowns or Needs Verification

  • Post-Merge Verification: The PR body correctly notes that verifying the cessation of 429s is a post-merge check. This is not a blocker for merge but should be monitored after the litellm-operator reconciles the new CR.
  • Anthropic Upstream Limits: The PR correctly identifies that the real ceiling is Anthropic's own rate limiting on the personal subscription. If 429s persist after this change, the issue is upstream of the proxy. This is a known limitation and does not block the merge.

@Aviator-Coding
Aviator-Coding merged commit 10ad92b into main Aug 31, 2026
18 checks passed
@Aviator-Coding
Aviator-Coding deleted the fm/homeops-litellm-claude-sub-rpm branch August 31, 2026 00:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant