feat(llm-gateway): gate premium models behind usage-based plans by adboio · Pull Request #69856 · PostHog/posthog

adboio · 2026-07-10T01:06:19Z

Problem

Fable is being removed from seat-based Code subscriptions — premium models will require a usage-based plan. The flip must be toggleable (timed with user comms) and shouldn't require client UI changes.

Changes

New env toggle LLM_GATEWAY_PREMIUM_MODEL_GATE_ENABLED (default off = today's behavior), matching this service's convention — deliberately not the gateway's first feature-flag evaluation. One ordering constraint at cutover: flip on only after the usage-based opt-in flow is verified live, so nobody loses fable without a path to get it back. Doubles as the instant rollback lever (flip off, no deploy).
Completion path: after plan resolution, a premium model on a gated product (posthog_code only, via ProductConfig.premium_models_gated) 403s unless the plan is usage-based. Not fail-open on unknown plan: the off-by-default setting is the safety valve; failing open would let broken plan resolution hand out premium models free.
/v1/models filters premium models for non-usage-based/anonymous callers when the gate is on — so dynamic model pickers (the Code app) need zero UI work. Auth is lazy behind pure-settings checks: gate off (or ungated product) stays byte-identical to today — anonymous, static, zero I/O (pinned by test). Never 401s.

How did you test this code?

Automated only: new cases across test_dependencies.py / test_models_api.py / test_product_config.py — deny/allow matrix per plan, ungated products unaffected, and two regression tests pinning the zero-I/O property. Gateway suite: 1223 passed.

Automatic notifications

Publish to changelog?
Alert Sales and Marketing teams?

Docs update

🤖 Agent context

Autonomy: Human-driven (agent-assisted)

pi session (fable) with worker subagents, tightened across three driver-review passes: a bespoke feature-flag evaluator was replaced with the env-var convention; an eager optional-auth route dependency was made lazy; the leftover one-line auth wrapper was deleted in favor of calling authenticate_request directly.

adboio · 2026-07-10T01:06:38Z

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

feat(llm-gateway): gate premium models behind usage-based plans #69856 👈 (View in Graphite)
feat(posthog-code): split seat-covered usage out of billed code credits #69855 : 1 other dependent PR (#69860 )
master

This stack of pull requests is managed by Graphite. Learn more about stacking.

trunk-io · 2026-07-10T01:23:38Z

_{View Full Report ↗︎ ⋅ Docs}

greptile-apps · 2026-07-10T17:44:12Z

Security Review

The PR adds a premium-model entitlement boundary, but two changed paths can break that boundary: token-bearing model-list requests can error instead of falling back to anonymous filtering, and completion requests can bypass the gate with a valid premium-model alias that does not exactly match the configured ID.

_{Reviews (1): Last reviewed commit: "feat(llm-gateway): gate premium models b..." | Re-trigger Greptile}

veria-ai · 2026-07-10T17:51:04Z

PR overview

This pull request adds policy logic in the LLM gateway to restrict access to premium models such as claude-fable-5 based on usage-based plan eligibility. The touched code centers on premium model entitlement checks for requests routed through gateway product integrations.

There is still one open security issue: the premium-model gate can be bypassed by calling sibling product message routes that accept the same PostHog Code OAuth credentials. That means an otherwise ineligible user may still reach a premium model despite the new plan check. One issue has already been addressed, so the review is trending down, but this remaining route-level bypass needs to be closed before the gating is complete.

Open issues (1)

Medium: Premium gate bypass through sibling product routes — services/llm-gateway/src/llm_gateway/services/premium_model_policy.py:12

Fixed/addressed: 1 · PR risk: 6/10

richardsolomou · 2026-07-13T11:12:21Z

+    user = await get_auth_service().authenticate_request(request, request.app.state.db_pool)
+    if user is None:
+        return True
+    plan_info = await resolve_plan_info(request, user.user_id, product)


can we resolve the product alias before passing it here? /array/v1/models and /twig/v1/models enable the gate through get_product_config, but resolve_plan_info gets the raw alias and returns no plan. usage-based users then lose premium models from the listing even though completion requests through those aliases are allowed.

richardsolomou · 2026-07-13T11:12:32Z

    )

+    model = await get_model_from_request(request)
+    await _check_premium_model_gate(product, model, plan_info.plan_key)


can we add one behavioral test through enforce_throttles or a completion endpoint? the current tests call _check_premium_model_gate directly, so removing this wiring or passing the wrong product/model/plan leaves them green while premium requests stop being gated.

richardsolomou · 2026-07-13T11:12:44Z



-def _build_response(product: str) -> ModelsResponse:
+async def _should_omit_premium_models(request: Request, product: str) -> bool:


could we share the premium-model policy between this path and _check_premium_model_gate? the product opt-in, setting, model set and plan rules are implemented twice, so model listing and request enforcement can drift independently.

richardsolomou · 2026-07-13T11:12:58Z

        product=product,
    )

+    model = await get_model_from_request(request)


this parses the request JSON again after enforce_product_access already extracted the model. API-key requests parse it once more for the end-user ID too. since bodies can be 10 MB, can we cache the parsed JSON or carry the model forward?

Fable (claude-fable-5) is being removed from seat-based PostHog Code subscriptions: once usage-based billing is live, premium models require a usage-based plan. The gate sits behind a new env setting LLM_GATEWAY_PREMIUM_MODEL_GATE_ENABLED (default OFF = today's behavior), flipped via config rollout at cutover, timed with user comms - matching this service's convention for runtime toggles (the gateway evaluates no feature flags, and this deliberately doesn't introduce the first one). - Completion path: after plan resolution in enforce_throttles, a premium model on a gated product (posthog_code only, via new ProductConfig.premium_models_gated) is 403'd unless the caller's plan is usage-based. Deliberately NOT fail-open on unknown plan: the off-by-default setting is the safety valve; fail-open would let a broken plan-resolution roundtrip hand out premium models for free. - /v1/models becomes plan-aware with lazy, optional auth: pure-settings checks short-circuit first, so with the gate off (or for ungated products) the endpoint stays byte-identical to today - anonymous, static, zero I/O (pinned by test). With the gate on, token-bearing requests cost cached auth + cached plan lookups (Django roundtrip only on cold cache); premium models are omitted for non-usage-based or anonymous callers, so clients with dynamic model pickers (the Code app) need zero UI work. Never 401s. Settings (premium_models list and the gate toggle) are env-overridable without code change.

hex-security-app · 2026-07-13T16:22:34Z

+        return True
+
+    normalized_model = model.lower()
+    if not normalized_model.startswith(_premium_model_prefixes()):


Premium policy still allows provider-prefixed premium aliases

The new shared policy only expands raw canonical IDs and Bedrock IDs, then uses startswith against the user-supplied model string. Anthropic/OpenAI-prefixed forms like anthropic/claude-fable-5 still do not match _premium_model_prefixes(), but those IDs are accepted elsewhere in the gateway and resolve to the same premium model. A non-usage-based posthog_code caller can therefore still request the premium model by using a provider-prefixed alias and bypass the new billing authorization check.

Prompt To Fix With AI

Canonicalize incoming model IDs before applying the premium-model policy. The policy should normalize provider-prefixed LiteLLM forms (for example `anthropic/claude-fable-5`, `openai/...` where relevant), canonical bare IDs, dated variants, and Bedrock aliases to the same underlying model identity before deciding whether the model is premium. Add regression tests covering provider-prefixed aliases on both the completion path and `/v1/models`.

_{Severity: medium | Confidence: 97% | React with 👍 if useful or 👎 if not}

veria-ai · 2026-07-13T16:34:51Z

+def is_premium_model_gate_active(product: str) -> bool:
+    config = get_product_config(product)
+    settings = get_settings()
+    return bool(config and config.premium_models_gated and settings.premium_model_gate_enabled)


Medium: Premium gate bypass through sibling product routes

A non-usage-based PostHog Code user can request claude-fable-5 through /background_agents/v1/messages or /slack_app/v1/messages. Those products accept the same PostHog Code OAuth application IDs and allow this model, but this product-based check leaves their gate disabled; enforce the entitlement independently of the caller-selected route, or restrict these sibling products to credentials that normal PostHog Code users cannot obtain.

…ode usage Orgs without Code usage billing get open models only (default @cf/zai-org/glm-5.2); orgs whose Code usage bills keep the full allowlist. Rebuild of the closed #69856 with its review findings addressed: - Keyed on the same code_usage_billing_active bit as the per-user cap lift (one signal, resolved once per request, fail-closed False), not on plan keys - so enforcement, the cap bypass, and the listing can never disagree about who pays. - Free-list matching reuses _model_matches_product_allowlist (prefix + Bedrock alias expansion), closing the exact-match hole where a date-suffixed alias variant slipped past the gate. - /{product}/v1/models filters to the free list for unbilled callers and fails closed to it on any resolution error - a token-bearing listing request can degrade but never 500 (the other #69856 finding). - Staff exempt via the existing is_usage_unlimited pattern. Off by default (LLM_GATEWAY_POSTHOG_CODE_MODEL_GATE_ENABLED) so this deploys inert ahead of the cutover; the free-model list is env-overridable without a deploy. Deliberately scoped to the posthog_code product surface (incl. array/twig aliases). Sibling products reachable with the same OAuth app (background_agents, conversations, signals, slack_app) are internal or separately billed surfaces whose pipelines pin non-free models; gating them on Code billing would break PostHog-paid flows for free orgs. Their exposure stays bounded by per-user/product cost caps, and closing the shared- credential provenance question remains the separate internal-product authorization design (see the revert in 8092b81). Generated-By: PostHog Code Task-Id: e6d23925-2a64-452b-aeea-fd349077f904

…ode usage (#70951) ## Problem Orgs not billed for Code usage should get open models only. The gateway currently serves the same posthog_code model allowlist to everyone. ## Changes Behind `LLM_GATEWAY_POSTHOG_CODE_MODEL_GATE_ENABLED` (default off, flipped at cutover): - posthog_code requests (including the array/twig aliases) from users whose org isn't billed for Code usage only pass models in `posthog_code_free_tier_models` (default `@cf/zai-org/glm-5.2`). Anything else 403s with the free list in the message. Staff exempt. - Keys on the same fail-closed `code_usage_billing_active` bit as #70946. - Matching goes through `_model_matches_product_allowlist`, so date-suffixed and Bedrock variants of a blocked model are blocked too. - Model extraction reads form-encoded bodies as well as JSON, and `model` is now a required form field on the audio transcription routes, as in OpenAI's API. It was optional with a server-side default, so the access checks ran with `model=None` while the handler routed a model upstream unchecked. Every route now requires a model at validation, making `model=None` safe to pass everywhere. This also makes per-product model allowlists apply to transcription for the first time; no caller is affected (the OpenAI SDKs require `model`, and no client of an allowlisted product uses these routes — checked this repo and posthog/code). - `/{product}/v1/models` always returns the full catalog. With the gate on, callers positively identified as free-tier (authenticated, non-staff, org not billed) get models outside the free tier marked `allowed: false` with `restriction_reason: "paid_plan_required"` instead of omitted, so clients can render them disabled with an upgrade prompt. The fields are additive with `allowed: true` defaults; all consumers parse leniently (posthog/code TS casts, codex 0.140.0 serde without `deny_unknown_fields`, OpenAI SDK). Anonymous callers and auth failures are never marked — an unidentifiable caller may be a billed org. Quota-fetch failures mark restricted off the same last-known billing bit enforcement reads, so marks match the 403s requests would get. Enforcement stays the actual gate. Cutover dependency in posthog/code: the gateway models fetches (`packages/agent/src/gateway-models.ts`, `packages/harness/src/extensions/posthog-provider/models.ts`) should send the user's token so free-tier users get accurate `allowed` marks (anonymous fetches see everything allowed), render `allowed: false` entries disabled with the upgrade prompt, and pick the best allowed model as the session default. The internal products on the same OAuth app (background_agents, conversations, signals, slack_app) are only ever driven by server-minted sandbox tokens, so they now require a server-minted credential: sandbox tokens carry an internal `internal_run:read` scope (added to `INTERNAL_SCOPES`), and those products reject OAuth callers that lack it (`requires_server_credential`, behind the same flag). This stops a user's own Code OAuth token from routing around the free-tier gate to premium models through them — those products bill elsewhere or not at all, so the escape would otherwise spend up to the sibling per-user caps on PostHog. Internal scopes can't be obtained via the consent flow or a personal API key, so the marker can't be forged; personal API keys are unaffected (they need an explicit, feature-gated `llm_gateway:read` scope, so the shared server-side gateway key still works). Supersedes #69856. ## How did you test this code? Full gateway unit suite green (1233 tests). Plus end-to-end against a live local stack — gateway from this branch (`debug=false`, so app-ID checks ran) wired to local Django/Postgres/Redis, real Anthropic calls, seeded free/billed/staff orgs with consent-style, server-minted, and personal-API-key credentials: - flag off: behavior unchanged — full unmarked listing, free-org token completes premium-model calls, consent token reaches `background_agents` - flag on: free org 403s with the paid-plan message on `/chat/completions`, `/messages`, and form-encoded transcription; `@cf/zai-org/glm-5.2` passes the gate; free-org server tokens gated too; billed org and staff complete real calls; blocks log `free_tier_model_blocked` - listing: free-tier callers (consent, server token, PAK) see premium models `allowed: false` / `paid_plan_required` with the free model still allowed; anonymous, invalid-token, billed, and staff callers get the full unmarked list; other products' listings untouched - server credential: consent tokens (free and billed) 403 on all four internal products; `internal_run:read` token passes; PAK on `signals` unaffected - transcription without `model` 422s on both routes; malformed JSON 422s, never 500s - simulated Django outage (quota cache cleared): billed org keeps model access via the last-known billing bit through the full 35s retry backoff, free org stays gated, `quota_fetch_failed` logged - found one deploy coupling: without Cloudflare credentials the registry omits the free model, so free-tier callers see everything restricted and free-model requests 503 — don't flip the flag on a deployment missing CF creds ## Automatic notifications - [ ] Publish to changelog? - [ ] Alert Sales and Marketing teams? ## Docs update None yet; free-tier docs land with the cutover comms. ## 🤖 Agent context **Autonomy:** Human-driven (agent-assisted) PostHog Code session (Claude). Skill: /writing-tests. Scoping to the product surface instead of the OAuth credential was deliberate: credential-wide gating would break the PostHog-paid sibling products above for free orgs. Review round 1: fixed the multipart transcription bypass hex-security found; reworked the listing after checking posthog/code (its models fetches are unauthenticated, so fail-closed anonymous filtering would have downgraded billed orgs' pickers at cutover). Round 2 (k11kirky): closed the omitted-model variant by deleting the transcription routes' server-side default — OpenAI's API requires the field and nothing depended on the default — and corrected the listing docs on auth fall-open vs deliberate quota fail-closed. Round 3 (adboio): listing marks restricted models (`allowed` + `restriction_reason`) instead of omitting them; additive-safety verified against all consumers including codex's pinned serde structs. Round 4 (veria-ai): closed the cross-product escape — the internal products share the Code OAuth app, so a user's own Code token could reach premium models through them and skip the gate. First tried confining the credential by scope *shape* (`*` present); reverted after finding the Code OAuth app can be provisioned with an explicit ceiling that grants `llm_gateway:read` (the demo generator does exactly this), which lets a user mint a non-wildcard gateway token and defeat the shape check. The shipped fix instead keys on an *internal* scope marker minted only server-side — unforgeable via any user path and independent of the app's ceiling. Verified against posthog/code that the desktop client only calls posthog_code and the internal products are driven solely by server-minted tokens, so real traffic is untouched. Round 5 (adboio): the free-tier gate's model extraction re-parsed the JSON body the product-access check had already parsed (and end-user extraction parsed it a third time) — the parsed dict is now cached per request; the server-credential opt-in is pinned by a PRODUCTS-derived invariant test so the next internal product on the Code app can't silently omit it. --- *Created with [PostHog Code](https://posthog.com/code?ref=pr)*

pr-assigner-resolver-posthog Bot assigned adboio Jul 10, 2026

This was referenced Jul 10, 2026

feat(posthog-code): bill LLM costs as pass-through credits (no markup) #69062

Merged

feat(posthog-code): enforce posthog_code_credits quota at the gateway #69324

Merged

feat(llm-gateway): usage-based posthog_code user cost limit #69325

Closed

This was referenced Jul 10, 2026

feat(posthog-code): split seat-covered usage out of billed code credits #69855

Closed

feat(posthog-code): seat purchase lockdown #69860

Closed

adboio force-pushed the posthog-code/usage-report-seat-split branch from 707c93a to dca9cda Compare July 10, 2026 15:57

adboio force-pushed the posthog-code/premium-model-plan-gate branch from 4efd4e0 to 1f1c18e Compare July 10, 2026 15:57

adboio force-pushed the posthog-code/usage-report-seat-split branch from dca9cda to 67ee0b1 Compare July 10, 2026 16:43

adboio force-pushed the posthog-code/premium-model-plan-gate branch 2 times, most recently from 46ec617 to f2936c6 Compare July 10, 2026 17:01

adboio force-pushed the posthog-code/usage-report-seat-split branch from 67ee0b1 to 0e3b603 Compare July 10, 2026 17:01

adboio force-pushed the posthog-code/premium-model-plan-gate branch 2 times, most recently from 9062dee to 6947115 Compare July 10, 2026 17:27

adboio force-pushed the posthog-code/usage-report-seat-split branch from 0e3b603 to 35bf9a2 Compare July 10, 2026 17:27

adboio force-pushed the posthog-code/premium-model-plan-gate branch from 6947115 to d36a451 Compare July 10, 2026 17:39

adboio marked this pull request as ready for review July 10, 2026 17:41

adboio requested review from a team July 10, 2026 17:41

greptile-apps Bot reviewed Jul 10, 2026

View reviewed changes

Comment thread services/llm-gateway/src/llm_gateway/api/models.py Outdated

Comment thread services/llm-gateway/src/llm_gateway/dependencies.py Outdated

adboio force-pushed the posthog-code/premium-model-plan-gate branch from d36a451 to e075b1f Compare July 10, 2026 17:47

adboio force-pushed the posthog-code/usage-report-seat-split branch from 35bf9a2 to 613c6b7 Compare July 10, 2026 17:47

hex-security-app Bot reviewed Jul 10, 2026

View reviewed changes

Comment thread services/llm-gateway/src/llm_gateway/dependencies.py Outdated

veria-ai Bot reviewed Jul 10, 2026

View reviewed changes

Comment thread services/llm-gateway/src/llm_gateway/dependencies.py Outdated

adboio force-pushed the posthog-code/premium-model-plan-gate branch from e075b1f to 7621892 Compare July 10, 2026 18:00

adboio force-pushed the posthog-code/usage-report-seat-split branch from 613c6b7 to ea81d28 Compare July 10, 2026 18:12

adboio force-pushed the posthog-code/premium-model-plan-gate branch from 7621892 to e700c7e Compare July 10, 2026 18:12

richardsolomou requested changes Jul 13, 2026

View reviewed changes

adboio force-pushed the posthog-code/usage-report-seat-split branch from ea81d28 to d35f3b7 Compare July 13, 2026 15:56

adboio force-pushed the posthog-code/premium-model-plan-gate branch from e700c7e to 31875aa Compare July 13, 2026 15:56

adboio force-pushed the posthog-code/premium-model-plan-gate branch from 31875aa to 2a0517d Compare July 13, 2026 16:21

hex-security-app Bot reviewed Jul 13, 2026

View reviewed changes

veria-ai Bot reviewed Jul 13, 2026

View reviewed changes

adboio closed this Jul 14, 2026

adboio mentioned this pull request Jul 15, 2026

feat(llm-gateway): gate posthog_code models for orgs not billed for Code usage #70951

Merged

2 tasks



		def _build_response(product: str) -> ModelsResponse:
		async def _should_omit_premium_models(request: Request, product: str) -> bool:

Uh oh!

Conversation

adboio commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Changes

How did you test this code?

Automatic notifications

Docs update

🤖 Agent context

Uh oh!

adboio commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

trunk-io Bot commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

greptile-apps Bot commented Jul 10, 2026

Security Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

veria-ai Bot commented Jul 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

PR overview

Open issues (1)

Uh oh!

richardsolomou Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

richardsolomou Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

richardsolomou Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

richardsolomou Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

hex-security-app Bot Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

veria-ai Bot Jul 13, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

adboio commented Jul 10, 2026 •

edited

Loading

adboio commented Jul 10, 2026 •

edited

Loading

trunk-io Bot commented Jul 10, 2026 •

edited

Loading

veria-ai Bot commented Jul 10, 2026 •

edited

Loading