Expose OpenAI models in the UI & make Claude model picker configurable

[lewtun]

Summary

• Add GPT-5.5 to the web model picker.
• Gate GPT-5.5 the same way as Bedrock/Anthropic Claude: HF org-member access plus the shared daily premium-model quota.
• Avoid the duplicate per-session model fetch in the chat input after model options load.
• Make the Claude UI option configurable with ML_INTERN_CLAUDE_MODEL_ID.
• Keep Bedrock as the production Claude default while allowing direct Anthropic for local development.
• Sync the frontend Claude option with the backend-resolved model ID.
• Add local dev server notes in AGENTS.md.

Deployment Notes

• To enable GPT-5.5 on the deployed Space, add OPENAI_API_KEY as a private Space secret and restart/rebuild the Space.
• ML_INTERN_CLAUDE_MODEL_ID is not required for GPT-5.5; it only controls the Claude picker entry between Bedrock and direct Anthropic.
• GPT-5.5 access is limited by HF org membership via HF_EMPLOYEE_ORG and the shared premium-model daily quota.

Validation

• npm run build
• uv run python -m py_compile backend/routes/agent.py backend/user_quotas.py backend/dependencies.py tests/unit/test_agent_model_gating.py
• uv run --extra dev pytest tests/unit/test_agent_model_gating.py tests/unit/test_user_quotas.py
• git diff --check
• Verified /api/config/model for default Bedrock and local Anthropic Claude overrides.
• Verified /api/config/model includes openai/gpt-5.5.

Closes #192

[github-actions]

Claude finished @lewtun's task in 3m 12s —— View job

---

PR Review: Expose OpenAI models in the UI

• Gather context and read changed files
• Review CLAUDE.md and REVIEW.md for guidelines
• Analyze backend changes (backend/routes/agent.py)
• Analyze frontend changes (frontend/src/components/Chat/ChatInput.tsx)
• Analyze config changes (configs/frontend_agent_config.json)
• Review new AGENTS.md file
• Post final review

---

No blocking issues — 2 P1

Verdict: ready to merge

What I checked

• End-to-end model ID flow: config JSON → substitute_env_vars → session_manager.config.model_name → _default_claude_model_id() → AVAILABLE_MODELS → /api/config/model → frontend useEffect → modelOptions → handleSelectModel → /api/session/{id}/model validation
• Auth gate correctness: _is_anthropic_model and _require_hf_for_anthropic both use "anthropic" in model_id. Both "anthropic/claude-opus-4-6" and "bedrock/us.anthropic.claude-opus-4-6-v1" contain "anthropic", so the HF-org gate still fires for either path. In local dev (no OAUTH_CLIENT_ID), auth is bypassed automatically, so the gate doesn't block local use.
• substitute_env_vars in agent/config.py:152–178: handles the ${ML_INTERN_MODEL_ID:-bedrock/...} syntax in the config JSON correctly.
• AVAILABLE_MODELS validation in create_session, restore_session_summary, set_session_model: all read from the same module-level AVAILABLE_MODELS list, so the model IDs are consistent.
• Module initialization order: session_manager is fully imported before _default_claude_model_id() is called, so session_manager.config is populated.
• findModelByPath with the new options parameter: the path?.includes(m.id) fallback still matches (e.g. 'bedrock/us.anthropic.claude-opus-4-6-v1'.includes('claude-opus') → true).
• isClaudeModel / isClaudePath: both check includes('anthropic'), consistent with backend.
• handleUseFreeModel dep array: modelOptions correctly added.

---

P1 — Double session-model fetch on mount

frontend/src/components/Chat/ChatInput.tsx:126–140

The second useEffect now lists modelOptions as a dependency. On initial mount with a sessionId already set, the effect fires once immediately, then fires again when the /api/config/model fetch completes and updates modelOptions. This causes two GET /api/session/{id} requests in a row on every fresh page load.

The fix is to decouple the session-fetch from modelOptions by reading the ref value or using a separate useRef to hold the latest options without triggering the effect:

const modelOptionsRef = useRef(modelOptions);
useEffect(() => { modelOptionsRef.current = modelOptions; }, [modelOptions]);

useEffect(() => {
  if (!sessionId) return;
  let cancelled = false;
  apiFetch(`/api/session/${sessionId}`)
    .then((res) => (res.ok ? res.json() : null))
    .then((data) => {
      if (cancelled) return;
      if (data?.model) {
        const model = findModelByPath(data.model, modelOptionsRef.current);
        if (model) setSelectedModelId(model.id);
      }
    })
    .catch(() => { /* ignore */ });
  return () => { cancelled = true; };
}, [sessionId]);  // modelOptions removed

Fix this →

---

P1 — setState call inside a setModelOptions updater

frontend/src/components/Chat/ChatInput.tsx:107–118

setModelOptions((options) => {
  const next = options.map(...);
  if (data.current) {
    const current = findModelByPath(data.current, next);
    if (current) setSelectedModelId(current.id);  // ← side effect inside updater
  }
  return next;
});

Calling setSelectedModelId inside a setModelOptions functional updater is a side effect in a function that should be pure. React 18 batches this correctly in practice, but React reserves the right to re-invoke updater functions (e.g. in Strict Mode). The intent is clearer and safer with both setters at the same level:

.then((data) => {
  if (cancelled || !data?.available) return;
  const claude = data.available.find(...);
  if (!claude?.id) return;

  const next = modelOptions.map((option) =>
    isClaudeModel(option) ? { ...option, modelPath: claude.id, name: claude.label ?? option.name } : option
  );
  setModelOptions(next);
  if (data.current) {
    const current = findModelByPath(data.current, next);
    if (current) setSelectedModelId(current.id);
  }
})

Note: this requires adding modelOptions as a captured value in the outer .then(), which is fine since the effect runs once (empty dep array).

Fix this →

---

P2 — PR title doesn't match the change

The title says "Expose OpenAI models in the UI" but the diff makes the Claude model ID configurable (Bedrock vs. direct Anthropic API key via ML_INTERN_MODEL_ID), with no OpenAI models added. Likely a Codex-generated title artifact.

[github-actions]

Claude finished @lewtun's task in 2m 17s —— View job

---

PR Review: Expose OpenAI models in the UI

• Gather context
• Read changed files
• Check for P0 issues (re-review: suppressing new P1s per policy)
• Post final review

---

LGTM
Verdict: ready to merge

What I checked

• Auth chain re-verified: _is_anthropic_model / _require_hf_for_anthropic use "anthropic" in model_id (agent.py:92–118); both bedrock/us.anthropic.claude-opus-4-6-v1 and anthropic/claude-opus-4-6 match, so the HF-org gate fires correctly under either ML_INTERN_CLAUDE_MODEL_ID value.
• AVAILABLE_MODELS validation in create_session, restore_session_summary, set_session_model (agent.py:344–346, 385–387, 447–449): all read from the same module-level list built at import time; consistent.
• _claude_picker_model_id() / AVAILABLE_MODELS initialization order (agent.py:47–89): session_manager is fully imported before _available_models() is called at module level, so session_manager.config.model_name is populated.
• findModelByPath with the options parameter (ChatInput.tsx:61–63): path?.includes(m.id) fallback is safe when path is defined; if path is undefined it returns falsy and no match is made.
• configs/frontend_agent_config.json: ${ML_INTERN_CLAUDE_MODEL_ID:-bedrock/us.anthropic.claude-opus-4-6-v1} syntax is handled correctly by substitute_env_vars.
• AGENTS.md: documentation-only addition, no behavioral changes.
• P1 from prior review (setState inside functional updater, ChatInput.tsx:107–118): the current code does not exhibit this pattern — setModelOptions(next) and setSelectedModelId(current.id) are called at the same level in the .then() handler. Resolved before this review pass.
• P1 from prior review (double fetch, ChatInput.tsx:136): modelOptions remains in the dep array — not re-flagging per re-review policy, but the prior recommendation to use a useRef to decouple the session fetch still stands.

[github-actions]

Claude finished @lewtun's task in 2m 51s —— View job

---

PR Review: Expose OpenAI models in the UI

• Gather context
• Read changed files
• Check for P0 issues (re-review: suppressing new P1s per policy)
• Post final review

---

1 P0
Verdict: changes requested

What I checked

• openai/gpt-5.5 routing in agent/core/llm_params.py:171–181: openai/ prefix triggers the direct-OpenAI path — no api_base override, no api_key set — LiteLLM uses OPENAI_API_KEY from the environment. Not routed through the HF router and gets no X-HF-Bill-To header.
• Billing gate coverage in set_session_model (agent.py:453–456), create_session (agent.py:350–357), restore_session_summary (agent.py:391–396): all three call _require_hf_for_anthropic, which checks "anthropic" in model_id. "openai/gpt-5.5" does not match → gate never fires.
• _enforce_claude_quota (agent.py:127–163): only charges quota when _is_anthropic_model is true. GPT-5.5 is exempt.
• Frontend model selector: handleSelectModel posts model.modelPath directly to /api/session/{id}/model; no client-side org-membership check for the GPT-5.5 entry.
• Prior P1 (setState inside updater, ChatInput.tsx:107–118): confirmed resolved — setModelOptions(next) and setSelectedModelId(current.id) are now at the same .then() level.
• Prior P1 (double fetch / modelOptions dep array, ChatInput.tsx:143): modelOptions still in the dep array — not re-flagging per re-review policy.
• findModelByPath with new options param: safe, path?.includes(m.id) fallback handles undefined path.
• AGENTS.md: documentation-only, no behavioral change.

---

P0 — openai/gpt-5.5 billed to OPENAI_API_KEY with no access gate

backend/routes/agent.py:73–78 (AVAILABLE_MODELS entry) and agent/core/llm_params.py:171–181 (routing)

openai/gpt-5.5 is marked tier: "pro" but is reachable by any authenticated user. When OPENAI_API_KEY is set in the Space environment, it will be billed to that key with zero guard:

• _require_hf_for_anthropic (agent.py:102–124) only fires when "anthropic" in model_id — "openai/gpt-5.5" returns false at agent.py:99.
• _resolve_llm_params for openai/ (llm_params.py:171–181) emits {"model": "openai/gpt-5.5"} with no api_base and no api_key — LiteLLM picks up OPENAI_API_KEY from the environment.
• The HF-router billing path (X-HF-Bill-To, llm_params.py:190–191) is only reached for the HuggingFace fallback branch — not for openai/ prefixed models.

The established pattern in this repo is to gate any model that consumes a paid API key: Anthropic models require HF-org membership because they bill to ANTHROPIC_API_KEY. The same logic applies to GPT-5.5 — once OPENAI_API_KEY is configured, all authenticated users can exhaust it freely.

Minimum fix — extend the gate to cover OpenAI models in all three write paths:

def _is_openai_model(model_id: str) -> bool:
    return model_id.startswith("openai/")

async def _require_hf_for_openai(request: Request, model_id: str) -> None:
    if not _is_openai_model(model_id):
        return
    if not await require_huggingface_org_member(request):
        raise HTTPException(
            status_code=403,
            detail={
                "error": "openai_restricted",
                "message": "GPT-5.5 is gated to HF staff. Pick a free model instead.",
            },
        )

Then call await _require_hf_for_openai(request, resolved_model) alongside _require_hf_for_anthropic in create_session (agent.py:357), restore_session_summary (agent.py:396), and set_session_model (agent.py:456).

Fix this →

[github-actions]

Claude finished @lewtun's task in 2m 1s —— View job

---

PR Review: Expose OpenAI models in the UI

• Gather context
• Read changed files (agent.py, user_quotas.py, llm_params.py, tests)
• Verify P0 fix: OpenAI gate added
• Check lewtun's inline comment (env var at agent.py:51)
• Post final review

---

LGTM
Verdict: ready to merge

What I checked

• P0 from prior review (GPT-5.5 billed to OPENAI_API_KEY with no gate): resolved. The fix introduces PREMIUM_GATED_MODEL_IDS = {"openai/gpt-5.5"} (agent.py:45) and a unified _is_premium_model() predicate (agent.py:103–104) that covers both anthropic models and the new set. _require_hf_for_premium_model (agent.py:107–128) replaces the former _require_hf_for_anthropic and is called in all three write paths: create_session:361, restore_session_summary:400, set_session_model:460. _enforce_premium_model_quota (agent.py:131–168) similarly covers GPT-5.5. New tests in tests/unit/test_agent_model_gating.py confirm the predicate, the gate rejection, and the quota charge for GPT-5.5.
• llm_params.py:171–181: the openai/ routing path still uses OPENAI_API_KEY from the environment with no api_base override — that's correct, and now the HF-org membership gate makes it safe.
• user_quotas.py: claude_counted / try_increment_claude are shared across premium models by design (user_quotas.py:6–16); the docstring makes this explicit. No issues.
• Prior P1 (setState inside functional updater, ChatInput.tsx:107–118): confirmed resolved (second review).
• Prior P1 (double fetch / modelOptions dep array, ChatInput.tsx): not re-flagging per re-review policy.

lewtun's inline question — agent.py:51

Why are we referencing this env var here?

The docstring references ML_INTERN_CLAUDE_MODEL_ID because that env var is the mechanism that controls what _claude_picker_model_id() returns, even though the function doesn't read it directly. The env var is substituted by substitute_env_vars when the JSON config is loaded (configs/frontend_agent_config.json:2 — ${ML_INTERN_CLAUDE_MODEL_ID:-bedrock/...}), which populates session_manager.config.model_name. The function then returns that value. The docstring is documenting the indirect control path — if you want to change what the Claude menu entry sends to the backend, you set ML_INTERN_CLAUDE_MODEL_ID. Nothing wrong here.

[lewtun]

We don't need to be backwards compatible

[github-actions]

Claude finished @lewtun's task in 1m 52s —— View job

---

PR Review: Expose OpenAI models in the UI

• Gather context
• Read changed files (agent.py, ChatInput.tsx, useUserQuota.ts, model.ts, tests)
• Check for P0 issues (re-review: suppressing new P1s per policy)
• Post final review

---

LGTM
Verdict: ready to merge

What I checked

• Prior P1 (double fetch / modelOptions dep, ChatInput.tsx:163): resolved. The latest commit introduces modelOptionsRef (line 95) kept in sync via a dedicated effect (lines 112–114), and sessionIdRef (line 96) kept in sync at lines 116–118. The session-fetch effect (lines 149–163) now depends only on [sessionId] and reads from modelOptionsRef.current — no double-fetch on mount.
• Prior P0 (GPT-5.5 billed with no gate): confirmed resolved from prior pass — PREMIUM_GATED_MODEL_IDS + _require_hf_for_premium_model cover all three write paths.
• Prior P1 (setState inside functional updater): confirmed resolved from prior pass — setModelOptions(next) and setSelectedModelId(current.id) called at the same .then() level.
• isPremiumPath in frontend/src/utils/model.ts:17–19: mirrors the backend _is_premium_model — isClaudePath (includes 'anthropic') plus exact match on 'openai/gpt-5.5'. Consistent.
• Tests (test_agent_model_gating.py): cover predicate, gate rejection for GPT-5.5, gate skip for free models, and quota charge for GPT-5.5. Coverage looks complete for the new gating logic.

lewtun's inline note — agent.py:504

We don't need to be backwards compatible

Safe to remove. useUserQuota.ts:35–37 reads premium_used_today / premium_daily_cap / premium_remaining as the primary fields; the ?? fallback to claude_* is dead code once those aliases are gone. The ?? chain will just fall to the hardcoded 0/1/0 defaults, which are never reached since premium_* will always be present in the response.

[github-actions]

Claude finished @lewtun's task in 1m 25s —— View job

---

PR Review: Expose OpenAI models in the UI

• Gather context
• Read changed files (agent.py, user_quotas.py, useUserQuota.ts, ChatInput.tsx, model.ts, tests)
• Verify prior fixes still in place
• Check new commits for P0 issues
• Post final review

---

LGTM
Verdict: ready to merge

What I checked

• fbce9e1 (remove legacy aliases): get_user_quota (agent.py:492–504) now returns only premium_used_today / premium_daily_cap / premium_remaining — the claude_* fields are gone. useUserQuota.ts:33–38 already reads premium_* directly with ?? 0/1/0 hardcoded fallbacks; nothing is left reading the removed aliases. New test test_user_quota_response_uses_premium_fields_only (test_agent_model_gating.py:86–101) explicitly asserts the response shape. Resolves lewtun's "we don't need to be backwards compatible" comment.
• 9a48f29 (avoid duplicate session fetch): modelOptionsRef (ChatInput.tsx:95) and sessionIdRef (ChatInput.tsx:96) kept in sync via dedicated effects (lines 112–118). Session-fetch effect (ChatInput.tsx:149–163) now depends only on [sessionId] and reads from modelOptionsRef.current — prior P1 (double-fetch on mount) confirmed resolved.
• Prior P1 (setState inside functional updater): setModelOptions(next) and setSelectedModelId(current.id) called at the same .then() level (ChatInput.tsx:136–140) — confirmed resolved from prior pass.
• Prior P0 (GPT-5.5 billed with no gate): PREMIUM_GATED_MODEL_IDS + _require_hf_for_premium_model (agent.py:45, 107–128) covers all three write paths (create_session:361, restore_session_summary:400, set_session_model:460) — confirmed resolved from prior pass.
• isPremiumPath (model.ts:17–19): covers both isClaudePath (includes 'anthropic') and exact match on GPT_55_MODEL_PATH. Mirrors backend _is_premium_model (agent.py:103–104). Consistent.
• useUserQuota.ts frontend fields align with get_user_quota backend keys with no mismatch.

[lewtun]

This is just for local dev and does not need to be tested

[lewtun]

Wouldn't it be cleaner to drop BEDROCK_CLAUDE_MODEL_ID in favour of having all models in PREMIUM_GATED_MODEL_IDS? Then the claude model picker would only be overidden if the developer sets ML_INTERN_CLAUDE_MODEL_ID

[github-actions]

Claude finished @lewtun's task in 1m 38s —— View job

---

PR Review: Expose OpenAI models in the UI

• Gather context
• Read changed files (new commits since last review)
• Check lewtun's inline comments for follow-up
• Check for P0 issues (re-review: suppressing new P1s per policy)
• Post final review

---

LGTM
Verdict: ready to merge

What I checked

• cddea75 (document GitHub CLI PR body workflow): documentation-only addition to AGENTS.md — 4 lines adding a note about gh pr edit --body-file. No behavioral change, no P0.
• All prior P0/P1 fixes confirmed in place from last review pass: PREMIUM_GATED_MODEL_IDS gate, setModelOptions/setSelectedModelId at same .then() level, modelOptionsRef decoupling, legacy claude_* alias removal.

lewtun's inline comments

test_agent_model_gating.py:26 — "This is just for local dev and does not need to be tested"

The assert agent._is_premium_model("anthropic/claude-opus-4-6") assertion is still present. It exercises the local-dev path that ML_INTERN_CLAUDE_MODEL_ID=anthropic/claude-opus-4-6 enables. Not a blocking issue — remove it if you'd prefer to keep test scope to production behavior only.

agent.py:45 — "Wouldn't it be cleaner to drop BEDROCK_CLAUDE_MODEL_ID in favour of having all models in PREMIUM_GATED_MODEL_IDS?"

The suggestion would replace the "anthropic" in model_id substring check with an explicit set membership check, making PREMIUM_GATED_MODEL_IDS = {"bedrock/us.anthropic.claude-opus-4-6-v1", "openai/gpt-5.5"} the single source of truth. The trade-off: any future ML_INTERN_CLAUDE_MODEL_ID value not in the set would silently bypass the gate, whereas the current "anthropic" in model_id approach catches all Anthropic variants automatically. Both approaches are valid — the explicit-set design is cleaner if model IDs are expected to be stable; the substring approach is more defensive. Not a blocking issue either way.

[github-actions]

Claude finished @lewtun's task in 2m 20s —— View job

---

PR Review: Expose OpenAI models in the UI

• Gather context
• Read changed files (latest commits since last review)
• Check for P0 issues (re-review: suppressing new P1s per policy)
• Post final review

---

LGTM
Verdict: ready to merge

What I checked

• a9a1931 (gate only deployed paid model IDs): The refactor replaces the "anthropic" in model_id substring check with an explicit GATED_MODEL_IDS = {"bedrock/us.anthropic.claude-opus-4-6-v1", "openai/gpt-5.5"} set (agent.py:44–48). I traced the full routing chain to verify the security posture is unchanged for the default production deployment:
  • Default production (ML_INTERN_CLAUDE_MODEL_ID unset): _claude_picker_model_id() returns "bedrock/us.anthropic.claude-opus-4-6-v1", which IS in GATED_MODEL_IDS → gate fires correctly.
  • Local dev (ML_INTERN_CLAUDE_MODEL_ID=anthropic/claude-opus-4-6): anthropic/claude-opus-4-6 is intentionally NOT in GATED_MODEL_IDS — this path uses the developer's own ANTHROPIC_API_KEY as confirmed in llm_params.py:140–161. Gate is intentionally silent. Behaviour is documented in the _require_hf_for_gated_model docstring (agent.py:108–111) and in tests (test_gated_model_predicate_includes_bedrock_claude_and_gpt55_only:27, test_gated_quota_skips_direct_anthropic).
• llm_params.py routing verified end-to-end: anthropic/ → direct Anthropic API + ANTHROPIC_API_KEY (llm_params.py:140); bedrock/ → AWS Converse (llm_params.py:163); openai/ → OPENAI_API_KEY (llm_params.py:171). All three gated model paths (bedrock/us.anthropic... and openai/gpt-5.5) bill to shared credentials and are covered by GATED_MODEL_IDS.
• isPremiumPath in model.ts (model.ts:17–19): CLAUDE_MODEL_PATH = 'bedrock/us.anthropic.claude-opus-4-6-v1' and GPT_55_MODEL_PATH = 'openai/gpt-5.5' match GATED_MODEL_IDS exactly. Frontend and backend definitions are consistent.
• All prior fixes confirmed in place: P0 (GPT-5.5 gate), P1 (setState inside updater), P1 (double session-fetch via modelOptionsRef), legacy claude_* alias removal.

[github-actions]

Claude finished @lewtun's task in 56s —— View job

---

PR Review: Expose OpenAI models in the UI

• Gather context
• Read changed files (latest commit 6165a25 Nits)
• Check for P0 issues (re-review: suppressing new P1s per policy)
• Post final review

---

LGTM
Verdict: ready to merge

What I checked

• 6165a25 (Nits): Documentation-only. Two docstrings updated in backend/routes/agent.py:
  • _claude_picker_model_id() (agent.py:52–58): rewritten to describe the ML_INTERN_CLAUDE_MODEL_ID → substitute_env_vars → session_manager.config.model_name → return path more clearly. Accurate.
  • _require_hf_for_gated_model() (agent.py:108–113): simplified — removed the now-stale note about anthropic/... being intentionally excluded (that context lives in GATED_MODEL_IDS itself and the tests). No behavioral change.
• All prior fixes confirmed in place from last review pass: GATED_MODEL_IDS set covers both bedrock/us.anthropic.claude-opus-4-6-v1 and openai/gpt-5.5; _require_hf_for_gated_model wired into create_session:361, restore_session_summary:400, set_session_model:460; modelOptionsRef decouples session fetch; legacy claude_* quota aliases removed.

[lewtun]

Added this to streamline agentic usage

[lewtun]

@akseljoonas @cmpatino this is handy to bypass the Bedrock hard-coded endpoint for local dev

[jagwar]

Non-blocking nit: git diff --check currently reports trailing whitespace in backend/routes/agent.py around the _require_hf_for_gated_model() docstring.

This does not affect runtime behavior, but the PR description lists git diff --check as validation, so either the trailing space should be removed or that validation line should be updated.