feat(llm): backend SSOT — managed text via OpenRouter Luna - #11283
Conversation
Make model_config + gateway overrides the product SSOT for managed OpenAI-family text, resolve gpt-5.6-luna via OpenRouter with shared vendor prefixing, and collapse former nano light routes onto Luna. Co-authored-by: Cursor <cursoragent@cursor.com>
File-isolated runs were charging OpenRouter client construction to the first TestGetLlm case and tripping the fast-unit CPU guard on push. Co-authored-by: Cursor <cursoragent@cursor.com>
Route former Gemini-lite and wrapped_analysis features through gpt-5.6-luna on OpenRouter so product text has one backend SSOT, keeping only Perplexity for web_search. Co-authored-by: Cursor <cursoragent@cursor.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_3fa87a44-247e-47e9-82db-e211af657941) |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 76a2ab2391
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| if provider == 'openrouter' and (model.startswith('gpt-') or model.startswith(('o1', 'o3', 'o4'))): | ||
| return 'openai' |
There was a problem hiding this comment.
When gateway feature mode is enabled and a user supplies an OpenAI BYOK key, this remapping causes get_or_create_omi_gateway_llm_for_byok to send only X-Omi-Byok-OpenAI-Key, while the newly generated route still has provider openrouter. The gateway executor checks has_provider_key('openrouter') before invoking the provider, so every migrated text feature fails with MISSING_BYOK_KEY instead of using the user's OpenAI key. Route these vendor-BYOK calls directly to OpenAI or provide a gateway route/provider override that matches the forwarded credential.
AGENTS.md reference: backend/AGENTS.md:L28-L28
Useful? React with 👍 / 👎.
| - route_artifact_id: route.public_shared_conversation_chat.2026_07_19.001 | ||
| artifact_digest: sha256:17fd222062a22973f979ffef0b5cf5ef6c90d569e56c6e8feff4893fc01da2d9 |
There was a problem hiding this comment.
These dated artifact IDs were already referenced as active or last-known-good routes in the parent configuration, but this commit changes their digests and providers in place. That makes the same route_artifact_id identify different content across deployments and removes the previous OpenAI artifacts needed for an operator rollback; it also directly contradicts the immutable-artifact contract in docs/doc/developer/backend/llm_gateway.mdx:490-492. Preserve the existing artifacts, create new IDs for the OpenRouter variants, and update the lane pointers.
Useful? React with 👍 / 👎.
| 'conv_action_items': ('gpt-5.6-luna', 'openrouter'), | ||
| 'conv_structure': ('gpt-5.6-luna', 'openrouter'), |
There was a problem hiding this comment.
When gateway feature mode is off, as in the default local/direct path, the checked-in environment templates supply OPENAI_API_KEY but not OPENROUTER_API_KEY. These newly migrated features therefore construct ChatOpenAI with an OpenRouter base URL but without an explicit key; ChatOpenAI falls back to the ambient OPENAI_API_KEY, causing that OpenAI credential to be sent to OpenRouter and the request to fail authentication. Require OPENROUTER_API_KEY before constructing this client and add it to the backend environment templates and guide.
AGENTS.md reference: AGENTS.md:L33-L33
Useful? React with 👍 / 👎.
| exceptions = { | ||
| 'web_search': 'perplexity', | ||
| 'wrapped_analysis': 'openrouter', | ||
| 'translation': 'gemini', |
There was a problem hiding this comment.
Running this live QoS suite now fails before making an API call because MODEL_QOS_PROFILES['byok']['translation'] is ('gpt-5.6-luna', 'openrouter'), while this exception still requires provider gemini. The same contradiction exists for the other former Gemini features in this mapping, so remove those stale exceptions or change their expected provider to OpenRouter.
Useful? React with 👍 / 👎.
| 'onboarding': ('gpt-5.6-luna', 'openrouter'), | ||
| 'app_integration': ('gpt-5.6-luna', 'openrouter'), | ||
| 'trends': ('gpt-5.6-luna', 'openrouter'), | ||
| 'translation': ('gpt-5.6-luna', 'openrouter'), |
There was a problem hiding this comment.
When the production translation policy falls back from NLLB, GeminiTranslationProvider now obtains an OpenRouter Luna client through get_llm('translation'), but the adapter, profile token, errors, display name, metrics, and record_fallback labels still identify the request as Gemini 2.5 Flash-Lite. As a result, translation fallback and outage telemetry attributes OpenRouter traffic to a retired provider, preventing operators from identifying the failing dependency. Migrate the translation provider enum/adapter/configuration and its tests to the actual managed OpenRouter route.
AGENTS.md reference: AGENTS.md:L88-L88
Useful? React with 👍 / 👎.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
6 issues found across 27 files
Confidence score: 3/5
- In
backend/tests/integration/test_qos_live_cp9.py,test_byok_mostly_openrouterexpects several BYOK routes to hitgeminiwhile current QoS profiles appear to be copied from a non-Gemini template, which risks routing behavior drifting from intended provider policy in production—reconcile the BYOK profile definitions with the test’s exception list (or update expectations if behavior intentionally changed). - In
backend/llm_gateway/config/cost_rate_cards.yaml, theopenrouter.gpt-5.6-lunapricing appears inconsistent with related cards, which could skew cost accounting, budget controls, or model selection logic—verify Luna’s source rates and correct the card to match the intended pricing basis. - In
backend/tests/unit/test_llm_provider_plugin_structure.pyandbackend/tests/unit/test_llm_gateway_config.py, test coverage/naming looks partially out of sync with the model migration, increasing the chance of false confidence or missed regressions—refresh assertions and rename stale tests to reflect the new routing baseline. - In
backend/docs/llm/model_endpoint_inventory.yamlandbackend/tests/integration/test_qos_real_llm.py, format inconsistencies and dead Gemini-key scaffolding are low-severity but can cause confusion for operators and future maintainers—normalizecurrent_provider_modelformatting and remove unusedHAS_GEMINI_KEY/outdated notes.
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="backend/tests/integration/test_qos_live_cp9.py">
<violation number="1" location="backend/tests/integration/test_qos_live_cp9.py:205">
P2: The exception list in test_byok_mostly_openrouter expects translation/session_titles/followup/onboarding/app_integration/trends to route to provider 'gemini', but every QoS profile (including 'byok') is a copy of _TWO_TIER_MODEL_PROFILE, where all of those features resolve to ('gpt-5.6-luna', 'openrouter'). The assert provider == 'gemini' will fail for each, so this test contradicts the SSOT it is meant to validate. The test should assert these former-Gemini features are 'openrouter' (matching the managed-text migration) rather than excluding them as gemini exceptions.</violation>
</file>
<file name="backend/docs/llm/model_endpoint_inventory.yaml">
<violation number="1" location="backend/docs/llm/model_endpoint_inventory.yaml:80">
P3: The current_provider_model format is inconsistent across the changed surfaces: line 80 writes 'openrouter openai/gpt-5.6-luna' (space) while lines 94/136/143 write 'openrouter/openai/gpt-5.6-luna' (slash), and line 94 mixes the lane into the model string ('via omi:auto:chat-agent') that other surfaces keep in gateway_lane_capability_needed. Recommend standardizing the provider/model shorthand so the inventory reads consistently.</violation>
</file>
<file name="backend/tests/integration/test_qos_real_llm.py">
<violation number="1" location="backend/tests/integration/test_qos_real_llm.py:173">
P3: After dropping the GEMINI skip guard, the module-level `HAS_GEMINI_KEY` constant (line 37) has no remaining use and is dead code. Consider removing it along with the now-unneeded GEMINI_API_KEY requirement note in the module docstring, since the former-Gemini features now route via OpenRouter.</violation>
</file>
<file name="backend/tests/unit/test_llm_provider_plugin_structure.py">
<violation number="1" location="backend/tests/unit/test_llm_provider_plugin_structure.py:32">
P2:</violation>
</file>
<file name="backend/tests/unit/test_llm_gateway_config.py">
<violation number="1" location="backend/tests/unit/test_llm_gateway_config.py:34">
P3: The test name `test_gateway_route_overrides_do_not_change_the_legacy_model_profile` is now stale — the whole point of these changes is that conv_discard (previously the gpt-5-nano legacy holdout) is migrated to gpt-5.6-luna, so the test actually asserts the legacy profile change. Rename it (e.g. `test_gateway_route_overrides_migrate_legacy_profile_to_luna` or similar) so the intent matches what it verifies.</violation>
</file>
<file name="backend/llm_gateway/config/cost_rate_cards.yaml">
<violation number="1" location="backend/llm_gateway/config/cost_rate_cards.yaml:33">
P2: The OpenRouter Luna rate looks inconsistent with the OpenRouter nano card and likely wrong: `openrouter.gpt-5-nano` matches direct-OpenAI nano 1:1 (50K/400K), but `openrouter.gpt-5.6-luna` is exactly 5x direct-OpenAI Luna (1M/6M vs 200K/1.2M). A pure routing markup would be ~5%, not 5x, so this rate is suspect and would distort COGS accounting and cost alerts for the primary managed route if it ships unconfirmed.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| exceptions = { | ||
| 'web_search': 'perplexity', | ||
| 'wrapped_analysis': 'openrouter', | ||
| 'translation': 'gemini', |
There was a problem hiding this comment.
P2: The exception list in test_byok_mostly_openrouter expects translation/session_titles/followup/onboarding/app_integration/trends to route to provider 'gemini', but every QoS profile (including 'byok') is a copy of _TWO_TIER_MODEL_PROFILE, where all of those features resolve to ('gpt-5.6-luna', 'openrouter'). The assert provider == 'gemini' will fail for each, so this test contradicts the SSOT it is meant to validate. The test should assert these former-Gemini features are 'openrouter' (matching the managed-text migration) rather than excluding them as gemini exceptions.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At backend/tests/integration/test_qos_live_cp9.py, line 205:
<comment>The exception list in test_byok_mostly_openrouter expects translation/session_titles/followup/onboarding/app_integration/trends to route to provider 'gemini', but every QoS profile (including 'byok') is a copy of _TWO_TIER_MODEL_PROFILE, where all of those features resolve to ('gpt-5.6-luna', 'openrouter'). The assert provider == 'gemini' will fail for each, so this test contradicts the SSOT it is meant to validate. The test should assert these former-Gemini features are 'openrouter' (matching the managed-text migration) rather than excluding them as gemini exceptions.</comment>
<file context>
@@ -188,14 +197,23 @@ def test_byok_profile_independent_of_active(self):
+ exceptions = {
+ 'web_search': 'perplexity',
+ 'wrapped_analysis': 'openrouter',
+ 'translation': 'gemini',
+ 'session_titles': 'gemini',
+ 'followup': 'gemini',
</file context>
|
|
||
| # Keep the real database package importable for sibling tests. | ||
| if 'database' not in sys.modules or isinstance(sys.modules.get('database'), MagicMock): | ||
| _database_pkg = types.ModuleType('database') |
There was a problem hiding this comment.
P2:
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At backend/tests/unit/test_llm_provider_plugin_structure.py, line 32:
<comment></comment>
<file context>
@@ -15,23 +23,79 @@
+
+# Keep the real database package importable for sibling tests.
+if 'database' not in sys.modules or isinstance(sys.modules.get('database'), MagicMock):
+ _database_pkg = types.ModuleType('database')
+ _database_pkg.__path__ = [str(BACKEND_DIR / 'database')]
+ sys.modules['database'] = _database_pkg
</file context>
| - rate_card_id: openrouter.openai.gpt-5.6-luna.2026-08-09 | ||
| provider: openrouter | ||
| model: openai/gpt-5.6-luna | ||
| input_micro_usd_per_million: 1000000 |
There was a problem hiding this comment.
P2: The OpenRouter Luna rate looks inconsistent with the OpenRouter nano card and likely wrong: openrouter.gpt-5-nano matches direct-OpenAI nano 1:1 (50K/400K), but openrouter.gpt-5.6-luna is exactly 5x direct-OpenAI Luna (1M/6M vs 200K/1.2M). A pure routing markup would be ~5%, not 5x, so this rate is suspect and would distort COGS accounting and cost alerts for the primary managed route if it ships unconfirmed.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At backend/llm_gateway/config/cost_rate_cards.yaml, line 33:
<comment>The OpenRouter Luna rate looks inconsistent with the OpenRouter nano card and likely wrong: `openrouter.gpt-5-nano` matches direct-OpenAI nano 1:1 (50K/400K), but `openrouter.gpt-5.6-luna` is exactly 5x direct-OpenAI Luna (1M/6M vs 200K/1.2M). A pure routing markup would be ~5%, not 5x, so this rate is suspect and would distort COGS accounting and cost alerts for the primary managed route if it ships unconfirmed.</comment>
<file context>
@@ -27,6 +27,20 @@ rate_cards:
+ - rate_card_id: openrouter.openai.gpt-5.6-luna.2026-08-09
+ provider: openrouter
+ model: openai/gpt-5.6-luna
+ input_micro_usd_per_million: 1000000
+ cached_input_micro_usd_per_million: 100000
+ output_micro_usd_per_million: 6000000
</file context>
| - surface: public_shared_conversation_chat | ||
| code_path: backend/routers/public_shared_conversation_chat.py:public_shared_conversation_chat | ||
| current_provider_model: gateway-only openai/gpt-5-nano | ||
| current_provider_model: gateway-only openrouter openai/gpt-5.6-luna |
There was a problem hiding this comment.
P3: The current_provider_model format is inconsistent across the changed surfaces: line 80 writes 'openrouter openai/gpt-5.6-luna' (space) while lines 94/136/143 write 'openrouter/openai/gpt-5.6-luna' (slash), and line 94 mixes the lane into the model string ('via omi:auto:chat-agent') that other surfaces keep in gateway_lane_capability_needed. Recommend standardizing the provider/model shorthand so the inventory reads consistently.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At backend/docs/llm/model_endpoint_inventory.yaml, line 80:
<comment>The current_provider_model format is inconsistent across the changed surfaces: line 80 writes 'openrouter openai/gpt-5.6-luna' (space) while lines 94/136/143 write 'openrouter/openai/gpt-5.6-luna' (slash), and line 94 mixes the lane into the model string ('via omi:auto:chat-agent') that other surfaces keep in gateway_lane_capability_needed. Recommend standardizing the provider/model shorthand so the inventory reads consistently.</comment>
<file context>
@@ -70,14 +70,14 @@ model_config_features:
- surface: public_shared_conversation_chat
code_path: backend/routers/public_shared_conversation_chat.py:public_shared_conversation_chat
- current_provider_model: gateway-only openai/gpt-5-nano
+ current_provider_model: gateway-only openrouter openai/gpt-5.6-luna
request_shape: non-streaming plain chat with server-resolved shared transcript, at most eight user/assistant history entries, and no tools or retrieval
gateway_lane_capability_needed: dedicated non-streaming no-tools omi:auto:public-shared-conversation-chat lane
</file context>
| current_provider_model: gateway-only openrouter openai/gpt-5.6-luna | |
| current_provider_model: gateway-only openrouter/openai/gpt-5.6-luna |
| 'trends', | ||
| ] | ||
|
|
||
| @pytest.mark.skipif(not HAS_GEMINI_KEY, reason="GEMINI_API_KEY not set") |
There was a problem hiding this comment.
P3: After dropping the GEMINI skip guard, the module-level HAS_GEMINI_KEY constant (line 37) has no remaining use and is dead code. Consider removing it along with the now-unneeded GEMINI_API_KEY requirement note in the module docstring, since the former-Gemini features now route via OpenRouter.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At backend/tests/integration/test_qos_real_llm.py, line 173:
<comment>After dropping the GEMINI skip guard, the module-level `HAS_GEMINI_KEY` constant (line 37) has no remaining use and is dead code. Consider removing it along with the now-unneeded GEMINI_API_KEY requirement note in the module docstring, since the former-Gemini features now route via OpenRouter.</comment>
<file context>
@@ -157,63 +155,59 @@ def test_openglass_feature_responds(self):
- def test_gemini_feature_responds(self, feature):
+ @pytest.mark.parametrize("feature", FORMER_GEMINI_FEATURES)
+ def test_former_gemini_feature_responds(self, feature):
model = get_model(feature)
- assert model == 'gemini-2.5-flash-lite', f"{feature} should be gemini-2.5-flash-lite in premium, got {model}"
+ assert model == 'gpt-5.6-luna', f"{feature} should be gpt-5.6-luna in premium, got {model}"
</file context>
| config = load_gateway_config(prod_mode=True) | ||
|
|
||
| assert get_model('conv_discard') == 'gpt-5-nano' | ||
| assert get_model('conv_discard') == 'gpt-5.6-luna' |
There was a problem hiding this comment.
P3: The test name test_gateway_route_overrides_do_not_change_the_legacy_model_profile is now stale — the whole point of these changes is that conv_discard (previously the gpt-5-nano legacy holdout) is migrated to gpt-5.6-luna, so the test actually asserts the legacy profile change. Rename it (e.g. test_gateway_route_overrides_migrate_legacy_profile_to_luna or similar) so the intent matches what it verifies.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At backend/tests/unit/test_llm_gateway_config.py, line 34:
<comment>The test name `test_gateway_route_overrides_do_not_change_the_legacy_model_profile` is now stale — the whole point of these changes is that conv_discard (previously the gpt-5-nano legacy holdout) is migrated to gpt-5.6-luna, so the test actually asserts the legacy profile change. Rename it (e.g. `test_gateway_route_overrides_migrate_legacy_profile_to_luna` or similar) so the intent matches what it verifies.</comment>
<file context>
@@ -24,26 +24,25 @@ def test_loads_default_gateway_config():
config = load_gateway_config(prod_mode=True)
- assert get_model('conv_discard') == 'gpt-5-nano'
+ assert get_model('conv_discard') == 'gpt-5.6-luna'
assert get_model('memories') == 'gpt-5.6-luna'
assert get_model('fair_use') == 'gpt-5.6-luna'
</file context>
Git-on-my-level
added
needs-tests
Git-on-my-level
left a comment
Git-on-my-level
left a comment
There was a problem hiding this comment.
Thanks for the careful consolidation here — the SSOT direction is coherent, and I like that the OpenRouter model-name prefixing is being centralized instead of duplicated.
I found one blocking issue before this can merge:
backend/utils/llm/clients.pynow intentionally remaps OpenRouter-hosted OpenAI-family BYOK traffic to the user's OpenAI key in gateway mode (provider=_effective_byok_provider(model, provider), soopenrouter+gpt-5.6-lunabecomesopenai).backend/utils/llm/gateway_byok.pythen forwards that key asX-Omi-Byok-openai-Key, which matches the stated product behavior for OpenAI-family BYOK remapping.- But the gateway executor still checks the route provider literally:
backend/llm_gateway/gateway/executor.pycallscredential_context.has_provider_key(provider_ref.provider), and the generated route provider is nowopenrouterfor these lanes (backend/llm_gateway/config/generated_route_overrides.yaml/backend/llm_gateway/config/route_artifacts.yaml). With only the forwarded OpenAI BYOK key present, the gateway will see noopenrouterkey and fail closed asmissing_byok_keybefore the provider call.backend/llm_gateway/gateway/credentials.pyalso has no effective-provider/remap layer for this check. - The tests do not cover that production path:
backend/tests/unit/test_llm_gateway_client_config.pyasserts that the backend sends provideropenaiinto the gateway BYOK helper, whilebackend/tests/unit/test_llm_gateway_openai_compatible.pyswitched the direct gateway API tests toX-Omi-Byok-OpenRouter-Key. Neither test exercises an OpenRouter route artifact receiving only an OpenAI forwarded BYOK key, so this regression can stay green.
Please add/fix the gateway-side BYOK effective-provider mapping (or otherwise make the route credential check accept the intended OpenAI key for OpenRouter OpenAI-family models) and add a focused gateway executor/openai-compatible test for an openrouter + openai/gpt-5.6-luna route with only X-Omi-Byok-OpenAI-Key present.
Other notes from the sampled files:
backend/utils/llm/openrouter_model_names.pyandbackend/utils/llm/providers.pylook like the right shared boundary for vendor-prefixing OpenRouter model IDs.backend/llm_gateway/routers/health.pycorrectly moves readiness toOPENROUTER_API_KEYwhen managed chat lanes are OpenRouter-backed.backend/llm_gateway/gateway/config_loader.pynow uses the shared prefix helper, keeping generated route artifacts and runtime provider requests aligned.backend/llm_gateway/gateway/executor.pycorrectly preserves GPT-5.6 cache fields for vendor-prefixedopenai/gpt-5.6-luna, but this is also where the BYOK provider-key check needs to stay consistent with the remap.backend/tests/unit/test_openrouter_model_names.pycovers the new helper, but the missing test is the cross-boundary BYOK/gateway credential path above.
Because this is model/provider routing and BYOK behavior, this still needs maintainer sign-off after the blocking credential-path fix.
Automated maintainer review. I may be wrong; please verify the concern against the code and tests.
by AI on behalf of David — if you need David’s attention urgently, please @Git-on-my-level and escalate with need human response.
Resolutions: - generated_route_overrides.yaml chat_agent: keep this branch's openrouter provider and main's reasoning_effort=none (#11285). The effort fix follows the model, not the provider, so it must survive the repoint. The direct-OpenAI prompt_cache_retention extra_body drops out with the provider, as this branch intends. - executor._sanitize_openai_chat_completions_request: main gated the tools-effort and temperature 400 guards on provider == 'openai'. The same upstream model is now reached through OpenRouter's openai/ namespace, so the guard follows the model and both live 400s stay covered; tests updated to the namespaced model id. - Two failures that predate the merge but only surface once managed text is on OpenRouter, ported from the stacked branch so this PR is green on its own: the session-titles output budget was provider-gated on gemini and vanished on repoint, and the product-capability synthetic plus the fake-upstream replay oracle registered their fake provider under a hardcoded "openai" key. Verified: backend/.venv/bin/python -m pytest on the llm_gateway config/executor/ openai-compatible/accounting/resolver/coverage/route-refs/readiness/output-budget, qos-tiers, openrouter-model-names, product-capability-synthetics, replay-oracle, process-conversation-usage-context, memories-create, rate-limiting and desktop-chat suites — 0 failures. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_0d8be48d-6a13-4e9b-8471-0848bb1bbdff) |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 77671a2e9f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| 'trends': ('gpt-5.6-luna', 'openrouter'), | ||
| 'translation': ('gpt-5.6-luna', 'openrouter'), | ||
| 'chat_agent': ('gpt-5.6-luna', 'openrouter'), | ||
| 'wrapped_analysis': ('gpt-5.6-luna', 'openrouter'), |
There was a problem hiding this comment.
When gateway feature mode is disabled and an OPENROUTER_API_KEY is configured, get_llm('wrapped_analysis') now builds the OpenRouter Luna client with the existing _OPENROUTER_TEMPERATURES['wrapped_analysis'] == 0.7. The gateway executor explicitly strips this value because GPT-5.6 accepts only the default temperature, but the direct client path has no equivalent sanitizer, so wrapped analysis receives an upstream 400 instead of a completion. Remove this stale Gemini-era option or sanitize it in the shared direct-provider construction path.
Useful? React with 👍 / 👎.
The pre-commit hook reformatted the app/ files this merge only carried across from main, using a different Dart SDK than CI's Flutter-bundled formatter — so the merge introduced formatting-only drift and the Formatting check failed on files this branch does not own. Restored to main's committed content. Verified: git diff origin/main -- app/ is empty for these files. Failure-Class: none Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…'s key The backend remaps OpenRouter-hosted OpenAI-family BYOK traffic to the user's OpenAI key and forwards X-Omi-Byok-OpenAI-Key (utils/llm/clients.py _effective_byok_provider), but the gateway executor checked the route's literal provider — now 'openrouter' for these lanes — so every BYOK request failed closed as missing_byok_key before the provider call, on a key the user did supply. The executor now follows the key: a BYOK request on an openrouter route whose model carries a vendor prefix is served by that vendor's provider with the prefix dropped, when the caller supplied that vendor's key. Managed (omi_paid) traffic is untouched and still uses Omi's OpenRouter account, and a BYOK caller without the matching vendor key still fails closed. The vendor mapping lives beside the existing prefixing helper, so both sides of the boundary read from one place. Verified: backend/.venv/bin/python -m pytest tests/unit/test_llm_gateway_openai_compatible.py tests/unit/test_llm_gateway_executor.py tests/unit/test_openrouter_model_names.py tests/unit/test_llm_gateway_client_config.py — 107 passed, including a new openrouter-route + OpenAI-only-key success case and its fail-closed counterpart; plus the config/accounting/resolver/coverage/readiness/route-refs/output-budget/synthetics/ replay-oracle/qos suites — 190 passed. Failure-Class: none Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_1f29f6d8-7ca4-4d42-a85a-418647ce2f22) |
|
Confirmed and fixed in 79fca60.
Two tests in Also in this head: the merge with |
…11284) ## Summary - Add `OpenRouterModelCatalog` that fetches/caches OpenRouter `GET /api/v1/models` for `context_length`, `max_completion_tokens`, and `supported_parameters`. - Clamp gateway provider requests for OpenRouter routes against live completion ceilings (rs_ai-inspired thin registry; OpenRouter is the numeric metadata source because rs_ai does not expose context windows). - Stacked on #11283 (`backend-ssot-openrouter`). ## Test plan - [x] `tests/unit/test_openrouter_model_catalog.py` via `test.sh` - [x] `tests/unit/test_llm_gateway_executor.py` still green - [ ] Optional live check: with `OPENROUTER_API_KEY`, confirm `openai/gpt-5.6-luna` resolves with non-null context/max completion Made with [Cursor](https://cursor.com) ## Failure class (fixes) Failure-Class: none The `fix:` commits here come from the merged parent branch; this PR adds the dynamic OpenRouter model catalog and its clamp. <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/BasedHardware/omi/pull/11284?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. -->
…sts (#11360) ## Summary Reverts the two OpenRouter PRs: #11283 (managed product text → OpenRouter Luna) and #11284 (dynamic OpenRouter model catalog). **No `OPENROUTER_API_KEY` is provisioned.** As merged, every managed product-text feature — chat, memories, knowledge graph, conversation processing, goals, notifications, wrapped — resolves to an OpenRouter route and would fail at the provider, and `llm_gateway/routers/health.py` requires that key before reporting ready, so the gateway would never pass readiness. Routing config returns exactly to its pre-#11283 state (`git diff` against the commit before that merge is empty for `model_config.py`, `llm_gateway/config/`, `health.py` and `clients.py`). ## Kept The SSOT work from #11286 and #11325 stays: `/v1/knowledge-graph/extract`, `/v1/memories/extract`, `/v1/connectors/synthesize`, `/v1/conversations/topic`, `/v1/users/ai-profile/synthesize` and deterministic KG ids. Those route through `get_llm(feature)`, so they follow whatever provider `model_config` names — now direct OpenAI/Anthropic again — and keep working. ## Product invariants affected - INV-AGENT-* - INV-CHAT-1 - INV-MEM-1 ## Failure class (fixes) Failure-Class: none ## Test plan - [x] `backend/test.sh` over the 41 gateway/qos/openrouter/SSOT-endpoint test files — all pass file-by-file (one pre-existing fast-unit CPU-time guard trip on `test_llm_gateway_deploy_contract.py`, 13/13 assertions pass, unrelated to this diff) - [x] Routing config byte-identical to pre-#11283 for `model_config.py`, `llm_gateway/config/`, `health.py`, `clients.py` - [ ] Dev backend deploy reports ready without `OPENROUTER_API_KEY` <!-- This is an auto-generated description by cubic. --> <a href="https://cubic.dev/pr/BasedHardware/omi/pull/11360?utm_source=github" target="_blank" rel="noopener noreferrer" data-no-image-dialog="true"><picture><source media="(prefers-color-scheme: dark)" srcset="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source media="(prefers-color-scheme: light)" srcset="https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img alt="Review in cubic" src="https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a> <!-- End of auto-generated description by cubic. --> <!-- CURSOR_SUMMARY --> --- > [!NOTE] > **High Risk** > Touches core LLM routing, gateway provider execution, readiness, and BYOK behavior for all managed text features; wrong config would break chat and background LLM workloads at scale. > > **Overview** > **Reverts managed product text routing from OpenRouter back to direct providers** so the stack can run without `OPENROUTER_API_KEY` and gateway `/ready` no longer blocks on that credential. > > `model_config` and gateway **generated route overrides** again send most features to **direct OpenAI** (`gpt-5.6-luna` / `gpt-5-nano`), **Gemini** for former flash-lite workloads, **Anthropic** for `chat_agent`, and **OpenRouter** only for `wrapped_analysis`. Inventory, route artifacts, and cost cards are aligned with that map; OpenRouter Luna/nano rate cards are removed. > > The gateway **executor** drops OpenRouter-specific request shaping: no BYOK vendor remapping on OpenRouter routes, no OpenRouter completion clamp, and GPT-5.6 sanitization applies only to `openai` provider refs. **Health** reports `managed_chat_provider: openai` and requires `OPENAI_API_KEY` instead of OpenRouter. > > **Deleted** the dynamic OpenRouter model catalog and shared vendor-prefix helpers; synthetics/replay harnesses register **`openai`** fakes directly again. QoS and gateway unit/integration tests are updated to match the pre–OpenRouter managed-text expectations. > > <sup>Reviewed by [Cursor Bugbot](https://cursor.com/bugbot) for commit 3b0aa79. Configure [here](https://www.cursor.com/dashboard/bugbot).</sup> <!-- /CURSOR_SUMMARY -->
…asedHardware#11283)" This reverts merge commit d7858c8 (PR BasedHardware#11283) via -m 1. No OPENROUTER_API_KEY is provisioned, so every managed product text feature this repointed — chat, memories, knowledge graph, conversation processing, goals, notifications and the rest — would fail at the provider, and the gateway readiness probe now requires that key before reporting ready. Managed text returns to its previous direct-provider routing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
![https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source](https://www.cubic.dev/buttons/review-in-cubic-dark.svg"><source){kind=link}
![https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img](https://www.cubic.dev/buttons/review-in-cubic-light.svg"><img){kind=link}
![https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a](https://www.cubic.dev/buttons/review-in-cubic-dark.svg"></picture></a){kind=link}
2 participants
Summary
model_config+ gateway overrides the product SSOT for managed text routing, withchat_agentand all former OpenAI/Gemini-lite/wrapped_analysisfeatures resolving togpt-5.6-lunaon OpenRouter (openai/gpt-5.6-luna).openai//google/), gateprompt_cache_retentionto direct OpenAI, and remap OpenAI-family BYOK on OpenRouter routes to the user's OpenAI key.web_search; update cost cards, inventory, readiness, and QoS/gateway unit tests.Test plan
BACKEND_UNIT_TEST_FILE_LIST=… bash test.shfor QoS + gateway unit files (including fast-unit CPU guard)OPENROUTER_API_KEYomi:auto:chat-agentMade with Cursor
Failure class (fixes)
Failure-Class: none
The only
fix:commit here restores main's Dart formatting on files this merge merelycarried across; it repairs no product contract.