Add OpenRouter pricing fallback for token costs#4886
Conversation
Agent Task Evaluation Results: 2/2 (100%)View detailed results
Check the evaluate-tasks job for detailed task execution logs. |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: db39960431
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| return llm | ||
|
|
||
| self.registered_llms[instance_id] = llm | ||
| self._pricing_model_names[llm.model] = self._get_pricing_model_name(llm) |
There was a problem hiding this comment.
register_llm stores _pricing_model_names under llm.model only, and calculate_cost later looks up overrides by the same plain model string. If both an OpenRouter client and a non-OpenRouter client use the same model id (for example openai/gpt-4o-mini), the second registration overwrites the first and all cost calculations for that model id use the wrong pricing source. This silently misprices usage in mixed-provider setups and makes results depend on registration order.
Useful? React with 👍 / 👎.
![cubic-dev-ai[bot]](https://avatars.githubusercontent.com/in/1082092?s=60&v=4){kind=small}
There was a problem hiding this comment.
1 issue found across 3 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="browser_use/tokens/service.py">
<violation number="1" location="browser_use/tokens/service.py:351">
P1: Pricing-source mapping is keyed only by model name, so same-named models from different providers overwrite each other and can produce incorrect cost calculations.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Fix all with cubic | Re-trigger cubic
| return llm | ||
|
|
||
| self.registered_llms[instance_id] = llm | ||
| self._pricing_model_names[llm.model] = self._get_pricing_model_name(llm) |
There was a problem hiding this comment.
P1: Pricing-source mapping is keyed only by model name, so same-named models from different providers overwrite each other and can produce incorrect cost calculations.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At browser_use/tokens/service.py, line 351:
<comment>Pricing-source mapping is keyed only by model name, so same-named models from different providers overwrite each other and can produce incorrect cost calculations.</comment>
<file context>
@@ -340,6 +348,7 @@ def register_llm(self, llm: BaseChatModel) -> BaseChatModel:
return llm
self.registered_llms[instance_id] = llm
+ self._pricing_model_names[llm.model] = self._get_pricing_model_name(llm)
# Store the original method
</file context>
1 participant
Summary
Tests
Summary by cubic
Adds a fallback for token pricing by fetching OpenRouter
/api/v1/modelsand converting prices to ourModelPricing. Also forces OpenRouter pricing forChatOpenRouterand OpenAI-compatible clients using the OpenRouter base URL to avoid mismatched rates.ModelPricing.openrouter/...andopenrouter-...model IDs; prefixesopenrouter/to disambiguate same-named upstream models when needed.Written for commit db39960. Summary will update on new commits. Review in cubic