[MISC] Fix zero cost tracking for OpenAI-compatible adapter#1985
Merged
Conversation
The OpenAI-compatible adapter prefixes the model with `custom_openai/`, which has no entry in LiteLLM's price map, so `cost_per_token` fails and usage is recorded with cost 0. Mirror the Azure adapter's `cost_model` approach: when the endpoint is OpenAI's own API, set `cost_model` to the bare model name so pricing resolves. Other gateways serve same-named models at different prices, so their cost is intentionally left unresolved rather than guessed. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
Contributor
WalkthroughAdds URL parsing and an endpoint-detection helper to identify the official OpenAI API host, and updates OpenAICompatibleLLMParameters.validate() to populate cost_model (by removing a leading ChangesOpenAI endpoint conditional cost model assignment
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
|
| Filename | Overview |
|---|---|
| unstract/sdk1/src/unstract/sdk1/adapters/base1.py | Adds _is_openai_api_endpoint() helper (HTTPS + hostname check) and a cost_model injection block in OpenAICompatibleLLMParameters.validate() — logic is correct, safe, and idempotent. |
| unstract/sdk1/tests/test_openai_compatible_adapter.py | Four new test cases added covering OpenAI endpoint cost-model resolution, sub-prefix preservation, non-OpenAI gateway (no cost_model), and re-validation idempotency — all previously raised concerns are now covered. |
Flowchart
%%{init: {'theme': 'neutral'}}%%
flowchart TD
A["validate(adapter_metadata)"] --> B["validate_model()\nmodel = custom_openai/<name>"]
B --> C["Pydantic model_dump()\nvalidated dict (no cost_model)"]
C --> D{"_is_openai_api_endpoint\n(api_base)?"}
D -- "Yes\nscheme==https AND\nhostname==api.openai.com" --> E["validated['cost_model'] =\nmodel[len('custom_openai/'):]"]
D -- "No\nother gateway / None" --> F["cost_model unset\n(stays 0 — honest)"]
E --> G["return validated"]
F --> G
G --> H["LLM.__init__ pops cost_model\nfor pricing lookup only"]
H --> I["litellm.cost_per_token\n(model=cost_model)"]
Reviews (2): Last reviewed commit: "[MISC] Address review: require HTTPS sch..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
- _is_openai_api_endpoint now requires the https scheme so a plain-http host cannot trigger cost-model resolution. - Add a test asserting cost_model survives validate() re-validation. Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
|
Contributor
Test ResultsSummary
Runner Tests - Full Report
SDK1 Tests - Full Report
|
vishnuszipstack
approved these changes
May 22, 2026
Deepak-Kesavan
approved these changes
May 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
LLM usage recorded by the OpenAI-compatible (
custom_openai) adapter always hascost_in_dollars = 0.Why
validate_modelprefixes the model withcustom_openai/._record_usageprices vialitellm.cost_per_token(model="custom_openai/..."), butcustom_openaiis a generic passthrough provider with no entry in LiteLLM's price map — the lookup fails and cost falls back to0.How
Mirror the Azure adapter's existing
cost_modelmechanism (model= routing identity,cost_model= pricing identity):api_basepoints at OpenAI's own HTTPS API host, setcost_modelto the bare model name (prefix stripped). LiteLLM's price map then resolves real OpenAI rates.cost_modelis left unset. Non-OpenAI gateways (vLLM, self-hosted, third-party resellers) serve same-named models at different — or no — market prices, so guessing OpenAI rates would be confidently wrong. Cost stays0(honest "unresolved"), unchanged from today.cost_modelis popped inLLM.__init__and never reacheslitellm.completion(), so routing is unaffected.Can this PR break any existing features. If yes, please list possible items. If no, please explain why. (PS: Admins do not merge the PR without this section filled)
No. The change only adds a
cost_modelkey on the OpenAI-endpoint path;cost_modelis consumed solely by cost lookup and never sent tolitellm.completion(). All other endpoints behave exactly as before (cost0). Routing, request params, and reasoning handling are untouched.Related Issues or PRs
Follow-up to #1983 (OpenAI gpt-5 / o-series support in the openai-compatible adapter).
Notes on Testing
4 new cases in
test_openai_compatible_adapter.py:cost_modelset for the OpenAI endpoint,openai/sub-prefix preserved,cost_modelabsent for other gateways, andcost_modelstable acrossvalidate()re-validation. Full suite: 22 tests pass.Screenshots
Checklist
I have read and understood the Contribution Guidelines.
🤖 Generated with Claude Code