You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
See #6677. LitellmProvider._query returns only {"response": ...} and discards the usage block, so a workflow cannot tell how many tokens an analysis consumed or what it cost. The only way to get that today is to skip the provider and call the API from a python step.
Fix
Return the usage fields next to the response:
prompt_tokens, completion_tokens, total_tokens, cost — read through .get() so a backend that reports no usage yields None rather than raising;
model as reported by the API, which can differ from the requested one when a proxy routes the call.
response keeps its meaning, so existing workflows are unaffected.
Tests
Two added to tests/providers/litellm_provider/test_litellm_response_parsing.py: usage and model surfaced from a normal response, and a response without a usage block leaving the fields None while the text still comes through. 9 tests in the file, all passing; black clean.
The unit-tests failure is unrelated to this change: the run reports 1078 passed, 84 skipped, 1 error, and the error is at setup of tests/test_auth.py::test_deleted_api_key_authentication[MULTI_TENANT], with HTTPSConnection(host='https', port=443) — a real network call. This PR only touches the return value of LitellmProvider._query; its two new tests are among the 1078 that passed.
The cause looks reproducible rather than random, if it helps:
tests/fixtures/client.py sets AUTH0_DOMAIN to https://auth0domain.com — with the scheme included — for the MULTI_TENANT string parameter.
_mock_oidc_discovery() is only applied when is_auth0 is true, which is set from request.param.get("AUTH_TYPE") == "AUTH0" and therefore only for the dict-style parameters. The MULTI_TENANT string path gets nullcontext().
ee/identitymanager/identity_managers/auth0/auth0_authverifier.py then runs at import time and builds the JWKS URL as f"https://{auth_domain}/.well-known/jwks.json", which with the scheme already present becomes https://https://auth0domain.com/.... urllib3 parses the host out of that as https, hence the connection attempt in the traceback.
So whenever that module is imported unmocked in this parameterisation, the setup depends on an outbound HTTPS request, which is why it surfaces intermittently across workers. Applying the discovery mock for MULTI_TENANT as well, or storing AUTH0_DOMAIN without the scheme, would remove the network dependency.
Happy to send that as a separate PR if it is useful. Could you re-run the job in the meantime? I cannot trigger it from a fork.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
EnhancementNew feature or requestlgtmThis PR has been approved by a maintainerProviderProviders related issuessize:XSThis PR changes 0-9 lines, ignoring generated files.
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
See #6677.
LitellmProvider._queryreturns only{"response": ...}and discards theusageblock, so a workflow cannot tell how many tokens an analysis consumed or what it cost. The only way to get that today is to skip the provider and call the API from apythonstep.Fix
Return the usage fields next to the response:
prompt_tokens,completion_tokens,total_tokens,cost— read through.get()so a backend that reports no usage yieldsNonerather than raising;modelas reported by the API, which can differ from the requested one when a proxy routes the call.responsekeeps its meaning, so existing workflows are unaffected.Tests
Two added to
tests/providers/litellm_provider/test_litellm_response_parsing.py: usage and model surfaced from a normal response, and a response without ausageblock leaving the fieldsNonewhile the text still comes through. 9 tests in the file, all passing;blackclean.Fixes #6677