fix(llm): pass LLM_EXTRA_HEADERS through ModelSettings so they reach the agent loop - #937
Merged
Merged
Conversation
Contributor
Author
🤖 Devin AI EngineerI'll be helping with this pull request! Here's what you should know: ✅ I will automatically:
Note: I can only respond to comments from users who have write access to this repository. ⚙️ Control Options:
|
Contributor
Greptile SummaryThis follow-up completes propagation and isolation of configured LLM headers across model-backed call paths.
Confidence Score: 5/5The PR appears safe to merge. The previously reported cross-provider header leak is fixed: configured dedicated dedupe models use only their own headers, while main headers are inherited solely when dedupe falls back to the main model, and no blocking failure remains. Important Files Changed
Reviews (4): Last reviewed commit: "feat(llm): DEDUPE_LLM_EXTRA_HEADERS for ..." | Re-trigger Greptile |
Contributor
Author
Contributor
Author
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Follow-up to #936:
LLM_EXTRA_HEADERSworked for directlitellm.acompletioncalls but was silently dropped inside the actual agent loop on the LiteLLM route. The Agents SDK'sLitellmModelalways passes its own mergedextra_headers=kwarg, and LiteLLM's fallback isheaders = headers or litellm.headers— since the SDK's kwarg is always non-empty (it includes the SDKUser-Agent), the globallitellm.headersset by_configure_extra_headersis never consulted. Reproduced by runningRunner.run_streamedagainst a header-enforcing mock gateway: thelitellm/...route got 401 while direct litellm calls passed.Fix: inject the headers at the SDK-supported point,
ModelSettings.extra_headers, which all three model classes (LitellmModel,OpenAIChatCompletionsModel,OpenAIResponsesModel) merge into every request (streaming and not).Unified call sites
All four LLM call sites now go through the same path —
StrixProvider().get_model()+make_model_settings()— so every configured knob (headers, per-request timeout, retry policy) applies everywhere automatically:New setting:
DEDUPE_LLM_EXTRA_HEADERS(JSON object) — custom headers for a dedicated dedupe model's endpoint, alongside the existingDEDUPE_LLM_API_KEY/DEDUPE_LLM_API_BASE. Documented indocs/advanced/configuration.mdx.The compaction rewrite also fixes a real pre-existing bug found while testing: the summarizer passed the raw Strix model string straight to
litellm.acompletion, so routing-prefixed models (litellm/...,any-llm/...,ollama/...) always failed the summary call (LiteLLM "provider not provided"), silently disabling compaction. Routing now matches the agent loop exactly, including nativeopenai/...and codex-subscription models.Also fixes a pre-existing mypy error in
interface/tui/app.py(tt = tt.parentnarrowing) that the pre-commit mypy hook surfaces on any commit touchinginterface/main.py.The global
litellm.headersmerge and default OpenAI client registration from #936 are kept as defense in depth.Verified end-to-end against a local mock OpenAI-compatible gateway that 401s when the headers are missing: all four call sites (agent loop with tool round-trip, warm-up, dedupe, compaction summary) send the headers on both the
litellm/...and nativeopenai/...routes — 8/8 pass; pre-patch the litellm agent-loop route failed with 401 and the litellm compaction call failed to route. Full test suite green (one pre-existing order-dependent flake intest_execution.pyfails identically on the base branch).Link to Devin session: https://app.devin.ai/sessions/54414ea3ecb34ade9189534e5c690969
Requested by: @0xallam