Skip to content

fix(llm): pass LLM_EXTRA_HEADERS through ModelSettings so they reach the agent loop - #937

Merged
0xallam merged 6 commits into
mainfrom
devin/1785374997-model-settings-extra-headers
Jul 30, 2026
Merged

fix(llm): pass LLM_EXTRA_HEADERS through ModelSettings so they reach the agent loop#937
0xallam merged 6 commits into
mainfrom
devin/1785374997-model-settings-extra-headers

Conversation

@devin-ai-integration

@devin-ai-integration devin-ai-integration Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Summary

Follow-up to #936: LLM_EXTRA_HEADERS worked for direct litellm.acompletion calls but was silently dropped inside the actual agent loop on the LiteLLM route. The Agents SDK's LitellmModel always passes its own merged extra_headers= kwarg, and LiteLLM's fallback is headers = headers or litellm.headers — since the SDK's kwarg is always non-empty (it includes the SDK User-Agent), the global litellm.headers set by _configure_extra_headers is never consulted. Reproduced by running Runner.run_streamed against a header-enforcing mock gateway: the litellm/... route got 401 while direct litellm calls passed.

Fix: inject the headers at the SDK-supported point, ModelSettings.extra_headers, which all three model classes (LitellmModel, OpenAIChatCompletionsModel, OpenAIResponsesModel) merge into every request (streaming and not).

Unified call sites

All four LLM call sites now go through the same path — StrixProvider().get_model() + make_model_settings() — so every configured knob (headers, per-request timeout, retry policy) applies everywhere automatically:

def make_model_settings(..., extra_headers: dict[str, str] | None = None) -> ModelSettings: ...

# core/runner.py (scan agent loop): extra_headers=settings.llm.extra_headers
# report/dedupe.py: extra_headers=dedupe.extra_headers if dedupe.model else llm.extra_headers
#   (a dedicated dedupe model may route to another provider and must never
#    receive the main endpoint's credentials — it gets its own
#    DEDUPE_LLM_EXTRA_HEADERS instead)
# interface/main.py warm-up (main + dedupe): make_model_settings(...) instead of bare ModelSettings()
# llm/compaction.py summarizer: was a raw litellm.acompletion — now
#   StrixProvider().get_model(model).get_response(...) with make_model_settings

New setting: DEDUPE_LLM_EXTRA_HEADERS (JSON object) — custom headers for a dedicated dedupe model's endpoint, alongside the existing DEDUPE_LLM_API_KEY/DEDUPE_LLM_API_BASE. Documented in docs/advanced/configuration.mdx.

The compaction rewrite also fixes a real pre-existing bug found while testing: the summarizer passed the raw Strix model string straight to litellm.acompletion, so routing-prefixed models (litellm/..., any-llm/..., ollama/...) always failed the summary call (LiteLLM "provider not provided"), silently disabling compaction. Routing now matches the agent loop exactly, including native openai/... and codex-subscription models.

Also fixes a pre-existing mypy error in interface/tui/app.py (tt = tt.parent narrowing) that the pre-commit mypy hook surfaces on any commit touching interface/main.py.

The global litellm.headers merge and default OpenAI client registration from #936 are kept as defense in depth.

Verified end-to-end against a local mock OpenAI-compatible gateway that 401s when the headers are missing: all four call sites (agent loop with tool round-trip, warm-up, dedupe, compaction summary) send the headers on both the litellm/... and native openai/... routes — 8/8 pass; pre-patch the litellm agent-loop route failed with 401 and the litellm compaction call failed to route. Full test suite green (one pre-existing order-dependent flake in test_execution.py fails identically on the base branch).

Link to Devin session: https://app.devin.ai/sessions/54414ea3ecb34ade9189534e5c690969
Requested by: @0xallam

@0xallam 0xallam self-assigned this Jul 30, 2026
@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment, CI, and merge conflict monitoring

Comment thread strix/report/dedupe.py Outdated
@greptile-apps

greptile-apps Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This follow-up completes propagation and isolation of configured LLM headers across model-backed call paths.

  • Passes main-model headers through shared ModelSettings construction for the agent loop, warm-up, and compaction.
  • Gives dedicated deduplication models separate configurable headers while preventing inheritance from the main model.
  • Routes compaction through StrixProvider and adds coverage for model routing and header propagation.
  • Fixes an unrelated TUI type-narrowing error exposed by the type-checking hook.

Confidence Score: 5/5

The PR appears safe to merge.

The previously reported cross-provider header leak is fixed: configured dedicated dedupe models use only their own headers, while main headers are inherited solely when dedupe falls back to the main model, and no blocking failure remains.

Important Files Changed

Filename Overview
strix/report/dedupe.py Selects dedicated dedupe headers whenever a dedupe model is configured and inherits main headers only for true main-model fallback.
strix/interface/main.py Aligns main and dedicated-dedupe warm-up requests with the shared runtime model-settings path.
strix/llm/compaction.py Replaces direct LiteLLM completion calls with provider-routed SDK requests carrying shared timeout and header settings.
strix/core/inputs.py Extends the common model-settings builder to copy and propagate optional per-request headers.
strix/core/runner.py Threads configured main-model headers into the agent-loop model settings.
strix/config/settings.py Adds the environment-backed dedicated dedupe header setting.
tests/test_dedupe_model.py Covers dedicated-header isolation and main-model fallback inheritance.
tests/test_compaction.py Covers provider routing, header propagation, response extraction, and compaction failure behavior.

Reviews (4): Last reviewed commit: "feat(llm): DEDUPE_LLM_EXTRA_HEADERS for ..." | Re-trigger Greptile

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptile

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptile

@devin-ai-integration

Copy link
Copy Markdown
Contributor Author

@greptile

@0xallam
0xallam merged commit d4e58b2 into main Jul 30, 2026
1 check passed
@0xallam
0xallam deleted the devin/1785374997-model-settings-extra-headers branch July 30, 2026 02:57
5hy7xz92nd-oss

This comment was marked as spam.

5hy7xz92nd-oss

This comment was marked as spam.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants