Skip to content

fix(agents): custom model provider routing#2530

Merged
daryllimyt merged 1 commit intomainfrom
fix/custom-model-provider-routing
Apr 15, 2026
Merged

fix(agents): custom model provider routing#2530
daryllimyt merged 1 commit intomainfrom
fix/custom-model-provider-routing

Conversation

@jordan-umusu
Copy link
Copy Markdown
Collaborator

@jordan-umusu jordan-umusu commented Apr 15, 2026

Summary by cubic

Fixes custom model provider routing with a passthrough mode and always proxies LLM traffic through the local bridge. Drops the "openai/" prefix for custom providers in both runtime routing and credential injection for correct upstream routing.

  • Refactors

    • Replaced litellm with custom-model-provider + passthrough on AgentConfig and SandboxAgentConfig (propagated through workflow schemas/payloads).
    • Added resolve_custom_model_provider_config_activity to load base_url, optional model_name, and passthrough from workspace credentials; applied in the durable workflow and management service.
    • Updated LLMSocketProxy: renamed litellm_url to upstream_url, added passthrough that strips Authorization and Anthropic-only fields before forwarding; proxy now always starts.
    • Removed openai/ prefix for custom-model-provider (runtime no longer maps to "openai"; gateway injects the raw model name).
    • Secrets API now returns aggregated registry definitions (sorted) and adds litellm with optional LITELLM_BASE_URL; frontend copy updated to reflect “aggregated” definitions.
  • Migration

    • Replace any litellm provider usage with custom-model-provider.
    • Configure workspace credentials: CUSTOM_MODEL_PROVIDER_BASE_URL (required), optional CUSTOM_MODEL_PROVIDER_MODEL_NAME, and CUSTOM_MODEL_PROVIDER_PASSTHROUGH=true to forward directly to your upstream.
    • No app config changes needed for the bridge; the LLM socket proxy and bridge start automatically.

Written for commit a4388c9. Summary will update on new commits.

@jordan-umusu jordan-umusu force-pushed the fix/custom-model-provider-routing branch from 9ed4c6a to 067fd57 Compare April 15, 2026 10:12
@jordan-umusu jordan-umusu marked this pull request as ready for review April 15, 2026 10:12
@jordan-umusu jordan-umusu temporarily deployed to internal-registry-ci April 15, 2026 10:12 — with GitHub Actions Inactive
@jordan-umusu jordan-umusu temporarily deployed to internal-registry-ci April 15, 2026 10:12 — with GitHub Actions Inactive
@zeropath-ai
Copy link
Copy Markdown

zeropath-ai bot commented Apr 15, 2026

No security or compliance issues detected. Reviewed everything up to a4388c9.

Security Overview
Detected Code Changes
Change Type Relevant files
Refactor ► tests/unit/test_agent_runtime.py
    Update expected model name for custom-model-provider in test data
► tracecat/agent/gateway.py
    Remove "openai/" prefix from custom model names
► tracecat/agent/runtime/claude_code/runtime.py
    Remove "custom-model-provider": "openai" mapping

@blacksmith-sh

This comment has been minimized.

Copy link
Copy Markdown
Contributor

@cubic-dev-ai cubic-dev-ai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No issues found across 2 files

Confidence score: 5/5

  • Automated review surfaced no issues in the provided summaries.
  • No files require special attention.

@jordan-umusu jordan-umusu force-pushed the fix/custom-model-provider-routing branch from 067fd57 to a4388c9 Compare April 15, 2026 12:02
@jordan-umusu jordan-umusu temporarily deployed to internal-registry-ci April 15, 2026 12:02 — with GitHub Actions Inactive
@jordan-umusu jordan-umusu temporarily deployed to internal-registry-ci April 15, 2026 12:02 — with GitHub Actions Inactive
@daryllimyt daryllimyt merged commit 2426e15 into main Apr 15, 2026
16 checks passed
@daryllimyt daryllimyt deleted the fix/custom-model-provider-routing branch April 15, 2026 12:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants