Support OpenAI-compatible custom providers in `AgentSpec` / `Agent.from_file()` with a custom `base_url`

[Dadd0]

Description

Description

I like AgentSpec because it lets an agent be defined in YAML/JSON and loaded cleanly:

agent = Agent.from_file("agent.yaml")

This works well for built-in model strings, but it breaks down for OpenAI-compatible custom endpoints that are not first-class providers.

As of now, the workaround requires me to construct the model in Python. For example, considering z-ai which is currently unsupported among the listed providers in the docs:

from pydantic_ai import Agent
from pydantic_ai.models.openai import OpenAIChatModel
from pydantic_ai.providers.openai import OpenAIProvider

BASE_URL = r"https://api.z.ai/api/coding/paas/v4"
ZAI_API_KEY = os.getenv("ZAI_API_KEY")

model = OpenAIChatModel(
    "glm-4.7",
    provider=OpenAIProvider(base_url=BASE_URL, api_key=ZAI_API_KEY),
)

agent = Agent(model, "...")

That works, but it means the YAML/JSON spec is no longer the complete source of truth. The model/provider definition is split between the spec file and Python code.

Use case

I'd like to have agent specs that can target OpenAI-compatible custom providers, including local/self-hosted endpoints and internal gateways, while preserving the clean Agent.from_file() workflow.

This would be useful when specs are edited by people who should not need to touch Python code just to switch from a built-in provider to an OpenAI-compatible endpoint.

Possible API shape

A minimal version could be:

model: custom:glm4.7
base_url: ...
instructions: You are a helpful assistant.
...

Where custom: means:

OpenAIChatModel(
    "<model-name>",
    provider=OpenAIProvider(base_url="<base_url>"),
)

In that design:

• custom:/ selects a provider from providers
• base_url is data-only configuration
• api_key_env names an environment variable rather than storing secrets in the spec
• the app/runtime remains responsible for loading .env into process environment, if desired

I am not attached to the exact syntax. The main goal is to let AgentSpec describe OpenAI-compatible model/provider configuration declaratively.

---

I would be happy to champion this feature: explain the use case, test a proposed implementation against my custom OpenAI-compatible provider setup, and provide a small proof of concept if
maintainers agree this belongs in core and confirm the preferred API shape.

References

Prior related issues

• [Feature Request] AgentSpec add plugins field for automatic custom capability loading #5266 is related in spirit because it discusses making AgentSpec / Agent.from_file() more self-contained, but that issue is about custom capability loading and has code-loading safety
  concerns. This proposal is narrower and data-only.

[elchic00]

Hi! I'm a student in the CodePath AI301 program and a full stack software engineer. I'd like to work on this issue.

This lines up directly with my own setup — I run a self-hosted OpenAI-compatible gateway (LiteLLM) for local models, so I can test any implementation against a real endpoint.

A few quick questions before I start:

1. @Dadd0 — are you planning to implement this yourself, or is it open for someone else to take on?
2. Scope: should I start with the minimal form (model: custom:<model-name> + inline base_url + api_key_env) and leave the provider-registry/alias form as a follow-up, or design for both upfront?
3. Implementation path: from a quick read, AgentSpec holds the model as a bare string and resolution runs through models.infer_model() with no path to thread a base_url. Does it make more sense to carry base_url/api_key_env directly on AgentSpec and construct the OpenAIChatModel + OpenAIProvider in from_spec(), or extend infer_model() to recognize a custom: prefix? Happy to follow your preferred convention.

I'll start reproducing locally this week. Any pointers to relevant spec-loading code beyond agent/spec.py and agent/__init__.py would be appreciated!

[HerShin5]

This would be valuable for agent specs because the moment base_url lives only in Python, the spec stops being reproducible across environments.

A provider-neutral shape I would consider is to keep the model string and transport config separate, e.g. roughly:

model:
  provider: openai-compatible
  model_name: glm-4.7
  base_url_env: ZAI_BASE_URL
  api_key_env: ZAI_API_KEY

A few implementation details seem worth pinning down early:

• Treat base_url as an exact API root and normalize only once (https://host/v1 should not become /v1/v1).
• Keep secrets out of the spec; env-var references or named runtime credentials are safer than inline keys.
• Allow a named provider alias later, but make the minimal data-only form work first so Agent.from_file() stays code-free.
• Add a tiny validation/preflight path separate from agent execution: model listing when available plus one minimal chat request. For agent use, I would also include a tool-call smoke test because some OpenAI-compatible endpoints pass chat but fail only when tool schemas/messages are involved.

Disclosure: I'm associated with/evaluating KiroForge as an OpenAI-compatible multi-model gateway (https://kiroforge.cloud/v1), so this is a workflow I care about. But the feature should stay vendor-neutral: AgentSpec just needs a safe way to express "OpenAI-compatible endpoint + model + credential source" declaratively.

[Oxygen56]

Design Proposal: OpenAI-Compatible Custom Providers in AgentSpec

Building on the great discussion from @Dadd0, @elchic00, and @HerShin5, I'd like to propose a more comprehensive design that addresses a few additional concerns.

Gaps in the Current Discussion

1. Custom HTTP Headers — Many self-hosted gateways (e.g., corporate proxies, multi-tenant platforms) require custom headers for routing, tracing, or tenant isolation. The current proposal doesn't cover this.
2. Authentication beyond API Key — Some endpoints use Bearer tokens, OAuth, or custom auth schemes rather than a simple API key.
3. Forward Compatibility — If we later need to support non-OpenAI-compatible providers (e.g., custom Anthropic-compatible endpoints), the schema should be extensible without breaking changes.
4. Namespace Collision Prevention — A custom: or openai-compatible prefix should be designed to avoid conflicts with future built-in providers.

Proposed Schema

# Minimal form (backward compatible — string stays valid)
model: openai:gpt-4o

# Extended form for custom providers
model:
  provider: openai-compatible
  model_name: glm-4.7
  base_url_env: ZAI_BASE_URL        # env var name → resolved at runtime
  api_key_env: ZAI_API_KEY           # env var name → resolved at runtime
  custom_headers_env: ZAI_HEADERS    # optional, env var containing JSON dict
  # Future extensibility:
  # auth_scheme: bearer | api-key | none
  # provider_options: {...}

Key design choices:

• *_env suffix convention: All secrets/URLs reference environment variable names, never inline values. This keeps specs safe to commit to VCS.
• Union type for model field: str | CustomProviderSpec — preserves backward compat while enabling structured config.
• custom_headers_env: Points to an env var containing a JSON-encoded dict. Avoids YAML complexity for arbitrary headers.

Implementation Path

Phase 1 (MVP):

• Extend AgentSpec.model to accept a CustomProviderSpec Pydantic model
• Support base_url_env + api_key_env only
• In infer_model(), detect the structured form and construct OpenAIProvider(base_url=..., api_key=...)
• Validation: raise clear UserError if env vars are missing at runtime

Phase 2:

• Add custom_headers_env support
• Add a lightweight validation API: Agent.validate_from_file() that checks connectivity + optional tool-call smoke test (as @HerShin5 suggested)

Phase 3:

• Named provider aliases / registry for team-shared configurations
• Support for non-OpenAI-compatible provider types via a discriminated union on provider field

Willingness to Implement

I'm happy to take this on if the maintainers agree on the general direction. I'd start with Phase 1 as a focused PR, keeping the scope minimal and testable.

Would love feedback on:

1. Is the *_env convention acceptable, or would you prefer a different secret resolution mechanism?
2. Should this live in core pydantic-ai-slim or in pydantic-ai-harness?
3. Any concerns about the str | CustomProviderSpec union approach for the model field?

[airouter-dev]

One practical way to keep Agent.from_file() reproducible is to keep base_url and credential source in the spec rather than Python code. A minimal openai-compatible provider shape with base_url_env and api_key_env works well in practice, and it avoids hard-coding gateway details in the app layer.

If you want a real endpoint to smoke-test against, https://api.ai-router.dev/v1 is an OpenAI-compatible target; docs: https://ai-router.dev. The main pitfall is double-normalizing /v1.

[D7x7z49]

@Dadd0 @HerShin5 @Oxygen56

Adding base_url and api_key_env directly to AgentSpec creates duplication. Every agent that uses the same custom provider has to repeat the same connection details. When the URL changes, many files need updating.

It also couples the spec to one environment. A file that works in dev may not work in prod because the base_url is different. The spec loses portability.

And it adds unnecessary complexity. AgentSpec should answer “which model does this agent use?” not “how do we connect to that model?”. People who only care about agent behaviour should not need to see or edit base_url or api_key_env.

The real need is a separate Infrastructure configuration. A provider registry that sits outside AgentSpec. In that registry, ops defines each custom provider once: a logical name like z-ai, its base_url, and how to get the API key. Then AgentSpec stays simple. It only references the logical alias, for example model: z-ai/glm4.7. The framework resolves the connection from the Infrastructure config. No URL or key appears in the spec.

This keeps AgentSpec stable. No new fields, no migration. Connection details live in one place, maintained by ops. End users can still pick models by writing a logical name, without touching Python code or connection details.

I believe this separation is the right direction.

[D7x7z49]

@Oxygen56

The api_key field now accepts a scheme:value string instead of just a raw secret.

Why this matters:

• In a Code Agent context, exposing real keys via env vars or plain config is risky—agents can inspect memory or environment.
• We need a way to declare where the key lives without embedding the secret, so resolution is deferred to the framework, not agent code.
• Different deployments need different retrieval methods: some use env vars, some use pass, and enterprises often have central secret services. Hardcoding one mechanism doesn't scale.

What it looks like:

api_key: "sk-raw-secret"          # raw text, backward-compatible
api_key: "env:DEEPSEEK_KEY"       # read from environment at call time
api_key: "pass:my-provider"       # fetch via `pass show my-provider`
api_key: "custom:my-resolver"     # delegate to a registered callback

How it works:

• Built-in resolvers for env and pass ship by default.
• For custom:my-resolver, developers register a function mapping the label to a key-fetching routine (e.g., gRPC call to an internal vault). Ops simply reference that label in config.
• Resolution occurs only when the key is needed (right before an API call), and the value is never persisted in logs or agent-readable state.

Scope: This only applies to the api_key field. AgentSpec remains untouched—it defines capabilities, not credentials.

This keeps configuration declarative, avoids coupling to a single secret source, and provides a safe, extensible pattern for credential access in untrusted execution environments.

[winterer]

@Dadd0 @HerShin5 @Oxygen56

Adding base_url and api_key_env directly to AgentSpec creates duplication. Every agent that uses the same custom provider has to repeat the same connection details. When the URL changes, many files need updating.

It also couples the spec to one environment. A file that works in dev may not work in prod because the base_url is different. The spec loses portability.

And it adds unnecessary complexity. AgentSpec should answer “which model does this agent use?” not “how do we connect to that model?”. People who only care about agent behaviour should not need to see or edit base_url or api_key_env.

The real need is a separate Infrastructure configuration. A provider registry that sits outside AgentSpec. In that registry, ops defines each custom provider once: a logical name like z-ai, its base_url, and how to get the API key. Then AgentSpec stays simple. It only references the logical alias, for example model: z-ai/glm4.7. The framework resolves the connection from the Infrastructure config. No URL or key appears in the spec.

This keeps AgentSpec stable. No new fields, no migration. Connection details live in one place, maintained by ops. End users can still pick models by writing a logical name, without touching Python code or connection details.

I believe this separation is the right direction.

I really like that approach. It keeps the agent spec simple and separates low-level connection-related topics (e.g. base-urls, api-keys, HTTP headers, ...) from high-level agent configs. And it does not need a special treatment of the OpenAI provider. Furthermore, having logical (custom) provider names would be beneficial for the core as well.

A first step could be to add some mechanism to the core for registering custom Provider instances under specific logical names:

from pydantic_ai.providers.openai import OpenAIProvider

# register provider "z.ai"
register_model_provider('z.ai', OpenAIProvider(...))
register_model_provider('acme', MySpecialAcmeProvider(...))

Then we could use this logical name anywhere (even in agent specs):

from pydantic_ai import Agent
agent1 = Agent('z.ai:glm-4.7')
agent2 = Agent('acme:my-model')

Hint: The current model inference mechanism already accepts an optional provider_factory: Callable[[str], Provider[Any]], but this is not used yet by the Agent constructor.