Providers

Providers

TinyAgents keeps model calls behind provider-neutral harness traits. Provider adapters translate ModelRequest into the provider wire format and normalize responses, usage, streaming chunks, and errors back into TinyAgents types.

Supported Provider Kinds

ProviderKind currently covers:

• openai
• anthropic
• ollama
• deepseek
• groq
• xai
• openrouter
• together
• mistral
• compatible

Most hosted providers listed here are reached through an OpenAI-compatible Chat Completions endpoint. The compatible kind is for any endpoint that implements that wire format but does not need a named preset.

Provider Specs

ProviderSpec is the portable provider configuration:

use tinyagents::harness::providers::{ProviderKind, ProviderSpec};

let spec = ProviderSpec::for_kind(ProviderKind::Groq)
    .with_model("llama-3.3-70b-versatile");

Each spec records:

• provider kind
• provider id used in profiles and normalized errors
• default model id
• base URL
• API-key environment variable, when required
• whether a real API key is required

OpenAI

export OPENAI_API_KEY=...
export OPENAI_MODEL=gpt-4.1-mini
cargo run --features openai --example openai_chat

use tinyagents::harness::providers::openai::OpenAiModel;

let model = OpenAiModel::from_env()?;

OpenAiModel::from_env() reads OPENAI_API_KEY, optional OPENAI_MODEL, and optional OPENAI_BASE_URL.

Named Compatible Providers

The OpenAI-compatible adapter includes convenience constructors:

use tinyagents::harness::providers::openai::OpenAiModel;

let anthropic = OpenAiModel::anthropic(std::env::var("ANTHROPIC_API_KEY")?);
let groq = OpenAiModel::groq(std::env::var("GROQ_API_KEY")?);
let ollama = OpenAiModel::ollama();

Named constructors set the provider id, default base URL, and default model. Override the model with with_model(...) when needed.

From ProviderSpec

use tinyagents::harness::providers::{ProviderKind, ProviderSpec};
use tinyagents::harness::providers::openai::OpenAiModel;

let spec = ProviderSpec::for_kind(ProviderKind::Mistral)
    .with_model("mistral-small-latest");

let model = OpenAiModel::from_spec_env(spec)?;

from_spec_env reads the spec's configured environment variable. Use from_spec(spec, api_key) when credentials come from another secret source.

Ollama

Start Ollama with an OpenAI-compatible endpoint available at http://localhost:11434/v1, then use:

use tinyagents::harness::providers::openai::OpenAiModel;

let model = OpenAiModel::ollama().with_model("llama3.2");

Ollama ignores the API key, so TinyAgents uses a placeholder.

Error And Stream Normalization

Providers should report failures as ProviderError with:

• provider id
• model id when known
• HTTP status when available
• provider error code when available
• human-readable message
• retryability hint
• raw provider payload when useful

Streaming providers emit ModelStreamItem values. Normal chunks become deltas, usage updates, tool-call chunks, or final messages. Provider-side stream errors become ModelStreamItem::ProviderFailed, so the accumulator and harness see the same normalized failure shape as non-streaming calls.

Provider Selection

ProviderKind::infer(...) supports LangChain-style explicit prefixes such as openai:gpt-4.1-mini, anthropic:claude-..., and ollama:llama3.2, plus conservative bare-model inference for common model families.

Prefer explicit ProviderSpec values in production. Inference is useful for configuration files, examples, and interactive workflows where the model string is the only user input.