Connecting Models

Connecting models

OpenAgentFleet normalises every provider behind one connector interface. A bot runs against a chain — try this model, fall back to that one — and chains can mix single providers with role-based combinations (one model for vision, another for reasoning, another for summaries).

Providers

Provider	What to enter	Notes
Ollama	Base URL http://host:11434, a model name	The local-first path; nothing leaves your network, runs cost $0.00
OpenAI	API key
Anthropic	API key
Gemini	API key	Key travels in a header, never a query string
OpenAI-compatible gateways	Base URL + key	LiteLLM, vLLM, OpenRouter, llama.cpp server, etc.

Add them in the console under Engines. Model lists are discovered live from the provider where the API supports it.

Vision matters

The agent loop perceives through screenshots. A chain whose active model can't see images falls back to the accessibility tree alone, which works but is weaker. For desktop driving, prefer a vision-capable model — on Ollama, the *-vision tags.

Chains and combinations

• Chain: an ordered list per bot — "these two models, and if neither answers, that one". Failures and rate limits fall through automatically.
• Combination: a named mapping from roles (vision, reasoning, chat, summarize, refine) to models. Use a combination anywhere a single provider would go, including inside a chain.

Embeddings and memory

Semantic memory works out of the box: the compose stack ships a local embedding sidecar (model2vec potion-base-8M, CPU-only, air-gap friendly). Configured providers that can embed are preferred for quality. GET /api/memory/fleet reports which scheme is live; EMBED_BASE_URL=off selects the hashed keyword fallback deliberately.

Cost telemetry

Cloud-model prices are fetched daily with an offline fallback table, and the source is reported rather than assumed. Cached tokens are counted and discounted. Console → Financials (or fleetctl financials).