Local models (Ollama) with buzz-agent: undocumented, undiscoverable, needs first-class provider alias

[bitcoinaudio]

Summary

Local models (Ollama, llama.cpp, vLLM, etc.) are already usable with buzz-agent through the generic OpenAI-compatible provider — but nothing in the repo says so. There is zero mention of ollama anywhere in the codebase or docs, and the configuration is undiscoverable. Operators reasonably assume local models are unsupported, or set env vars on the relay (which never calls an LLM) and conclude "it doesn't work."

What works today (undocumented)

buzz-agent accepts any OpenAI-compatible endpoint:

BUZZ_AGENT_PROVIDER=openai        # or "openai-compat"
OPENAI_COMPAT_BASE_URL=http://localhost:11434/v1   # Ollama
OPENAI_COMPAT_MODEL=qwen2.5:7b-instruct
OPENAI_COMPAT_API_KEY=ollama      # any non-empty string; Ollama ignores it
OPENAI_COMPAT_API=chat            # or auto → chat for non-*.openai.com hosts

Verified against a local Ollama (qwen2.5:7b-instruct) — chat completions work, and tool-capable models are reported via /api/tags (capabilities: ["completion","tools"]).

The gaps

1. Docs gap — no docs/ page, README section, or example covers local models. The string "ollama" appears nowhere in the repo.
2. UX gap in buzz-agent config — Ollama requires a dummy API key (req("OPENAI_COMPAT_API_KEY")), and the operator must know the magic base URL http://localhost:11434/v1. A first-class BUZZ_AGENT_PROVIDER=ollama alias could default both (key optional, base URL defaulted) while still allowing overrides.
3. Architecture confusion — env vars set on the buzz-relay container/process silently do nothing for model selection; the LLM config lives wherever the ACP runtime (buzz-agent et al.) runs. Worth stating explicitly in deployment docs.
4. (Follow-up, separate PR) Desktop discoverability — the desktop agent config offers "OpenAI-compatible" as a provider but requires an API key and gives no hint that Ollama works; an "Ollama (local)" preset with no required key and a prefilled base URL would make this one click.

Proposal

• Add docs/local-models.md documenting the recipe above (Ollama first; note it applies to any OpenAI-compatible server).
• Add a first-class ollama provider alias in buzz-agent (crates/buzz-agent/src/config.rs): maps to the OpenAI-compatible path, defaults OPENAI_COMPAT_BASE_URL to http://localhost:11434/v1, makes the API key optional (default ollama), and defaults OPENAI_COMPAT_API to chat. Overrides via the existing env vars keep working.
• Unit tests for the new resolution paths.

I have a PR ready implementing exactly this.