Choosing a Model

Choosing a Model

SkynetClaw is model-independent. The council, the memory, the governance gate, and the grading loop do not care which engine answers. Swapping the model changes cost and quality — not the architecture.

---

Local — Ollama (default)

ollama pull llama3.1:8b          # reasoning
ollama pull qwen2.5-coder:7b     # execution
ollama serve

Nothing leaves your machine. No GPU required — a 7–8B model runs on CPU; it is slower, not broken.

Local — any OpenAI-compatible server

llama.cpp's llama-server, LM Studio, vLLM and similar all expose /v1:

curl -X POST http://127.0.0.1:8766/api/connections \
  -H 'Content-Type: application/json' \
  -d '{"api_type":"custom","label":"llama.cpp","base_url":"http://127.0.0.1:8080/v1"}'

Context ceiling matters. A server started with a small --ctx-size rejects long prompts, and the failure usually surfaces several steps later as a confusing model error rather than as "prompt too long". If long tasks fail oddly, check that first.

Cloud — ten providers

api_type	Provider
openai · anthropic · gemini · groq · openrouter
deepseek · xai · mistral · together · custom

Put the key in .env, then register a connection with the matching api_type.

curl -X POST http://127.0.0.1:8766/api/connections \
  -H 'Content-Type: application/json' \
  -d '{"api_type":"anthropic","label":"Claude","api_key":"sk-ant-..."}'

curl http://127.0.0.1:8766/api/connections
curl -X POST http://127.0.0.1:8766/api/connections/<id>/activate
curl http://127.0.0.1:8766/api/connections/<id>/ping

---

Why reasoning and execution are separate

"model":      "llama3.1:8b",       // deliberation — rare, long, quality matters
"exec_model": "qwen2.5-coder:7b",  // tool calls — frequent, short, speed matters

The model that reasons does not have to be the model that executes. Tool loops happen many times per task; deliberation happens once. A small fast model on the execution path keeps the loop cheap without dulling the thinking. Setting both to the same tag works — it just costs more time per tool call.

A common arrangement:

Path	Model	Why
Deliberation	a strong cloud model	the council is where quality pays
Execution	a small local model	frequent, and free
Sensitive work	local only	nothing leaves the machine

---

Which local model to start with

You have	Try	Expect
CPU, 8 GB RAM	llama3.2:3b	chat works; weak on long tool chains
CPU, 16 GB RAM	llama3.1:8b	the sensible default
GPU, 8–12 GB	llama3.1:8b / qwen2.5:14b	comfortable
GPU, 24 GB+	a 30B-class model	strong deliberation

An honest note. Below roughly 7B, models struggle with long multi-step tool chains and with returning strict JSON. That is not a defect in SkynetClaw: the governance gate will correctly reject malformed output and the run will fail loudly rather than silently accept nonsense. A small model failing at execution is the system working. Use a larger model, or route only execution to a code-tuned one.

---

Inspecting routing

curl http://127.0.0.1:8766/api/models        # what the active connection offers
curl http://127.0.0.1:8766/api/providers     # supported provider types
curl http://127.0.0.1:8766/api/router/config
curl http://127.0.0.1:8766/api/router/audit  # what actually routed where

📖 Repo copy: docs/MODELS.md

→ Next: Architecture