Goal: Define and land a skill runtime that works against local LLMs (Ollama, llama.cpp, vLLM, LM Studio, Jan). Grounds: RFC-AI-0004 Principle 3 — Vendor neutrality, which explicitly names "a local-Ollama wrapper" as a target runtime, and RFC-AI-0004's "no cloud-only skills" consequence.
Why local LLM:
- Sovereign deployments — projects bound by data-residency rules (EU, government, defence-orbit OSS) that can't send issue / mail content to a US cloud API
- Air-gapped triage — security teams that need to assess
<security-list> traffic without an internet connection
- Cost ceiling — projects unable to commit to a per-token cloud bill, but happy to run a 70B model on shared infra
What "parity" means (calibrated for what local LLMs can realistically do):
- Skills under
.claude/skills/<name>/SKILL.md are invokable via a local-LLM-backed agent loop
- The
tools/* bridges are reachable — these are language-agnostic CLI calls, no LLM context needed
- Sandbox / HITL primitives map to the same
bubblewrap baseline; the local-LLM runtime is the new piece
- Realistic scoping: some skills (multi-step reasoning over long contexts —
security-issue-triage, pr-management-code-review) may need a high-capability local model (Llama 70B, Qwen 72B, DeepSeek). Document the per-skill model-size floor so adopters can pick a model that actually works.
Suggested approach:
- Pick an agent-loop frontend that supports local LLMs natively — candidates:
aider --model ollama/..., goose with a local backend, continue.dev, a fresh thin wrapper around llama.cpp's OpenAI-compatible API
- Document the model-size floor empirically (per skill, against the existing eval suite at
tools/skill-evals/)
- Land a
setup-local-llm skill family alongside setup-isolated-* for the runtime-side install
Reference:
Goal: Define and land a skill runtime that works against local LLMs (Ollama, llama.cpp, vLLM, LM Studio, Jan). Grounds: RFC-AI-0004 Principle 3 — Vendor neutrality, which explicitly names "a local-Ollama wrapper" as a target runtime, and RFC-AI-0004's "no cloud-only skills" consequence.
Why local LLM:
<security-list>traffic without an internet connectionWhat "parity" means (calibrated for what local LLMs can realistically do):
.claude/skills/<name>/SKILL.mdare invokable via a local-LLM-backed agent looptools/*bridges are reachable — these are language-agnostic CLI calls, no LLM context neededbubblewrapbaseline; the local-LLM runtime is the new piecesecurity-issue-triage,pr-management-code-review) may need a high-capability local model (Llama 70B, Qwen 72B, DeepSeek). Document the per-skill model-size floor so adopters can pick a model that actually works.Suggested approach:
aider --model ollama/...,goosewith a local backend,continue.dev, a fresh thin wrapper aroundllama.cpp's OpenAI-compatible APItools/skill-evals/)setup-local-llmskill family alongsidesetup-isolated-*for the runtime-side installReference:
docs/rfcs/RFC-AI-0004.md.claude/skills/tools/skill-evals/