Added
-
ollama adapter — local-model backend (
internal/ai/ollamacli.go, closes #30).
teststop now ships a first-class ollama adapter that calls the ollama HTTP API at
localhost:11434. No subprocess, no API key, no account quota.- Default model:
qwen3.6:latest(36B Q4_K_M). Override withTESTSTOP_MODEL. stream: false,think: false,num_ctx: 32768per request.- Automatic
<think>…</think>block stripping for qwen3 models on older ollama builds. - JSON-only output constraint appended to the mandate for local models (local models are
less instruction-following than cloud models; cloud adapters are unaffected). IsOllamaAvailable()pingslocalhost:11434with a 2s timeout for fast detection.
- Default model:
-
Auto-detection precedence changed: ollama → claude → copilot.
TESTSTOP_CLI=auto(the default) now prefers ollama when the local server is reachable.
Cloud CLIs (claude, copilot) are still fully supported but opt-in:
TESTSTOP_CLI=claudeorTESTSTOP_CLI=copilot.TESTSTOP_CLI=ollamaforces ollama only.Quality tradeoff (measured, against waymark API project):
Model Scenarios Time Notes qwen3.6:latest29–31 ~3–4 min High specificity; IDOR, race conditions, token expiry gemma4:latest52 ~8 min Most thorough; covers auth, concurrency, edge inputs qwen3:4b— — Not viable; outputs reasoning prose, never generates JSON Scenario depth and edge-case creativity from local models are moderately lower than
claude's output. For production runs where quota is not a concern,
TESTSTOP_CLI=clauderemains the highest-quality choice. -
Resilient JSON parsing for local models (
ParseScenariosFromJSON). Local models
produce sloppier output than cloud CLIs. The parser now uses a three-pass strategy:
(1) direct unmarshal; (2) extract[…]from prose-wrapped output (qwen3:4b pattern —
reasoning before JSON); (3) sanitize invalid JSON escape sequences then retry (gemma4
pattern — emits\xNNhex notation which is not valid JSON). Cloud adapter output
is unaffected (pass 1 always succeeds for claude/copilot). -
Design document (
docs/design/ollama-adapter.md). Records the HTTP-vs-CLI decision,
num_ctxrationale,think: falseapproach, and JSON-suffix strategy.