Skip to content

v0.10.0 — cross-framework provider switching + llama.cpp validated

Choose a tag to compare

@ahwurm ahwurm released this 27 Jul 17:27
· 292 commits to main since this release

LocalHarness now speaks to more than one local inference framework and can switch between models served on different backends.

What's new

  • Switch across frameworks: /model re-points the live agent between models served on different endpoints, re-probing capabilities and token counting on the way. The picker groups models by framework + host.
  • llama.cpp validated as a second provider: auto-detected, exact token counting via its /tokenize, served-window discovery, and a real multi-step agentic tool-call round trip verified end to end on GPU.

Honest scope

  • Ollama and LM Studio use the same OpenAI-compatible client path, but their full on-GPU certification — and the harness launching non-vLLM servers itself (lifecycle) — land in 0.11. This release is switch + attach + llama.cpp validation.
  • Pre-1.0: interfaces and config schema may still change.