Skip to content

v0.4.0 — grounded answering, dashboard, one-command setup

Choose a tag to compare

View all tags
@awaistechnologist awaistechnologist released this 13 Aug 09:15
· 9 commits to main since this release
v0.4.0
4218883

First tagged release. llm-sidecar is a local sidecar that gives every tool on your machine grounded, cited, routed AI — and never asks you which model to use.

Works with no API key at all if you have Ollama.

git clone https://github.com/awaistechnologist/llm-sidecar
cd llm-sidecar && ./install.sh && ./run.sh

What's in it

Grounded answering — answer() searches, reads the pages, and answers from those pages only, with citations. Says "not in the sources" rather than guessing, and reports conflicts between sources.

Verification — grade claims against live evidence (supported / contradicted / unverified), each cited. Claims that snippets can't settle are automatically re-checked against full page text.

Routed inference — local Ollama, free OpenRouter, or paid, without naming a model. Every candidate is probed with a live call before use, because a catalogue entry is not a working model.

Four ways in — Python library, OpenAI-compatible HTTP daemon, MCP server (12 tools), and a CLI. Plus a dashboard: chat with per-reply cost receipts, every capability in a Tools tab, and settings.

Structured operations — summarise, classify, extract, all at temperature 0 so they're repeatable and cached.

Operational — hardware advisor (which Ollama models actually fit your RAM), usage ledger, response cache with size limits, one-command SearXNG setup.

Notable design choices

  • The daemon treats the model field as a request, not an instruction — a tool hardcoding gpt-4o gets a verified working model and never finds out.
  • Structured output is treated as a capability: a model that can't produce valid JSON is rotated away from, exactly like a rate-limited one.
  • The dashboard is one HTML file with no build step and no external requests — no CDN, no fonts, no analytics.
  • The daemon binds loopback and never returns your API key, only a masked preview.

Known limitations

Verification is only as good as retrieval. Streaming bypasses the completion cache. No request queue, so it's a single-user tool. No embeddings or cross-session memory — different product. See the README's "Honest limitations" section.

Tests

152, fully offline. Live-provider behaviour verified by hand.

Assets 2
Loading