Repository navigation
v3.1.0 — LLM backend abstraction
Highlights
-
LLM backend abstraction. Every LLM call now routes through a swappable backend interface (
src/ragbot/llm/). Two backends ship:- litellm (default) — wraps
litellm.completion(). Best provider/model coverage, handles long-tail provider quirks. Pinned>=1.83.0(post-March-2026 supply-chain incident range). - direct — opt-in. Calls
anthropic,openai, andgoogle-genaiSDKs directly. Smaller dependency surface, no third-party gateway.
Selection:
RAGBOT_LLM_BACKEND={litellm|direct}. Adding a new backend (Bifrost, Portkey, OpenRouter, etc.) is one new file plus one selection arm. - litellm (default) — wraps
-
Web UI controls for reasoning effort (
auto,off,minimal,low,medium,high) and the cross-workspace skills auto-include toggle. -
/api/chatacceptsthinking_effortandadditional_workspaces— the same surface as the CLI flags. -
Provider-quirk handling lives in backends. Backends absorb GPT-5.x
max_completion_tokensrename, Claude 4.7+thinking.type.adaptiveshape, and the Anthropic-thinking-requires-temperature=1constraint. The chat code path stays clean.
Bug fixes (caught during manual testing)
- Chat CLI workspace lookup ignored
~/.synthesis/console.yamldiscovery; now honours overrides only and falls through to the full discovery chain. get_relevant_contextbudget loop usedbreakinstead ofcontinue, blocking smaller results when an oversized whole-file script chunk topped the rankings.- Anthropic + thinking now forces
temperature=1.0automatically. - Claude 4.7+ uses the new
thinking.type.adaptiveshape instead of LiteLLM's olderenabledshape (which 4.7+ rejects). transformers>=5.6.0pinned to tracktokenizers 0.22.xcompatibility (older transformers crashedsentence_transformersimport).
API additions
ChatRequest.thinking_effort: 'auto'|'off'|'minimal'|'low'|'medium'|'high'ChatRequest.additional_workspaces: string[](empty array opts out of skills auto-include; omitted lets the server auto-include)
Tests
- 26 new tests for the LLM backend abstraction (contract, selection, kwargs builder, direct healthcheck).
- 6 regression tests covering the bug fixes above.
- Final pass: 284 passed, 5 skipped.
Strategic note
LiteLLM remains a defensible default for ragbot in April 2026, primarily because of its provider/model coverage breadth. The March-2026 supply-chain incident (1.82.7–1.82.8) is bounded by the >=1.83.0 pin, but the API-compatibility lag for Claude 4.7+ shows the recurring tax of multi-provider gateways. The abstraction layer in this release is the response: keep LiteLLM as the default, make swap-in alternatives a configuration change rather than a code rewrite.
Setup
See README.md, CONFIGURE.md, and README-DOCKER.md.