Skip to content

v0.1.0 — Agentic RAG reference system

Choose a tag to compare

View all tags
@uehlingeric uehlingeric released this 12 Jul 04:07
v0.1.0
dc434d6

Provider-agnostic agentic RAG reference system, built end to end in 8 planned weeks — the plans, decisions (ADRs), and weekly review artifacts ship in the repo.

What it is

Ask a question about the NIST corpus (SP 800-53r5, SP 800-171r3, AI RMF, FIPS 199/200); get a citation-grounded answer or an honest refusal — through a vanilla pipeline or a LangGraph planner/critic agent loop, over hybrid retrieval (BM25 + dense + RRF), behind always-on guardrails, with an OTel span per stage. Four providers (Anthropic API/Bedrock, Google API/Vertex, OpenAI, Ollama) behind one adapter; the local path needs no keys.

Benchmark headlines (canonical run, 24 configs, guardrails on)

  • The agent loop cuts cloud-provider false refusals in every retrieval config (anthropic mean 0.28 → 0.18, google 0.37 → 0.26; largest gains on the weakest retriever) while rubric scores hold within 0.06 — at ~2× generation cost and latency.
  • The same loop makes the local 8B model worse on every rubric dimension. Agentic pipelines amplify model quality, in both directions.
  • 0 of 1,104 answers tripped a guardrail: the published numbers are the production path.
  • Full tables and methodology: docs/benchmarks.md; honest caveats: docs/limitations.md.

Quickstart

Verified on a clean machine: clone to first cited answer in 23 seconds (after Ollama model pull). make demo runs the full API + Ollama + Jaeger stack with zero cloud dependencies.

See CHANGELOG.md for the full feature list.

🤖 Generated with Claude Code

Assets 2
Loading