Repository navigation
Releases
Releases · grigortodorov/rag-best-practices
v0.10.0 — Source conflict check
ragpractices 0.10.0 — Source conflict check
Highlights
- Pre-answer conflict detector (
ragpractices.conflict_check/ CLIconflict-check): flag when retrieved sources disagree on numbers (similar local context or strong keyword buckets like day/percent) or negation polarity on shared phrases—before the model invents a compromise. - Optional
--answer/--answer-filenotes when a draft sided with one camp; decision staysconflict. - Fail-closed CI gate: exit 1 on
conflict, 0 onok.--jsonfor machine-readable reports. - Complements
claim-check,cite-check,ground, andcanary. Educational heuristic—not an NLI/contradiction model.
Example
ragpractices conflict-check --sources examples/conflict-docs.jsonl
# decision: conflict (30 vs 90 days)Tests
151 → 159 unit tests (all green). Stdlib-only.
v0.9.0 — Claim-support audit
ragpractices 0.9.0 — Claim-support audit
Highlights
- New `claim-check`: claim-level content-word overlap (`supported` / `weak` / `unsupported`) plus hard number / date / id faithfulness.
- Decision gate: `pass` / `rewrite` / `abstain` (fail-closed; CLI exits 1 on rewrite/abstain for CI).
- Complements existing aggregate `ground`, `cite-check`, and `decide` — does not replace them.
- Educational heuristic only — not production NLI.
API
`ClaimResult`, `EntityHit`, `ClaimCheckReport`, `check_claims`, `split_claims`, `extract_sensitive_tokens`
CLI
```bash
ragpractices claim-check --answer "..." --sources examples/hybrid-docs.txt
ragpractices claim-check --answer "..." --sources docs.jsonl --supported-threshold 0.55 --weak-threshold 0.30
```
Tests
140 → 151 unit tests (all green). Stdlib-only; no new dependencies.
ragpractices 0.8.0
ragpractices 0.8.0
Niche tools that most RAG repos describe but rarely ship as commands.
Added
- Chunking A/B (
ragpractices chunk-ab) — compare fixed / fixed_small / fixed_large / headings with document-level hit@k + MRR after chunking - Citation span verifier (
ragpractices cite-check) — quoted spans and[n]markers vs sources (supported / unsupported / unused / orphans) - Lost-in-the-middle stress (
ragpractices position-stress) — gold evidence at first / middle / last with token estimates and attention-risk labels (no LLM) - Index canaries (
ragpractices canary) — probe queries with expected ids + optional ACL filters; CI fails if any probe fails
Changed
- Version 0.8.0 · 140 unit tests
- PyPI publish workflow is manual only (
workflow_dispatch) until trusted publishing is configured
See CHANGELOG.md.
ragpractices 0.7.0
ragpractices 0.7.0
Highlights
- Strategy compare —
ragpractices compareruns golden cases throughkeyword,hybrid,rewrite_hybrid,hybrid_rerank(hit@k + MRR + ranking) - Grounded prompt templates —
ragpractices prompt(cite/abstain/--clarify); templates only, no API calls - Metadata + ACL filters — fail-closed
filter_docs/ragpractices filter; optional pipeline"filter";examples/acl-docs.jsonl - HTML ingest + content hashing — stdlib
html.parser, sha256content_hash;ragpractices html/ ingest.html - Eval CI gate —
--min-hit-rate/--min-mrr; CI requires hit@k ≥ 0.6 on the golden set - Expanded golden sets (
golden-retrieval.jsonl,golden-retrieval-hard.jsonl)
Stdlib-only. 113 unit tests. See CHANGELOG.md.