Skip to content

Releases

v0.10.0 — Source conflict check

Choose a tag to compare

@grigortodorov grigortodorov released this 23 Sep 18:25

ragpractices 0.10.0 — Source conflict check

Highlights

  • Pre-answer conflict detector (ragpractices.conflict_check / CLI conflict-check): flag when retrieved sources disagree on numbers (similar local context or strong keyword buckets like day/percent) or negation polarity on shared phrases—before the model invents a compromise.
  • Optional --answer / --answer-file notes when a draft sided with one camp; decision stays conflict.
  • Fail-closed CI gate: exit 1 on conflict, 0 on ok. --json for machine-readable reports.
  • Complements claim-check, cite-check, ground, and canary. Educational heuristic—not an NLI/contradiction model.

Example

ragpractices conflict-check --sources examples/conflict-docs.jsonl
# decision: conflict (30 vs 90 days)

Tests

151 → 159 unit tests (all green). Stdlib-only.

v0.9.0 — Claim-support audit

Choose a tag to compare

@grigortodorov grigortodorov released this 22 Sep 17:39

ragpractices 0.9.0 — Claim-support audit

Highlights

  • New `claim-check`: claim-level content-word overlap (`supported` / `weak` / `unsupported`) plus hard number / date / id faithfulness.
  • Decision gate: `pass` / `rewrite` / `abstain` (fail-closed; CLI exits 1 on rewrite/abstain for CI).
  • Complements existing aggregate `ground`, `cite-check`, and `decide` — does not replace them.
  • Educational heuristic only — not production NLI.

API

`ClaimResult`, `EntityHit`, `ClaimCheckReport`, `check_claims`, `split_claims`, `extract_sensitive_tokens`

CLI

```bash
ragpractices claim-check --answer "..." --sources examples/hybrid-docs.txt
ragpractices claim-check --answer "..." --sources docs.jsonl --supported-threshold 0.55 --weak-threshold 0.30
```

Tests

140 → 151 unit tests (all green). Stdlib-only; no new dependencies.

ragpractices 0.8.0

Choose a tag to compare

@grigortodorov grigortodorov released this 22 Sep 12:47

ragpractices 0.8.0

Niche tools that most RAG repos describe but rarely ship as commands.

Added

  • Chunking A/B (ragpractices chunk-ab) — compare fixed / fixed_small / fixed_large / headings with document-level hit@k + MRR after chunking
  • Citation span verifier (ragpractices cite-check) — quoted spans and [n] markers vs sources (supported / unsupported / unused / orphans)
  • Lost-in-the-middle stress (ragpractices position-stress) — gold evidence at first / middle / last with token estimates and attention-risk labels (no LLM)
  • Index canaries (ragpractices canary) — probe queries with expected ids + optional ACL filters; CI fails if any probe fails

Changed

  • Version 0.8.0 · 140 unit tests
  • PyPI publish workflow is manual only (workflow_dispatch) until trusted publishing is configured

See CHANGELOG.md.

ragpractices 0.7.0

Choose a tag to compare

@grigortodorov grigortodorov released this 22 Sep 12:37

ragpractices 0.7.0

Highlights

  • Strategy compare — ragpractices compare runs golden cases through keyword, hybrid, rewrite_hybrid, hybrid_rerank (hit@k + MRR + ranking)
  • Grounded prompt templates — ragpractices prompt (cite / abstain / --clarify); templates only, no API calls
  • Metadata + ACL filters — fail-closed filter_docs / ragpractices filter; optional pipeline "filter"; examples/acl-docs.jsonl
  • HTML ingest + content hashing — stdlib html.parser, sha256 content_hash; ragpractices html / ingest .html
  • Eval CI gate — --min-hit-rate / --min-mrr; CI requires hit@k ≥ 0.6 on the golden set
  • Expanded golden sets (golden-retrieval.jsonl, golden-retrieval-hard.jsonl)

Stdlib-only. 113 unit tests. See CHANGELOG.md.