Repository navigation
RAGOps v2.6.0
RAGOps v2.6.0
New
ragops counterfactual: alters the evidence of each case and re-asks your own system (--answer-command). Perturbations:remove_support,swap_number(default), and opt-indrop_citation,inject_instruction(planted canary; one obeyed injection blocks) andabstain_without_evidence(--perturbation all). Exit 0 PASS, 2 BLOCK, 3 INCONCLUSIVE. See counterfactual-test.md.ragops mine: turns flagged production logs (thumbs down, low rating, escalation) into new regression cases in a new directory.ragops stability: blocks when repeated runs of the same questions vary too much, optionally against a baseline.ragops triageandcalibrateconfidence band: from human labels, routes cases to auto-pass, auto-fail or human review.- Claude Code / Cowork plugin marketplace:
/plugin marketplace add thangldw/ragops, then/plugin install ragops@ragops. The skill now covers the new commands. See claude-plugin.md. - Mutation testing now also covers the release policy engine; boundary tests pin gate semantics.
Limits
counterfactual was demonstrated on three toy systems only and has not been measured on a real RAG system; redundant evidence can cause false flags. mine, stability and triage were tested on constructed data, not real logs or labels. The Langfuse/OpenAI/Anthropic adapters still follow documented export shapes without a live export.
Compatibility
No breaking changes. The calibration report schema gains an optional confidence_band field.