Skip to content

v5.1.0

Latest

Choose a tag to compare

@yafitzdev yafitzdev released this 01 Mar 04:53
· 38 commits to main since this release

🎯 Data Quality Fixes + Documentation Refresh

This release fixes data labeling issues discovered after v5.0.0 and brings all documentation up to date with the v5 architecture.

🔧 Data Fixes

  • Relabeled 199 trustworthy-with-gap cases to abstain — cases where context had insufficient evidence were incorrectly labeled as trustworthy
  • Removed 10 mislabeled single-context dispute cases — disputes require multiple sources; single-context cases cannot be genuine disputes
  • Fixed 51 broken queries across the dataset
  • Updated test suite to match new labels

📚 Documentation Refresh

  • README.md: Added Mermaid evaluation flow diagram, emoji section headers, links to evaluation guide and mode decision tree
  • docs/evaluation-guide.md: Full rewrite for v5 — 4 categories, cross-cutting quality checks, updated counts/difficulty/FAQ
  • docs/mode-decision-tree.md: Updated category names (qualification → trustworthy_hedged, confidence → trustworthy_direct), rewrote grounding/relevance as cross-cutting checks
  • docs/GOVERNANCE_CASE_TAXONOMY.md: Updated version, case counts, removed stale path and broken link
  • CHANGELOG.md: Restored emoji consistency on v4.0–v5.0 subsection headers
  • Archived completed v5-plan docs and multi-source proposal to docs/roadmap/archive/
  • Fixed stale difficulty breakdown (trustworthy_hedged: 435/725 → 428/732 medium/hard)

📊 Benchmark Totals (unchanged)

Category Cases
Abstention 685
Dispute 675
Trustworthy Hedged 1,160
Trustworthy Direct 400
Total (Tier 1) 2,920