Skip to content

v0.3.0 - Qualification Category Expansion

Choose a tag to compare

@yafitzdev yafitzdev released this 02 Feb 08:15
· 117 commits to main since this release

Changes

Qualification Category Expanded (10 → 40 test cases)

Added 30 new test cases across 6 subcategories to comprehensively test epistemic uncertainty handling:

Subcategory Count Description
causal_without_evidence 12 "Why" questions where context only shows "what/that"
correlation_causation 8 Classic confounding variable traps
prediction_insufficient_data 9 Future predictions with only historical data
small_sample 4 Generalizations from tiny samples
incomplete_evidence 5 Partial information presented as complete
attribution_error 2 Multiple factors contributing to single outcome

Difficulty distribution: 10 easy, 15 medium, 15 hard

Corpus Expansion

  • Added 28 new documents to support qualification test cases
  • Total corpus: 128 documents (was 100)

Test Coverage

Category Cases
Abstention 10
Dispute 10
Qualification 40
Confidence 10
Grounding 5
Relevance 5
Total 80

Files Changed

  • data/qualification/qualification.json - Expanded from 10 to 40 cases
  • data/corpus/documents.jsonl - Added 28 new documents (128 total)
  • data/corpus/manifest.json - Updated version and counts
  • data/queries/query_mappings.json - Added 30 new qualification mappings