Repository navigation
v0.2.0 - Handcrafted Test Cases + Corpus
FITZ-GOV v0.2.0
Complete governance benchmark with handcrafted test cases and curated corpus.
Contents
| Component | Count | Description |
|---|---|---|
| Test Cases | 50 | Handcrafted governance scenarios |
| Corpus Documents | 100 | Curated documents for retrieval testing |
Test Cases by Category
| Category | Cases | Tests |
|---|---|---|
| abstention | 10 | Queries with irrelevant context |
| dispute | 10 | Queries with conflicting sources |
| qualification | 10 | Causal queries without causal evidence |
| confidence | 10 | Queries with clear answers |
| grounding | 5 | Hallucination traps |
| relevance | 5 | Off-topic traps |
Corpus
100 documents across 20+ domains designed to match test queries:
- Decoy documents for abstention testing
- Contradicting documents for dispute testing
- Correlation-only documents for qualification testing
- Clear-answer documents for confidence testing
Evaluation Modes
- Mode A: Injected contexts (tests governance only)
- Mode B: Full pipeline with retrieval (tests retrieval + governance)