Skip to content

v0.2.0

Choose a tag to compare

@chef11102548c-cell chef11102548c-cell released this 11 Sep 12:09
· 4 commits to main since this release

Iteration 2 — v0.2.0 "AI core"

Must-include

# Feature สถานะ
1 RAG / KB ingestion (pgvector + BM25, 36-position playbook) ✅
2 Router + MoE specialist agents (Classifier → IP / Liability / Termination / General) ✅
3 Real LLM calls + Pydantic structured output (5 providers via providers.py) ✅
4 LLM-as-a-Judge grounding check (judge.v1.jinja, excerpt ≥ 4 words verbatim) ✅
5 Eval script on gold subset (scripts/run_eval.py --limit 3, ~18 min) ✅
6 eval_report.md checked in ✅

Eval Results (3 contracts / 22 clauses)

Metric Score
segmentation_f1 100%
citation_validity 100%
classification_accuracy 45.45%
risk_accuracy 50.00%

classification/risk ต่ำเพราะ gold label เปลี่ยน ไม่ใช่ pipeline แย่ลง
(CUAD annotate 41 คำถาม ไม่ได้จำแนก clause type ตรงๆ)


Pipeline

Upload → Parse → Segment → Router (MoE)
  → RAG (pgvector + BM25) → Risk Score → Judge → Report

Agents: MetadataExtractor · ClauseClassifier · RiskScorerAgent · JudgeAgent · OverrideAgent
RAG sources: data/playbook/positions.yaml (36 positions) + CUAD v1 (12 contracts)
Embedding: gemini-embedding-001 768 dim


Pass Bar

✅ Pipeline รันบน sample inputs ได้
✅ Eval metrics รายงานได้ (ไม่ต้องถึง final threshold)
✅ 268 offline tests + 11 live LLM tests ผ่าน