v1.0.0 — first stable release
Meridian 1.0 — a from-scratch grounded RAG engine over biomedical literature. Every ML component (tokenizer, dense retriever, ANN index, reranker, NLI verifier) is trained in-house; every answer is cited, verified, or refused.
This release clears the gate set in Phase 12: every benchmark section carries real measured numbers — no TBD cells, no metric estimated in prose.
Measured results
| Component | Result |
|---|---|
| BM25 retrieval (PubMedQA dev) | R@5 0.987 |
| Dense retriever (from scratch) | R@5 0.38 ± 0.02 (4 seeds) |
| Cross-encoder rerank | pure 0.029 → base-fused 0.983 |
| ANN index | HNSW recall@10 0.996 @ 0.262 ms (below brute force) |
| NLI verifier (SNLI dev, chance 0.333) | 0.783 |
| Gate-1 calibration | 0.801 coverage @ 0.000 error |
| Latency P50 | search 1.33 ms · embed 2.56 ms · verify 31.6 ms · rerank 421 ms |
Charts regenerate from committed measurement JSON (benchmarks/results/) via scripts/plot_summary.py — nothing is hand-drawn.
Findings we publish rather than hide
- MLM Stage-0 pretraining adds nothing measurable to retrieval (0.371 ± 0.022 vs random-init 0.382 ± 0.023, 4 seeds) — a seed-averaged ablation that overturned our own hypothesis.
- BM25 beats the from-scratch dense retriever on this lexically-easy task.
- The verifier does not transfer to biomedical text: 0.437 hallucination rate on verbatim quotes is a verifier domain gap (SNLI→PubMed), not a generator failure. SciNLI adaptation is the named fix.
- Learning rate dominated capacity: the same 384×6 model scored 0.607 at lr 1e-3 vs 0.783 at 3e-4.
Scope (honest)
Laptop-scale and production-inspired, not production-grade. Three items are explicitly not run rather than estimated: the ~200K domain-filtered PubMed corpus, the Phase-7 generator pipeline (and its generate latency), and verifier–human agreement. PyPI publication is deliberately deferred.
Not medical advice — a research literature assistant.
See BENCHMARKS.md and CHANGELOG.md.