v1.0.1 — PyPI packaging

Packaging release for the first PyPI publication. No functional change from
v1.0.0.

Fixed

• Development Status classifier 2 - Pre-Alpha → 4 - Beta — Pre-Alpha contradicted a 1.0 release on the rendered PyPI page.
• Added a Documentation project URL pointing at BENCHMARKS.md.
• Version bumped in pyproject.toml and meridian.__version__ so the published artifact matches its git tag exactly.

Verified before release

• twine check PASSED on both sdist and wheel.
• Wheel installed into a clean Python 3.12 venv, resolving polaris-nlp==1.4.0 and zenith-nlp==1.1.0 from PyPI.
• meridian ingest + meridian ask run end-to-end from the installed package, returning a GROUNDED cited answer.
• Gates: Black, Ruff, mypy --strict, 254 tests, 95.7% coverage.

pip install meridian-rag

v1.0.0 — first stable release

Meridian 1.0 — a from-scratch grounded RAG engine over biomedical literature. Every ML component (tokenizer, dense retriever, ANN index, reranker, NLI verifier) is trained in-house; every answer is cited, verified, or refused.

This release clears the gate set in Phase 12: every benchmark section carries real measured numbers — no TBD cells, no metric estimated in prose.

Measured results

Component	Result
BM25 retrieval (PubMedQA dev)	R@5 0.987
Dense retriever (from scratch)	R@5 0.38 ± 0.02 (4 seeds)
Cross-encoder rerank	pure 0.029 → base-fused 0.983
ANN index	HNSW recall@10 0.996 @ 0.262 ms (below brute force)
NLI verifier (SNLI dev, chance 0.333)	0.783
Gate-1 calibration	0.801 coverage @ 0.000 error
Latency P50	search 1.33 ms · embed 2.56 ms · verify 31.6 ms · rerank 421 ms

Charts regenerate from committed measurement JSON (benchmarks/results/) via scripts/plot_summary.py — nothing is hand-drawn.

Findings we publish rather than hide

• MLM Stage-0 pretraining adds nothing measurable to retrieval (0.371 ± 0.022 vs random-init 0.382 ± 0.023, 4 seeds) — a seed-averaged ablation that overturned our own hypothesis.
• BM25 beats the from-scratch dense retriever on this lexically-easy task.
• The verifier does not transfer to biomedical text: 0.437 hallucination rate on verbatim quotes is a verifier domain gap (SNLI→PubMed), not a generator failure. SciNLI adaptation is the named fix.
• Learning rate dominated capacity: the same 384×6 model scored 0.607 at lr 1e-3 vs 0.783 at 3e-4.

Scope (honest)

Laptop-scale and production-inspired, not production-grade. Three items are explicitly not run rather than estimated: the ~200K domain-filtered PubMed corpus, the Phase-7 generator pipeline (and its generate latency), and verifier–human agreement. PyPI publication is deliberately deferred.

Not medical advice — a research literature assistant.

See BENCHMARKS.md and CHANGELOG.md.

v0.10.0 — Phase 11: evaluation campaign + error attribution

The definitive measured story of the whole system.

• Error attribution (oracle-substitution): blame each wrong answer on the earliest pipeline stage (retrieval/rerank/generation/verification) an oracle repairs — the attribution pie.
• PubMedQA headline scorer (yes/no/maybe accuracy).
• docs/design/error-attribution.md (method + pending real results).

The real test-split campaign is run once, at the end, with all trained artifacts.

v0.9.0 — Phase 10: serving & performance

The whole pipeline behind one API.

• FastAPI: /health, /passages, /ask, SSE /ask/stream, /metrics; per-stage latency instrumentation (StageTimer).
• Docker: Dockerfile + docker-compose.yml (api + demo UI + Postgres profile) and a zero-framework demo/index.html (clickable PubMed citations, GROUNDED/ABSTAIN badge, disclaimer).
• int8 + speculative decoding already exist in Zenith. Latency numbers are earned under real load.

v0.8.0 — Phase 9: abstention & calibration

The system knows what it doesn't know, measurably.

• Gate 1 RetrievalGate: abstain on low retrieval confidence (top score + top-1/top-k margin), before generation.
• Gate 2 answerability classifier: Polaris pair head (num_classes=2) over the question + top passages.
• Risk–coverage: risk_coverage_curve / operating_point (~80% coverage target, ADR-0007). Off-domain/personal-advice questions abstain.

v0.7.0 — Phase 8: faithfulness verifier (NLI)

The system's conscience — no unverified sentence reaches the user.

• NLI verifier: Polaris SentencePairClassifier (num_classes=3, entail/neutral/contradict) from the Stage-0 trunk; 3-class training; artifact.
• verify_grounded_answer: per-sentence NLI(cited span → sentence) with citation precision, citation recall, and hallucination-rate metrics; grounded iff every content sentence is cited and entailed.
• Fail-safe ladder (answer_with_verification): generate → verify → extractive fallback → abstain.

v0.6.0 — Phase 7: grounded generator (Zenith)

Fluent cited answers, with the extractive path retained as fallback forever.

• Consumes Zenith 1.1.0: constrained-decoding hook, AllowedTokens, Generator.abstained, instruct.grounded.
• Citation-constrained decoding: a [n] may only reference a retrieved passage — malformed citations are structurally impossible (ADR-0006).
• generation.: Zenith DecoderLM artifact (LoRA-aware save/load), grounded-SFT example builders, LoRA SFT via CausalLMTrainer, and answer_grounded (retrieve → constrained decode → parse [n] to PMIDs, or abstain).
• CLI meridian ask --answerer generated --generator DIR; extractive stays the default.

Answer quality pending the real grounded-SFT run.

v0.5.0 — Phase 6: cross-encoder reranker

Precision at the top of the ranking, where the generator reads.

• Consumes Polaris 1.4.0: SentencePairClassifier (num_classes=1), collate_pairs ([CLS] q [SEP] passage [SEP]), train_bpe special-token reservation, native tokenizer save/load.
• Tokenizer now reserves // so one vocabulary serves the MLM, embedder, and pair models (resolves ADR-0003's deferral).
• meridian.reranker: SentencePairClassifier config/artifact, pointwise pair-sample builders, and BCE train_reranker (init from the shared Stage-0 trunk).
• RerankingRetriever: rerank a base retriever's top-N candidates with the cross-encoder; config-gated via 'meridian ask --rerank --reranker DIR' and evaluate.py --rerank (default off). Composes with bm25/dense/hybrid.

Verified end-to-end offline. nDCG@10 lift and rerank latency pending the real training run (adopt only if it improves — honesty gate).

v0.4.0 — Phase 5: hard negatives + hybrid retrieval

The dense-retrieval quality phase.

• Hard-negative mining: pool top non-relevant passages from BM25 + dense into (anchor, positive, negatives) triples for retriever v2 (the in-batch vs +hard-neg ablation), reusing Polaris 1.2.0's hard-negative InfoNCE support.
• Hybrid RRF: HybridRetriever fuses BM25 + dense by reciprocal rank fusion (no score normalization); meridian ask --retriever hybrid.
• Failure taxonomy: eval.sample_misses + a committed taxonomy scaffold (vocabulary gap / granularity / annotation noise) feeding the Phase 11 attribution study.

All implemented and verified offline; retriever-v2 delta, hybrid row, and audited miss counts pending the trained retriever on the frozen dev split.

v0.3.0 — Phase 4: ANN index from scratch (IVF + HNSW)

Sub-linear vector search, built from scratch and benchmarked against the brute-force ground truth. No external ANN library.

• IVFIndex: seeded k-means (k-means++ init) → nlist cells; nprobe search with exact rerank in probed cells (nprobe=nlist recovers brute-force recall).
• HNSWIndex: from-scratch hierarchical navigable small-world graph (M / efConstruction / efSearch); property-tested to ≥0.90 recall vs brute force across seeds.
• Backend switch by config: meridian ask --ann {none,ivf,hnsw}, and in evaluate.py; DenseRetriever now searches any VectorIndex.
• Benchmark + ADR-0005: committed recall/latency figure — HNSW reaches recall@10 ≈ 0.999 at efSearch=16, below brute-force latency (synthetic N=2000). Default backend: HNSW (reaffirmed from real embeddings before v1.0).

Real recall/latency/RAM on the corpus pending the Phase-3 embeddings.