v0.3.0 — Darimi, embeddings, semantic search, dedup

What's new since v0.2.0

New collection

• Sunan ad-Darimi — 2,757 hadiths scraped from sunnah.com, machine-translated to English via Gemini Flash (~30k → 39k total hadiths across 8 collections).

Vector search

• gemini-embedding-001 (768-dim) embeddings for every hadith, indexed via pgvector HNSW.
• /api/search/semantic?q=... — cosine similarity search; finds hadiths by meaning ("fasting on Mondays" surfaces relevant Ramadan/voluntary-fast hadiths even without exact word matches).
• Keyword/Semantic toggle in the UI with per-result relevance score.

Narrator data quality

• Dedup pass: 84 strict-match narrator merges with audit table for rollback.
• Ibn Abbas / al-Abbas fix: 2,313 chain rows re-pointed to the correct sahabi. Top-10 sahaba now matches classical history (Abu Hurayra → Aisha → Ibn Abbas).
• Reliability scoring: 10-rank Ibn Hajar marātib via Arabic keyword matching; per-narrator aggregate 0–1 score shown as HSL gradient.

Hadith classification

• All 39,058 hadiths now tagged with hadith_type (marfu / mawquf / maqtu / mursal / balagh / fatwa) via heuristic Arabic-text scanner.
• Edit UI: click the type badge in the detail panel to correct misclassifications.

Schema additions

• hadiths: hadith_type, slug (e.g. Bu.1.1, Ma.4.2, Da.1.650), external_ref, embedding vector(768)
• narrators: reliability_score
• jarh_tadil.evaluation_type expanded from 3 → 10 ranks
• New tables: narrator_merges, chain_refixes (audit logs)

Database asset

hadith_db.sql.gz (136 MB) — full Postgres dump. Restore with:

createdb hadith
gunzip -c hadith_db.sql.gz | psql -d hadith

Note: the embedding column requires the pgvector extension installed beforehand:

sudo apt install postgresql-16-pgvector
sudo -u postgres psql -d hadith -c "CREATE EXTENSION vector;"

Scale

• 11,479 narrators (Tahdhib al-Kamal)
• 39,058 hadiths across 8 books (Kutub al-Sittah + Muwatta + Darimi)
• 175,818 chain links, ~95% matched to narrators
• 28,159 jarh/ta'dil evaluations across 7,528 narrators
• 39,058 hadiths × 768-dim embeddings

Co-Authored-By: Claude Opus 4.7 (1M context)

v0.2.0 — Kutub al-Sittah hadith + narrator chains

34,441 hadith from the Kutub al-Sittah with narrator chain matching.

Includes:

• 11,562 narrators from Tahdhib al-Kamal
• 34,441 hadith (Bukhari, Muslim, Abu Dawud, Tirmidhi, Nasa'i, Ibn Majah)
• 175,818 chain links (98.4% matched to narrator records)
• 28,159 jarh/ta'dil evaluations
• 74,656 teacher links, 90,456 student links
• Persistent narrator name cache (23,653 entries)

Restore with: gunzip -c hadith_db.sql.gz | psql -U hadith hadith

v0.1.0 — Tahdhib al-Kamal narrators

11,562 narrators from Tahdhib al-Kamal (al-Mizzi, d. 742 AH) extracted via Gemini 3 Flash.

Includes:

• 8,113 main narrator biographies
• 262 female narrators
• 2,872 cross-references
• 74,656 teacher links
• 90,456 student links
• 28,159 jarh/ta'dil evaluations
• 15,544 name aliases

Restore with: gunzip -c hadith_db.sql.gz | psql -U hadith hadith