Architecture

Architecture

For contributors. The one rule that governs every decision:

QQL parser knows syntax.
Source handlers know Islamic-book structure.
Repository knows storage.
FFI module knows the C ABI.

query → lexer → parser → AST → validation → source resolver → repository → serializer → UTF-8 String

The layers

Module	Knows	Never
lexer.rs	ASCII tokens, byte offsets	what a source is
parser.rs	the grammar	Surah counts, source codes, the default source
ast.rs	Query / Reference / Range	JSON
registry.rs	which codes exist, the default	book structure
sources/*.rs	one collection's structure	the filesystem, the grammar
repo.rs	reading and caching files	what any file means
search.rs, vector.rs, fulltext.rs	matching	which records exist
ffi.rs	the C ABI	query logic

The clearest consequence:

Q:500:999    parses cleanly → the Quran resolver rejects it
XYZ:1:2      parses cleanly → the registry rejects it

tests/parser.rs must pass with no data directory at all. If it ever needs one, a layer has leaked.

Things that are easy to get wrong

Adding a collection must not touch the parser. A new source is one impl Source, one registry line, one data path. If your diff reaches lexer.rs or parser.rs, the design is being violated.

The parser must not learn the default source. Reference::source is Option<String>; Registry::DEFAULT_CODE owns the answer and Context::execute substitutes it, so resolvers always see a concrete code. Inheritance across ; is the parser's job, because "reuse the previous code" needs no knowledge of what codes mean.

Reference::expand(max) is the only place that does ordering, within-reference dedup and bounds checking. Resolvers call it and must not re-implement any of the three. It is also what stops Q:1:1-4294967295 from trying to allocate four billion entries.

The AST is plain structs, never serde_json::Value. Deriving Serialize is fine; building it from JSON is not.

Repository stays schema-free. It caches Arc<dyn Any + Send + Sync> keyed by path and downcasts on read, so file schemas live next to their resolver rather than in the loader.

Contracts tests exist to pin

• Query order is preserved and never sorted. No .sort(), no BTreeSet for item expansion. The single exception is ranked search, which says so with "ranked": true.
• Duplicates go within a reference, stay across references.
• Ranges are inclusive; codes uppercase with to_ascii_uppercase; whitespace around tokens is legal.
• Every return is valid JSON, errors included. Wire codes come from an exhaustive match with no wildcard arm, so a new variant fails to compile until its code is chosen.
• Arabic passes through byte-for-byte. Never from_utf8_lossy on scripture — reject invalid UTF-8 instead.

Data traps that cost real debugging

These are recorded because each one silently returns wrong scripture rather than failing:

• by_book/ hadith files number id across all nine books at once (Bukhari 1–7277, Muslim 7278–14736, …). The per-book citation number is idInBook. Using id works for Bukhari and is wrong everywhere else. A test asserts the first canonical number opens chapter 1 for every collection that has a canonical map.
• The sequential position within a book is not a citation number. The same trap, one level out: six collections (DA RS BM AM MK SM) publish no number QQL can source, so HadithCollection::chaptered declares the axis absent and CODE::N is refused. Exposing the position would answer real citations with the wrong hadith.
• Hisnul Muslim chapters are stored out of order — array position 0 is chapter 27 — so they are found by id, never by position.
• Sunan an-Nasa'i has a chapter numbered 35.2, so chapter ids deserialize as serde_json::Number, not u32.
• The Hisnul Muslim file has a UTF-8 BOM, two objects with duplicate keys, and one misspelled field. The BOM is stripped in repo.rs (storage); the rest is absorbed by a serde_json::Map newtype with accessors — the one deliberate exception to "no Value in schemas", documented in place.
• Unnumbered chapters (introduction.json, 35b.json, 8b.json) are unreachable by the grammar, so index builders skip them. Document counts below a collection's total are correct, not truncation.

Safety

#![deny(unsafe_code)] crate-wide, with a single allow inside src/ffi.rs — about 230 lines, the entire audit surface.

• Every extern "C" function wraps its body in catch_unwind. A panic unwinding across the boundary is undefined behaviour.
• CString::into_raw out, qql_free_string → CString::from_raw back.
• Context::execute takes &mut self, so the compiler prevents concurrent use. Across the ABI it cannot, hence the documented warning.

Search internals

Three engines, chosen by the query and never by build flags. MatchKind::is_ranked() is the single place that decides which produce ranked output.

• Exact — a folded substring scan over the resolved scope. Source-agnostic: Context resolves the scope as an ordinary reference and filters.
• Vector — flat int8 cosine scan, no ANN index. The embedder is a signed hash projection duplicated in src/vector.rs and scripts/build-vectors.py — the two must stay identical or every score is wrong. 256 dimensions is measured, not guessed: at 128, collisions outranked real matches on short queries.
• Full text — tantivy. fulltext::build enumerates records by issuing ordinary queries (CODE:1, CODE:2, …) rather than reaching inside, so it cannot disagree with the resolver about numbering.

Both index formats key on (primary, number), so a scope filters during the scan and every hit resolves back through the ordinary reference path. The index never has to agree with resolvers about record shape, only numbering.

Style

• No unwrap/expect in library code; fine in tests and the CLI.
• No as casts on parsed input. Overflow is QQL_EXPECTED_NUMBER, never a wrap or a panic.
• Lexer tokens borrow &str slices from the query rather than allocating.
• Exhaustive match wherever a new variant should force a review.
• #![deny(missing_docs)].
• Dependencies stay few: serde, serde_json, and tantivy only behind a feature. CLI arg parsing is std::env::args — no clap for a handful of flags.