- Multi-db file load dropped databases — and the exit-save then made it permanent (
corm_load_file / _corm_load): loading walked the mapping by counting the file's registered handles, one _corm_load per id, each using the iterated handle as the load target. That only pairs correctly when the id list's order matches the on-disk block order, which it does not (blocks are emitted in ids_iter order, handles are registered as they are opened). Any database whose block the walk failed to reach came back as an empty map — and since corm_save() runs at process exit, a reader re-wrote the file from the emptiness, permanently truncating the block it had missed. Repro on the pre-1.4.1 library: a 3-db file opened 2 of 3 dbs empty; a 2-db file opened one empty and shrank 52 → 42 bytes on read. This is the "silently dropped" class that 1.4.0's F4 entry hit on the libstoma sidecar. corm_load_file now takes the requesting hd from corm_open and walks the mapping by each block's on-disk size prefix, so any block whose lid matches is loaded into the handle that asked for it, whatever order the file is in.
- Off-by-8 block skip (
_corm_load): the dbid-mismatch branch returned mm + size - mm_start after mm had already advanced past the 8-byte [dbid][size] block header, so the walk drifted 8 bytes too far per skipped block and read the next block's lid/size from mid-payload bytes. The resulting garbage length can walk past the end of the mapping (one 3-db repro segfaulted here). The skip now honors the on-disk size exactly.
- Bounds-checked block walk (
corm_load_file): every block's size prefix is validated (>= 8 and mm + block <= end) and a non-matching lid is skipped without entering _corm_load, so a truncated, corrupt, or foreign file stops at the mapping end instead of running off it.
corm_close + save no longer zeroes the file (corm_save_file): the file->size == 0 early return ran after open() + ftruncate(), so once every handle for a file was closed, the next corm_save() truncated the file to 0 bytes and threw away its contents. The early return now precedes both, keeping the data on disk (verified 0 → 20 bytes for a closed one-key map). Side effect: a path that never held data no longer gets a 0-byte file created for it.- Regression coverage:
test_persist (wired into make test via test.sh) drives a three-phase cross-process round trip — write, read back in a fresh process in a different order than the writer used, then save-after-close. The read phase is what the pre-1.4.1 loader fails: it drops a database and segfaults; the close phase catches the 0-byte truncation.
- On-disk format and all public API signatures are unchanged; files written by 1.4.0 and earlier load correctly.
1.4.0
- Renamed
libqmap → libcorm: the qmap_* API and headers are now corm_* (include/ttypt/corm.h, qmap.pc → corm.pc); the full test/example suite was migrated to the new name.
- The Recall Kernel (
include/ttypt/rec.h, src/rec.c) — a domain-free "filter by axis → join → rank" loop over uniform 32-bit refs:
rec_set_t: arena-backed candidate sets (rec_set_push, then rec_set_seal = sort + dedup; rec_set_intersect/_union/_subtract are sorted merge-joins; rec_set_count/rec_set_at; drain any corm handle with rec_set_fill_corm_iter).rec_rank_t: bounded top-k min-heap with a score threshold — rec_rank_new(top_k, min_score)/rec_rank_push (O(log k))/rec_rank_sorted (best-first, ties by ascending ref); scoring via rec_score_fn.- Refs are
rec_ref_t (uint32_t), are never interpreted by the kernel, and never an axis's internal key.
- Exact vs approximate fills: an approximate axis (ANN/semantic top-m) must declare itself —
rec_set_set_approx/rec_set_approx/rec_set_recall_bound — and joins propagate it (intersect/union → approximate when either operand is, bound = min; subtract keeps the left operand's flag), so intersecting with an approximate set bounds final recall.
rec_query engine + axis plugin registry — registry-driven query composition running wholly inside libcorm with zero axis dependencies: rec_axis_register/rec_axis_set_ctx, rec_axis_t (name/fill/rank/ctx/decode), and rec_query_run with rec_join_t (AND/OR/NOT), top_k/min_score, and an optional consumer_score callback fed rec_axis_score_t[] per surviving ref. dlopen-loading of axis plugins happens on the consumer/CLI side only.- Rank composition: the first rank-capable axis in query order wins (preorder in
corm_expr_eval, kernel fallback in rec_query_run); labeled instances label:axis rank by the per-ref MAX; --rank=A (or scoped --rank@A) pins the ranker to one labeled instance; --score multi-ranker combining deferred.
- New kernel features: proximity / approximate ranking, the
QM_RANGE_GE range-eval flag, and corm_get_ktype to ask a handle's key type.
- corm CLI composition (
src/corm.c): -X expression trees over a @-roster of axes (dlopen by name from $CORM_AXIS_PATH), -p "<whole record string>" fan-out to every bound axis, --params per-axis retrieval grammar, -g/-m/-c read-back rendering, and per-axis CLI long-options via rec_axis_cli_options/rec_axis_config_arg (scoped --name@label, precedence leaf spec > @label > @axis > unscoped > env). Typed param parsing rides the rec_cli_* helpers (rec_cli_int/_uint/_float/_size/_str_dup/_str_set + _b variants) backed by rec_spec_scan's key=value spec grammar (quoted strings included).
- Axis plugin conventions (optional, CLI-specific, never declared/exported by libcorm):
rec_axis_open(spec) (opaque spec → store ctx), and the Phase 2A store half rec_axis_store/rec_axis_store_typed/rec_axis_unstore/rec_axis_readback — the consumer passes (ref, whole-string value) blindly; each axis parses in its own grammar; unstore is idempotent (delete cost ∝ entries the ref owns, never O(store)).
- Test & bench suite:
rec_test (sets/joins/ranks + 1C exactness matrix), bench_rec (kernel merge-joins vs hand-rolled parity), rec_axis_test/rec_axis_store_test/rec_cli_test/rec_axis_bench, a mock two-axis plugin (librec_axis_mock.c) proving the dlopen shape end-to-end, plus the new test-cli.sh, test-fanout.sh, test-mm.sh, test-real.sh, test-roster.sh, and test-shortcircuit.sh shells (with in-repo plugin-stub starters librec_axis_{fold,plain,probe,stub,zed}.c).
- Rename + axis-boilerplate passes above all ride on
rec_ref_t = uint32_t; refs and axis params are CLI parameters everywhere.
-X grammar pinned: verified against the built 2B-3 parser (mm-plan PHASE-2-CLI.md D10) — quote + whole-string VALUE semantics, unary NOT with the use EXCEPT hint, uppercase reserved keywords. Two behaviors are now pinned by gate rows: EXCEPT chains left-to-right (except-chain, test-cli.sh), and the first rank-capable leaf in preorder wins even with two rankers in one expression (stoma-over-sepal, test-real.sh).- F4 — duplicate-open fix:
corm_open now aliases the live handle when the same (file, map) is opened twice with the same key/value shape (record type, key/value types, table mask), while still registered in the file's ids. Previously the second handle orphaned the first and its as-of-open copy won the exit-save — libstoma's sidecar-scan mirror-open of an already-held primary silently dropped -p seeds (pre-existing roster) and -d forgets. Real-file TDD via test-mm.sh (RED-A/RED-B) → green.
- 2B-5 mm dialect (mm-plan
2B-5-IMPLEMENTATION.md): pi-mm recipes as documented CLI invocations — explicit-ref store (-p REF:"<DATE>:<TEXT>"), bounded-window joint ∧ stoma composed search, idempotent roster-backed forget, and the enumerate+forget reset loop (no single-flag reset exists; the -1 empty sentinel is skipped). test-mm.sh gate wired into make test.
- CLI tuning:
QDBE_MASK shrinks 32767 → 4095 (D11 — an initial hint, auto-grow); new CORM_MASK env override (validated 2^n-1) shared by the CLI opens, the test plugins, and libstoma's sidecar rebuild so co-opened files match. gen_put/gen_del/gen_del_all now return the op exit status.
- 2B-4 write fan-out (mm-plan
2B-4-IMPLEMENTATION.md): -p/-d/-D write to the union of {primary} ∪ {@} ∪ {target} — composed mode only, classic path byte-identical. Whole -p payloads fan out as (ref, blob, len, qtype) via the additive rec_axis_store_typed when vtype != CM_STR (D12), else string rec_axis_store; binary-payload-on-text-only axes loud-skip; -d/-D collapse to idempotent rec_axis_unstore; ref operands are literal u32 or primary reverse-view names; loud partials (attempt-all/report-all/nonzero). A per-slot store-capability table is built at bind time (read-only detection); rec_axis_readback stays dlsym'd with no CLI surface.
- Test plugins: write-capable
librec_axis_fold (.wr stash feeds fill) + string-only librec_axis_plain; test-fanout.sh gate + dlopen-only tests/fanout_verify.c probe.
Verification
gpg --verify CHECKSUMS.asc
sha256sum -c CHECKSUMS
