v0.17.0 — the comparison cycle, answered by measurement

Macrame 0.17.0 — the ledger meets its reference

Crate macrame-db 0.17.0, wheel 0.17.0. Schema v18 → v19 (0.16.1, unchanged since);
snapshot container v4 and its v3 header unchanged; MSRV 1.88 unchanged; DbError unchanged
at 42 variants. Public surface 1,763 → 1,781 items. Decisions D-274 … D-277 — four
of them across the three releases from 0.16.1 to 0.16.3, on dev/0.17.0 before this cut.

This cycle began with a table rather than a review: the same correctness battery run
against ladybug 0.20.3, an HNSW-based graph engine, on the same box — macrame 0.16.0
plugged in as it stood, then 0.16.1, 0.16.2 and 0.16.3 in sequence as they landed
(D-274…D-277).
The plan document (docs/macrame-perf-diagnostics.md) is the cycle's own acceptance list:
every item in it is closed with a number, a landed decision, or a named upstream boundary,
and the plan's status header says which is which.

The finding the cycle exists for is that the 0.15.0 review was right that something was
wrong and nobody had measured where. The edge bulk was superlinear at 4× per doubling
(8.6–12 s for 16,000 edges); the vector build was superlinear in dimension (4× dims, 7.8×
time); the footprint was 3× its reference; and the comparison table's own ladder cells
carried a scaling figure that turned out to belong to a different data shape than the one
the harness ships. Every one of those is now either fixed, shipped as an opt-in recipe, or
closed with the measurement that says the remaining gap is not this crate's to fix.

Breaking changes

None. Every surface change in the cycle is additive, and every behavioral change is opt-in
by signature: upsert_embeddings, bulk_import and every shipped read path behave exactly
as 0.16.0 did. The one schema change is the v19 rung (0.16.1, D-274), which is
forward-only in the same sense as every rung before it — an older crate cannot open a
v19 file — and is trigger-only: no table gains or loses a column, so no ETL is owed.

The three new CommandKind variants (DropEmbeddingIndex, RebuildEmbeddingIndex,
LinksCurrentMirror) arrive at the end of the declaration order, per the persisted
contract CommandKind::index documents: no counter is renumbered, no Python histogram
axis is relabelled.

Added

• Database::bulk_embeddings[_with] (0.16.2, D-276). Load vectors without the DiskANN
  index in the way, then rebuild it in one pass. 1.4–1.8× at every width measured (2.62 s
  vs 3.78 s at dim 64; 19.7 s vs 31.0 s at 256; 39.0 s vs 56.0 s at 512; 48.6 s vs 89.2 s
  at 5,000 × 256). The trade is in the docstring: between the drop and the rebuild the
  model is unsearchable and not dimension-checked at the storage layer — the index is
  that check (D-037) — the crate-side check still applies to every row the method loads,
  and a failed or cancelled load rebuilds before it reports, so this API cannot leave a
  file disarmed.
• Database::bulk_import_deferred[_with] (0.16.3, D-277). The materialization-skipping
  bulk: drop the links_current mirror trigger, load through the same chunked path
  bulk_import uses, restore the trigger before rebuilding, and re-derive the
  projection with rebuild_current_chunked. 2.05× end-to-end at 16k random-pair edges
  (5.08 s → 2.29 s + 0.19 s rebuild). The ledger is complete at every instant of the
  window — the log mirror and the single-open guard stay up — so what lags is Doctrine
  VI's derivative projection, never the past. A failure restores and rebuilds before it
  reports; an empty load touches nothing; and a future dropped mid-load leaves the
  projection stale but auditable (audit_current) and repairable
  (rebuild_current_chunked). The shipped path never has a window; that is why this is a
  signature and not a default or a keyword.
• CommandKind::DropEmbeddingIndex, RebuildEmbeddingIndex, LinksCurrentMirror
  (0.16.2/0.16.3) — the three DDL turns the two recipes own, each exempt from the chunk
  budget by D-233's stated criterion (one statement, atomic by necessity), each in the
  CHUNK_BUDGET exemption table with its measured hold, and each carrying the Rust and
  Python metric surfaces.
• The WAL recipe is documented into the docstrings (0.16.1, D-275):
  wal_autocheckpoint = 10_000 at open is worth 1.36× on a random-order 16k bulk, and
  the knob is a caller's choice at open — no default moved (D-155's rule).
• The measurement harness is checked in: benchmarks/diagnostics/spike_ladder.py
  (the edge ladder, concept contrast, call-size sweep, and the WAL arms behind
  --edges-wal/--edges-hub), examples/vector_build_probe.rs and
  examples/bulk_skip_probe.rs — each rerunnable, each the evidence for its register
  entry.

Fixed

• The fresh-file bulk import was superlinear, and the fix is one character plus a rung
  (0.16.1, D-274). On any database with rows and no statistics — which is exactly the
  state of a file mid-import — the planner served trg_links_single_open's EXISTS probe
  from idx_lc_lineage_cut with only branch_id bound: a whole-lineage scan per trigger
  firing, O(rows) per row. The plan pin had been green throughout because its fixture was
  empty — D-273's partial-index lesson arrived at from the other side. The branch predicate
  carries a unary + now (D-250's idiom), schema v18 → v19, per-row cost flat, 16k edges
  8.6–12 s → 2.22 s (3.9×).
• The plan registry is swept across the statistics-free state (0.16.1, D-275).
  every_justified_index_is_the_one_the_planner_picks_without_statistics pins every
  justified index as the one the planner picks on a populated, sqlite_stat1-less
  database, with a fixture guard asserting the statistics table's absence. It passes
  clean: D-274 was the only instance in the current registry.
• The 0.15.0 review's A-6, closed across 0.15.22/0.15.23 before this cycle began
  (D-264/D-265, carried in the 0.16.0 note): a hot read loop no longer starves close(),
  and close(timeout=…) bounds the wait with CloseTimeoutError.

Performance

Against ladybug 0.20.3 on the same box (the comparison that opened the cycle, updated as
each release landed):

Workload	macrame 0.16.0	macrame 0.17.0	ladybug 0.20.3
Write 2k docs + 4k edges	1,335 ms	927 ms (bulk_import, 0.16.1's fix)	1,010 ms
Edge ladder 16k (near-chain)	29.2 s	2.32 s, per-row flat 0.135→0.148 ms	not measured
Edge ladder 16k (random-pair)	—	3.40 s with the WAL recipe; 2.48 s with bulk_import_deferred	—
Vector build 2k×256	82 s at 5k×256	19.7 s at 2k×256 (bulk_embeddings)	3.5 s
Vector top-10 @512	16.4 ms	9.9 ms (99% is the DiskANN search itself)	4.5 ms
Keyword, full 20k-row match	—	~32 ms (all 20,000 rows)	47 ms (18k rows)
Traversals	0.1 ms	0.12–0.23 ms, flat to 5 hops	3.8–5.9 ms

Macrame is ahead of its reference on writes of every size, on every traversal, and on
keyword search; behind on vector ingestion by 5.6–7× at the best available recipe — and
that gap is priced and named (below) rather than rounded into a claim.

Measurements that closed questions

• The ladder residual is the data shape, not a defect (plan §9.2, 2026-09-11). The
  near-chain ladder the harness ships is already linear post-D-274. The "~2.5× per 2×" the
  comparison table carried belongs to the random-pair shape, where it decomposes into the
  WAL recipe (1.36× at 16k), the writer cache (~5%, saturating at 64 MiB), and the
  maintained materialization — which is F1's lever, now shipped (D-277).
• The footprint is two-thirds DiskANN (plan §9.4). 234 MB at the comparison's exact
  shape (20k docs, 40k edges, 20k×64 vectors), of which 156.4 MB is the DiskANN index
  shadow for 5.3 MB of raw vectors — 30× the vector bytes, the physical counterpart of
  the build-cost finding. The ledger's own ~50 MB is roughly the reference's size.
• The keyword cell closes with macrame ahead (plan §9.5), plus the recording that the
  first pass printed 32,237 ms — a formatting bug, not a slow query — and the bisect that
  caught it is part of the cell.
• Two candidate optimizations closed by measurement rather than left open (D-275):
  the per-row overlap guard is 1.4 µs/row (batching saves 0.6), and a multi-row INSERT
  is 7% slower than per-row executes on the edge path.

Documentation and gates

• docs/macrame-perf-diagnostics.md grew from a plan into the cycle's record: §0b (the
  execution record), §9 (the follow-on cycle, all five items closed in place), and a
  status header that says what landed and what is genuinely out of scope.
• s13-decision-register.md reaches D-277; the architecture README's revision history
  reaches 0.17.0; docs/quickref.md carries the v19 rung, the new APIs, and the two
  new decision rows; appendices.md names 1,781 items and documents both recipes in
  the _with-family note; the CHUNK_BUDGET exemption table carries the three new rows,
  kept in agreement with exempt_from_budget by the test that reads the source.
• api-review-0.17.0.md is generated and checked in: +7 distinct items, 0 removed.

Acceptance

The plan document's five follow-on items (§9), all closed:

#	Item	State
1	Vector build, 8.5–10× behind	Landed as D-276 (0.16.2): the recipe ships callable; the residual 5.6–7× is DiskANN-vs-HNSW build economics inside libSQL, and the query side is flat in corpus size and inside §9's ≤20 ms line
2	Edge-ladder residual ~2.5×/2×	Closed (§9.2): near-chain is linear; the residual is the random-pair shape — recipe 1.36×, cache ~5% saturating, rema...