Add the note catalog: typed rows, and queries by path and ULID (CRDT step 3) - #131
Merged
Conversation
…step 3) One row per note in metadata.db, keyed by ULID: path, merge policy, state, the content hash and the size, mtime, and sketch beside it, plus this device's view of the seed claim. The table is bf_catalog, and the queries are byPath and byUlid. Splits the row types from their storage. catalog.dart holds MergePolicy, NoteState, and CatalogRow and imports neither dart:io nor sqlite3 — crdt_lf itself is pure Dart, so a row compiles on every platform even where the table cannot exist. catalog_io.dart is the SQLite half. The seam exports the types unconditionally and the storage only on dart:io. merge_policy is an open enum rather than a boolean, and both enums are stored as their names rather than as ordinals: an ordinal re-numbers itself the moment vector ink is added, silently reinterpreting every row already on disk. An unrecognised extension derives blobLww, defaulting toward the recoverable failure — last-writer-wins on text loses an edit that still exists in the loser's history, while character-merging a PNG produces a file nobody can recover. Uniqueness on path is a partial index over non-tombstoned rows rather than a column constraint. A tombstone keeps the path it died at, so deleting a note and later creating another at that same path is ordinary use that a total UNIQUE would reject. The partial index states what is actually true — a live, history-pending, or unavailable note owns its path exclusively — and is what lets byPath return one row rather than a list. byUlid still finds tombstones, because a note's identity outlives its file. A row that cannot be read is refused rather than half-read, matching how the store already treats a schema version or a peer identity it cannot parse. Half a seed claim is refused too: reading a lone seeded_by as "nobody has seeded this" would invite a second device to seed a document that already has a history, which is the one thing note identity must never allow. The seed claim is an OperationId. That is exactly the (seeded_by, seed_hlc) pair the design names, and its compareTo is exactly the locked tiebreak comparator, so contested claims resolve through the library's ordering instead of a second one written here. Moves MetadataDatabaseException and EngramStoreCollisionException into store_exceptions.dart so catalog_io.dart can raise them without importing the file that imports it. Both platform builds now re-export one pair of classes rather than declaring look-alikes a catch clause would tell apart; existing imports are unaffected. Corrects the design's diagnostic example, which still named an unprefixed `catalog` table from before the bf_ rule. No user-facing change: the catalog has no UI, so docs/manual-test-plan.md is untouched — no cases added, changed, or invalidated. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Sep 5, 2026
|
Coverage after merging worktree-crdt-catalog into main will be
Coverage Report |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds the note catalog: one row per note in
metadata.db, keyed by ULID,carrying
path,merge_policy,state,materialized_hash,size,mtime_utc,sketch, and this device's view of the seed claim. The table isbf_catalog; the queries arebyPathandbyUlid.The row types are split from their storage.
catalog.dartholdsMergePolicy,NoteState, andCatalogRowand imports neitherdart:ionorsqlite3—crdt_lfitself is pure Dart, so a row compiles on every platformeven where the table cannot exist.
catalog_io.dartis the SQLite half. Theseam exports the types unconditionally and the storage only on
dart:io.merge_policyis an open enum rather than a boolean, and both enums arestored as their names rather than as ordinals: an ordinal re-numbers itself
the moment vector ink is added, silently reinterpreting every row already on
disk. An unrecognised extension derives
blobLww, defaulting toward therecoverable failure — last-writer-wins on text loses an edit that still exists
in the loser's history, while character-merging a PNG produces a file nobody
can recover.
Uniqueness on
pathis a partial index over non-tombstoned rows ratherthan a column constraint. A tombstone keeps the path it died at, so deleting a
note and later creating another at that same path is ordinary use that a total
UNIQUEwould reject. The partial index states what is actually true — alive, history-pending, or unavailable note owns its path exclusively — and is
what lets
byPathreturn one row rather than a list.byUlidstill findstombstones, because a note's identity outlives its file.
A row that cannot be read is refused rather than half-read, matching how the
store already treats a schema version or a peer identity it cannot parse. Half
a seed claim is refused too: reading a lone
seeded_byas "nobody has seededthis" would invite a second device to seed a document that already has a
history, which is the one thing note identity must never allow.
The seed claim is an
OperationId. That is exactly the(seeded_by, seed_hlc)pair the design names, and itscompareTois exactly the lockedtiebreak comparator, so contested claims resolve through the library's
ordering instead of a second one written here.
Two judgment calls worth a look
store_exceptions.dart.MetadataDatabaseExceptionandEngramStoreCollisionExceptionmoved out ofmetadata_db_io.dartsocatalog_io.dartcan raise them without importing the file that imports it.Both platform builds now re-export one pair of classes rather than declaring
look-alikes a
catchclause would tell apart. Existing imports areunaffected — the io file re-exports them.
byPathexcludes tombstones. Defensible either way; the alternative isreturning them and making every caller filter. Excluded here because
returning a tombstone is what would let the next scan resurrect dead history
under unrelated content.
Scope
Step 3 as the plan specifies it: the table, a typed row, and the two queries.
The scan that populates it is step 11, the materializer that sets
materialized_hashis step 8, and the sketch is computed in step 11 — thisstep only stores them.
Tests
catalog_test.dartcovers the value types (policy derivation includingdotfiles, multiple extensions, and a dot in a directory name; both parsers;
row equality field by field, with byte-wise sketch comparison).
catalog_io_test.dartcovers the table (every column round-tripping, UTCnormalisation of
mtime, whole-row upsert semantics, the path-uniquenessrules in all four states, and every refusal path). The plan's named test —
select path, ulid from bf_cataloganswering on its own — is in thethe human-readable claimgroup.New files are at 100% line coverage; the tree is at 98.50%.
Docs
Corrects the design's diagnostic example, which still named an unprefixed
catalogtable from before thebf_rule landed in step 2.docs/manual-test-plan.mdis untouched: the catalog has no UI, so nocases were added, changed, or invalidated.
🤖 Generated with Claude Code