Skip to content

v1.2.9: Mirror schema v2: chronological date sort and index-backed category filter

Choose a tag to compare

View all tags
@cyanheads cyanheads released this 29 May 07:08
· 14 commits to main since this release
v1.2.9
This tag was signed with the committer’s verified signature.
cyanheads Casey Hand
SSH Key Fingerprint: lkZCAfVCK5qsrcvEYCs0BnVRDCX7pKsdCZkw5yYKMuw
Verified
Learn about vigilant mode.
583283f
This commit was signed with the committer’s verified signature.
cyanheads Casey Hand
SSH Key Fingerprint: lkZCAfVCK5qsrcvEYCs0BnVRDCX7pKsdCZkw5yYKMuw
Verified
Learn about vigilant mode.

Mirror schema v2: chronological date sort and index-backed category filter

Two coordinated fixes to the OAI-PMH mirror (both gated on the schema v1→v2 version bump). Existing mirror databases are upgraded in-place on first open — no re-harvest required.

Fixed:

  • sort_by submitted/updated returning wrong order (#18) — OAI version dates (RFC 2822) and YYYY-MM-DD datestamp fallbacks now normalized to ISO 8601 in rawToRow before persisting. ORDER BY is now chronological, not lexicographic over the weekday prefix.

  • category-only queries taking ~2 minutes (#19) — paper_categories(category, paper_id, updated) junction table with composite index on (category, updated) replaces unindexable LIKE queries on the space-joined categories column. COUNT and membership lookups are now index-backed; ordered single-category browse pages off the composite index with no papers scan.

Changed:

  • MIRROR_SCHEMA_VERSION 1→2. MirrorStore.open runs batched, resumable backfill when storedVersion < 2 (5 000-row batches, per-batch commits, schema_version written only after full completion).

256 tests pass; bun run devcheck clean.

Assets 3
Loading