fix(catalog): index physicalDescription, personality, role, motivations, significance in search_tsv#546
Merged
Merged
Conversation
…ns, significance in search_tsv Catalog full-text search currently misses bible-promoted characters' main narrative text — search_tsv only included description/notes/background/summary and bypassed the character canon fields physicalDescription + personality (and the type-specific role/motivations/significance fields). Postgres can't ALTER the expression of a STORED generated column, so the fix is DROP + re-ADD inside the idempotent ensureSchema DDL block. The block already runs on every boot (PR #542 made it unconditional), so the ALTERs apply to existing installs automatically and are no-ops on fresh installs because the new CREATE TABLE no longer declares search_tsv inline. Mirrored in server/scripts/init-db.sql so fresh-install schemas match what ensureSchema produces. PORTOS_SCHEMA_VERSIONS.catalog bumped to 2 — older peers still on 1 get a 412 from /api/catalog/sync/apply against newer peers, preventing them from pushing pre-expansion-shape rows that would mismatch the indexed expression.
…sion detected Previously ensureSchema() unconditionally DROP+ADDed catalog_ingredients.search_tsv on every boot to swap the generation expression. Postgres treats that as a table rewrite under an AccessExclusive lock plus a GIN index rebuild — fine the first time but pure waste on every subsequent boot of an already-v2 install (and ensureSchema fires once from memoryBackend init and once from the catalog backfill chain). Gate the DROP on a pg_attrdef inspection: only drop when the existing generation expression is missing 'physicalDescription' (a v2-only token). Fresh installs skip the drop because the column doesn't exist yet; already-v2 installs skip it because the expression already matches; only an upgrading v1 install pays the rewrite. ADD COLUMN IF NOT EXISTS still does the right thing in all three cases. Mirrored byte-for-byte into init-db.sql.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Catalog full-text search (
catalog_ingredients.search_tsv) currently misses bible-promoted characters' main narrative text. The STORED generatedtsvectoronly indexesname(weight A) andpayload->>description|notes|background|summary(weight B), so a search for "tall" or "introverted" against a character whose data lives inpayload.physicalDescriptionandpayload.personalityreturns no hits.This PR expands the indexed expression to cover the character canon fields plus the type-specific
role/motivations/significancefields used by place/object/concept payloads:payload->>'description'payload->>'physicalDescription'(new)payload->>'personality'(new)payload->>'background'payload->>'summary'payload->>'notes'payload->>'role'(new)payload->>'motivations'(new)payload->>'significance'(new)Schema migration
Postgres can't
ALTERthe expression of aSTOREDgenerated column, so the column isDROPped and re-ADDed. Done inside the existingcatalogDDLblock inserver/lib/db.js, which already runs on every boot (PR #542 made it unconditional). The whole block is idempotent: a fresh install'sDROP IF EXISTSis a no-op (column didn't exist) andADD IF NOT EXISTScreates it; an existing v1 install'sDROP IF EXISTSremoves the narrow expression andADD IF NOT EXISTSinstalls the expanded one. The expandedidx_catalog_ing_ftsGIN index rebuilds automatically against the re-added column.Mirrored in
server/scripts/init-db.sqlso fresh-install schemas land in the same end state asensureSchemawould produce (matches the[catalog-ddl-drift-test]PLAN invariant).Wire-protocol version bump
PORTOS_SCHEMA_VERSIONS.catalogbumped from1→2. The existing per-category gate inserver/services/catalogSync.jsapplyRemoteChangesenforces the contract:catalog(not ahead) → accepted. The v2 receiver re-derivessearch_tsvlocally via the STORED expression.CATALOG_SCHEMA_VERSION_AHEAD, preventing pre-expansion-shape rows from mismatching the indexed expression on the older install.PLAN entry:
[catalog-fts-character-fields].Test plan
cd server && npm test— 8204 pass / 7 skip / 0 failPORTOS_SCHEMA_VERSIONS.catalog === 1— the live registry isn't asserted at a specific catalog value, so no test update neededserver/lib/db.jscatalogDDL stays idempotent (DROP IF EXISTS + ADD IF NOT EXISTS)server/scripts/init-db.sqlmirrorsensureSchema(no drift)