Problem
jobs/flowsheet-etl/job.ts (bulk + incremental) inserts rows into flowsheet but does not trigger metadata enrichment for them. They rely on the daily cron to catch up. Per Epic C #1 (the metadata_status column), this means ETL-inserted rows have metadata_status DEFAULT 'pending' and the consumer should pick them up via CDC.
Today the ETL also doesn't broadcast liveFs events because it inserts via raw db.insert(flowsheet) which bypasses the BS internal-route's updateLastModified + broadcast. So even after Epic C, the ETL-inserted rows wouldn't reach the consumer via CDC unless we either (a) emit CDC events properly (already happens via PG triggers — confirm), or (b) the consumer doesn't depend on the broadcast and only on the PG trigger.
Verification + adjustment
Out of scope
- Performance: bulk ETL inserts millions of rows on the rebuild path. The consumer can't enrich those at 50 req/min; the cron is the safety net there.
Acceptance
Related
Problem
jobs/flowsheet-etl/job.ts(bulk + incremental) inserts rows intoflowsheetbut does not trigger metadata enrichment for them. They rely on the daily cron to catch up. Per Epic C #1 (themetadata_statuscolumn), this means ETL-inserted rows havemetadata_status DEFAULT 'pending'and the consumer should pick them up via CDC.Today the ETL also doesn't broadcast
liveFsevents because it inserts via rawdb.insert(flowsheet)which bypasses the BS internal-route'supdateLastModified+broadcast. So even after Epic C, the ETL-inserted rows wouldn't reach the consumer via CDC unless we either (a) emit CDC events properly (already happens via PG triggers — confirm), or (b) the consumer doesn't depend on the broadcast and only on the PG trigger.Verification + adjustment
db.insert(flowsheet).values(...)triggers the same CDC PG trigger (shared/database/src/migrations/0045_cdc_notify_triggers.sql) that webhook/controller inserts do. (Yes, the trigger is on the table, not the path — so all writes notify. But verify.)updateevent), wire it.Out of scope
Acceptance
metadata_status='pending'lifecycle as other writes.Related
jobs/flowsheet-etl/job.ts