feat(seed-contract): PR 2a — runSeed envelope dual-write + 91 seeders migrated

[koala73]

Summary

Full Unified Seed Contract PR 2 — writers, readers, bundles, direct writers, all in dual-write mode. Builds on PR #3095 (foundation).

What's in this PR (8 commits)

1. runSeed contract mode + 91 seeders migrated — every scripts/seed-*.mjs exports declareRecords and writes {_seed, data} envelope to canonical key alongside legacy seed-meta:*.
2. Internal reader unwrap — readSeedSnapshot, verifySeedKey, redisGet, 18 seeders' local helpers, seed-forecasts pipeline batch, seed-energy-spine redisMget. Addresses both P1 findings from review.
3. External reader unwrap — server/_shared/redis.ts (getRawJson, getCachedJson, getCachedJsonBatch, inherited by cachedFetchJson), api/_upstash-json.js (readJsonFromUpstash → covers all api/mcp.ts tool responses), api/bootstrap.js batch reader.
4. 3 more server-side readers unwrap — resilience/v1/_shared.ts + get-resilience-ranking.ts + infrastructure/v1/_shared.ts:mgetJson.
5. consumer-prices-core/src/jobs/publish.ts — 5 canonical keys (overview, movers 7d/30d, freshness, categories 7d/30d/90d, retailer-spread, basket-series) now produce envelopes.
6. scripts/ais-relay.cjs — 32 canonical-key write sites produce envelopes via envelopeWrite(key, data, ttl, meta) helper.
7. 15 seed-bundle-*.mjs — 54 sections across 12 bundles declare canonicalKey alongside seedMetaKey. _bundle-runner.mjs prefers canonicalKey, gates on envelope _seed.fetchedAt directly.
8. Tests — new tests/seed-utils-envelope-reads.test.mjs (7 cases); updated tests/cross-source-signals-regulatory.test.mjs + tests/ucdp-seed-resilience.test.mjs for static-analysis pattern changes.

Verification

• npm run test:data — 5307/5307 pass
• npm run typecheck:all — clean
• node --check clean on every modified script
• Contract conformance: 84/86 seeders full descriptor (2 pre-existing soft-warns)
• Post-sweep grep for unwrapped JSON.parse(redis_result) in api/server — no regressions remain
• Staging dry-run: deploy 2-3 seeders to Railway cron, verify envelope shape + legacy seed-meta both populated

Dual-write invariant (PR 2 phase)

Every seeder now writes:

• Envelope to canonical data key (new, source of truth going forward)
• Legacy seed-meta:<domain>:<resource> (preserved, keeps api/health.js + any unmigrated reader happy)

Every reader (getCachedJson, readJsonFromUpstash, readSeedSnapshot, etc.) now unwraps envelopes automatically — legacy bare-shape values pass through unchanged.

What PR 3 does (not in this PR)

• Stop writing legacy seed-meta:* keys.
• Collapse api/health.js 5-registry system (SEED_META, ON_DEMAND_KEYS, EMPTY_DATA_OK_KEYS, STANDALONE_KEYS, BOOTSTRAP_KEYS, CASCADE_GROUPS) to single SEEDS registry. Deferred because it requires switching health from STRLEN to GET on ~100 data keys (operational risk: doubles Upstash payload per check). Cleaner once legacy is gone.
• Remove fallback branches in readers.
• Delete writeSeedMeta / writeFreshnessMetadata helpers.

Test plan

• Unit tests pass
• Staging deploy & verify /api/health reports no regression
• Spot-check envelope shape: economic:fsi-eu:v1, climate:zone-normals:v1, wildfire:fires:v1, market:commodities:v1:..., consumer-prices:overview:ae
• Verify api/bootstrap + api/mcp responses contain no _seed field

Plan: docs/plans/2026-04-14-002-fix-runseed-zero-record-lockout-plan.md

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
worldmonitor	Ready	Preview, Comment	Apr 15, 2026 4:56am

[greptile-apps]

Greptile Summary

This PR wires the contract-mode dual-write path into runSeed and migrates 91 seeders to export declareRecords plus the full descriptor fields (schemaVersion, sourceVersion, maxStaleMin). The approach is safe for legacy readers because the envelope write is additive alongside the existing seed-meta:* key.

• P1 — extra-key envelope mislabels empty writes as 'OK': in _seed-utils.mjs line 917, state: ekCount > 0 ? 'OK' : (zeroIsValid ? 'OK_ZERO' : 'OK') has 'OK' where it should be 'RETRY'. Any extra key that returns 0 records (and zeroIsValid is false) will be written to Redis with state: 'OK', causing downstream health readers to treat it as healthy once envelope reads are enabled in PR 2c.
• P2 — 39 of the 91 migrated seeders are missing the isMain guard: they export declareRecords but call runSeed() unconditionally at module level. The conformance test reads files as text today so it doesn't trigger, but any future test that imports one of these modules will fire runSeed() against Redis and kill the test process via process.exit().

Confidence Score: 4/5

Safe to merge for the dual-write path itself; the P1 extra-key state bug should be fixed before envelope reads are enabled in PR 2c.

One P1 bug exists: extra-key envelopes mislabel 0-record writes as state 'OK' instead of 'RETRY'. This is latent today because no reader inspects _seed.state yet (PR 2c), but it will silently corrupt health signals once readers migrate. The P2 isMain gap affects 39 seeders and is a test-safety risk rather than a production correctness issue now.

scripts/_seed-utils.mjs — the ekEnvelope state ternary at line 917 is the only blocking fix needed.

Important Files Changed

Filename	Overview
scripts/_seed-utils.mjs	Core runSeed engine with contract mode dual-write state machine; contains P1 bug where extra-key envelope uses 'OK' instead of 'RETRY' when ekCount=0 and zeroIsValid=false
scripts/_seed-contract.mjs	validateDescriptor and resolveRecordCount implementations look correct; soft-warn behavior during PR 2 is intentional and well-documented
scripts/_seed-envelope-source.mjs	buildEnvelope / unwrapEnvelope / stripSeedEnvelope are clean and match the spec; no issues
scripts/seed-fsi-eu.mjs	Reference implementation of contract migration with proper isMain guard and full descriptor fields; no issues
scripts/seed-aviation.mjs	Exports declareRecords but calls runSeed() unconditionally at module level — missing isMain guard that ~39 other migrated seeders also lack
scripts/seed-jodi-gas.mjs	Uses extraKeys in contract mode — the LNG vulnerability extra key would be subject to the P1 ekEnvelope state bug if it ever returns 0 records
scripts/seed-aaii-sentiment.mjs	Correctly uses isMain guard, exports declareRecords, and carries both legacy recordCount and new declareRecords during dual-write transition period
scripts/seed-climate-news.mjs	Missing isMain guard — calls runSeed() at module level while exporting declareRecords

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[runSeed with declareRecords] --> B{contractMode?}
    B -- No --> C[Legacy: bare data + seed-meta write]
    B -- Yes --> D[resolveRecordCount for canonical key]
    D --> E{count gt 0?}
    E -- Yes --> F[contractState = OK]
    E -- No --> G{zeroIsValid?}
    G -- Yes --> H[contractState = OK-ZERO]
    G -- No --> I[contractState = RETRY - skip publish, extend TTL, exit 0]
    F --> J[atomicPublish with envelope wrapper]
    H --> J
    J --> K[writeFreshnessMetadata - dual-write seed-meta]
    K --> L{extraKeys present?}
    L -- Yes --> M[resolveRecordCount per extra key]
    M --> N{ekCount gt 0?}
    N -- Yes --> O[ekState = OK]
    N -- No --> P{zeroIsValid?}
    P -- Yes --> Q[ekState = OK-ZERO]
    P -- No --> R[ekState = OK BUG should be RETRY]
    O --> S[writeExtraKey with envelope]
    Q --> S
    R --> S
    I --> T[exit 0]

Loading

_{Reviews (1): Last reviewed commit: "feat(seed-contract): PR 2a — runSeed env..." | Re-trigger Greptile}

[greptile-apps]

Extra-key envelope reports 'OK' instead of 'RETRY' on zero-record write

When ekCount === 0 and zeroIsValid is false, the final branch evaluates to 'OK' — the same value used for a healthy write. Any future reader that inspects _seed.state on the extra-key envelope will incorrectly treat the key as healthy, masking the empty-data condition that the RETRY state is meant to signal. The canonical key correctly aborts publishing on RETRY; the extra-key path has no such guard and always writes — and now mislabels that write.

Suggested change

	state: ekCount > 0 ? 'OK' : (zeroIsValid ? 'OK_ZERO' : 'OK'),
	state: ekCount > 0 ? 'OK' : (zeroIsValid ? 'OK_ZERO' : 'RETRY'),

[greptile-apps]

Missing isMain guard on seeders that export declareRecords

seed-aviation.mjs exports declareRecords (for contract conformance) but calls runSeed() unconditionally at the module level. The comment in seed-fsi-eu.mjs explicitly marks this guard as "required for scripts that export AND call runSeed at top level — prevents runSeed() from firing when this module is imported in tests or CI." The conformance test reads files as text so it doesn't trigger this today, but any future test that does import { declareRecords } from './seed-aviation.mjs' will immediately fire runSeed(), attempting a Redis write and calling process.exit(), killing the test runner.

The same pattern appears in ~39 other seeders in this PR (e.g. seed-climate-news.mjs, seed-conflict-intel.mjs, seed-bigmac.mjs …). seed-fsi-eu.mjs is the reference implementation:

const isMain = process.argv[1] && import.meta.url.endsWith(process.argv[1].replace(/^file:\/\//, ''));
if (isMain) {
  runSeed(...).catch(...);
}