fix(seeds): migrate IMF seeders to SDMX 3.0 API

[koala73]

Summary

• IMF DataMapper API (www.imf.org/external/datamapper/api/v1) is now blocked by Akamai WAF via JA3 TLS fingerprinting (HTTP 403 for all non-browser clients, including residential proxies)
• The old SDMX endpoint (dataservices.imf.org) was decommissioned by IMF in 2025
• Migrates all 3 IMF-consuming seeders to the new SDMX 3.0 API at api.imf.org/external/sdmx/3.0/ which works without proxy
• Adds shared imfSdmxFetchIndicator() to _seed-utils.mjs that returns data in the same ISO3-keyed shape

Indicator mapping (DataMapper to WEO SDMX)

Old (DataMapper/FM)	New (WEO SDMX)	Coverage
GGR_G01_GDP_PT	GGR_NGDP	207 countries
GGXCNL_G01_GDP_PT	GGXCNL_NGDP	207 countries
GGXWDG_NGDP_PT	GGXWDG_NGDP	204 countries
PCPIPCH	PCPIPCH	210 countries
BCA_NGDPD	BCA_NGDPD	208 countries
NGDPD	NGDPD	verified

Test plan

• imfSdmxFetchIndicator smoke test: 208 countries, correct values for USA/GBR/DEU
• All fiscal-space indicators verified: revenue, balance, debt
• node --test tests/seed-bundle-resilience-recovery.test.mjs passes (13/13)
• node --test tests/national-debt-seed.test.mjs passes (10/10)
• All 3 scripts import cleanly
• Pre-push hooks pass (typecheck, boundaries, seed tests)
• Deploy to Railway and verify seed runs complete successfully

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
worldmonitor	Ready	Preview, Comment	Apr 12, 2026 5:29pm

[greptile-apps]

Greptile Summary

This PR migrates all three IMF-consuming seeders from the blocked DataMapper API to the new SDMX 3.0 endpoint (api.imf.org), adding a shared imfSdmxFetchIndicator() helper to _seed-utils.mjs. The new helper works without a proxy and returns ISO3-keyed data in the same shape the seeders already expect.

• COUNTRY dimension position is hardcoded to split(':')[0] in imfSdmxFetchIndicator, but the dimension is found by ID rather than position. For the WEO database this happens to be correct, but the exported helper supports an FM database option where the dimension order differs, silently producing wrong country-to-value mappings for any future caller.

Confidence Score: 4/5

Safe to merge for WEO usage, but the COUNTRY dimension position bug in imfSdmxFetchIndicator should be fixed before further database options are used.

One P1 finding: the exported helper hardcodes COUNTRY at series-key position 0 despite resolving it by ID, which would silently produce wrong country mappings for any non-WEO database. The fix is a one-liner (use findIndex instead of hard-coding 0). All other findings are P2. The migration itself is correct and well-tested for the three affected seeders.

scripts/_seed-utils.mjs — specifically the imfSdmxFetchIndicator dimension-position parsing (lines 423–435).

Important Files Changed

Filename	Overview
scripts/_seed-utils.mjs	Adds imfSdmxFetchIndicator(): COUNTRY dimension position hardcoded to index 0, but lookup is by ID — fragile assumption that silently breaks for non-WEO databases; no retry logic unlike peer helpers.
scripts/seed-imf-macro.mjs	Migrated to SDMX 3.0 via imfSdmxFetchIndicator; uses Promise.all for parallel fetches; dynamic sourceVersion; no issues found.
scripts/seed-national-debt.mjs	Migrated correctly; hardcodes '2024' as year key in both fetch options and computeEntries, and sourceVersion string — will silently produce empty entries if IMF stops publishing 2024 vintages.
scripts/seed-recovery-fiscal-space.mjs	Migrated correctly; uses sequential await sleep(1500) between indicator fetches while the equivalent macro seeder uses Promise.all — adds 3 s of unnecessary latency with no documented rate-limit rationale.

Sequence Diagram

sequenceDiagram
    participant S as Seed Script
    participant U as _seed-utils.mjs
    participant IMF as api.imf.org (SDMX 3.0)
    participant R as Upstash Redis

    S->>U: imfSdmxFetchIndicator(indicator, { years })
    U->>IMF: GET /external/sdmx/3.0/data/dataflow/IMF.RES/WEO/+/*.{indicator}.A
    IMF-->>U: SDMX JSON (structures + dataSets)
    U->>U: parse dimensions.series[0] → countryValues
    U->>U: parse dimensions.observation → timeValues
    U->>U: build { iso3 → { year → value } }
    U-->>S: Record<ISO3, Record<year, number>>

    S->>S: computeEntries / build countries map
    S->>U: atomicPublish(canonicalKey, data)
    U->>R: SET staging key (5 min TTL)
    U->>R: SET canonical key (35 day TTL)
    U->>R: DEL staging key
    S->>U: writeFreshnessMetadata(domain, resource, ...)
    U->>R: SET seed-meta:domain:resource

Loading

_{Reviews (1): Last reviewed commit: "fix(seeds): migrate IMF seeders from blo..." | Re-trigger Greptile}