Skip to content

feat(climate): add climate disasters seed + ListClimateDisasters RPC …#2535

Merged
koala73 merged 1 commit intomainfrom
feat/add-climate-disaster-alerts-seeder
Mar 30, 2026
Merged

feat(climate): add climate disasters seed + ListClimateDisasters RPC …#2535
koala73 merged 1 commit intomainfrom
feat/add-climate-disaster-alerts-seeder

Conversation

@FayezBast
Copy link
Copy Markdown
Collaborator

Summary

Adds climate disaster ingestion and API exposure for the climate domain.

  • New seeder: scripts/seed-climate-disasters.mjs
  • Sources:
    • ReliefWeb disasters (FL, TC, DR, HT, WF; status alert/current/ongoing)
    • Reused/filtered natural:events:v1 (GDACS + NASA FIRMS only; no GDACS re-seeding)
  • Writes Redis key: climate:disasters:v1
  • TTL: 21600 (6h)
  • Output persisted in Redis with required snake_case shape:
    • id, type, name, country, country_code, lat, lng, severity, started_at, status, affected_population, source, source_url
  • New proto + RPC:
    • ClimateDisaster
    • ListClimateDisasters
  • Handler/wiring added in climate service, gateway cache tier, bootstrap/health/seed-health/MCP integrations
  • Added/updated tests for seeder shape + cache-read behavior

Operational note: RELIEFWEB_APPNAME must be set (approved value) on Railway for seed-climate-disasters.
I cannot set Railway secrets from this code PR directly.

Type of change

  • Bug fix
  • New feature
  • New data source / feed
  • Documentation
  • CI / Build / Infrastructure

Affected areas

  • API endpoints (/api/*)
  • Other: proto contracts, seed scripts, Redis cache/bootstrap/health wiring

Checklist

  • No API keys or secrets committed

@mintlify
Copy link
Copy Markdown

mintlify bot commented Mar 30, 2026

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
WorldMonitor 🟢 Ready View Preview Mar 30, 2026, 3:50 AM

@vercel
Copy link
Copy Markdown

vercel bot commented Mar 30, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

1 Skipped Deployment
Project Deployment Actions Updated (UTC)
worldmonitor Ignored Ignored Mar 30, 2026 3:49am

Request Review

@greptile-apps
Copy link
Copy Markdown

greptile-apps bot commented Mar 30, 2026

Greptile Summary

This PR introduces end-to-end climate disaster ingestion: a new Railway seed script (seed-climate-disasters.mjs) pulls from ReliefWeb and filters the existing natural:events:v1 cache (GDACS + NASA FIRMS only), writes the result to climate:disasters:v1 with a 6-hour TTL, and exposes it via a new ListClimateDisasters RPC wired from proto → generated stubs → handler → gateway → bootstrap/health/seed-health/MCP.

  • Seeder logic is solid: two-source Promise.allSettled with graceful partial-failure, deduplication by ID + daily-bucket fingerprint, and a capped output of 300 events.
  • Handler correctly reads from Redis via getCachedJson, normalises both snake_case and camelCase field names, and implements offset-based pagination.
  • Proto and generated stubs follow project conventions; int64_encoding = INT64_ENCODING_NUMBER on started_at is appropriate for JS consumers.
  • Bootstrap, health, and seed-health wiring all use correct TTL multiples and intervals consistent with the 6-hour cron cadence.
  • Tests are comprehensive, covering type-filter shape, appname validation, source filtering, severity mapping, coordinate-based country lookup, partial-source resilience, and Redis output shape.
  • Three P2 findings: (1) MCP get_climate_data staleness threshold (120 min) doesn't cover the disaster feed's 6-hour cadence; (2) affectedPopulation is always 0 for ReliefWeb events because the population fields are not requested from the API; (3) multi-country or uncoded ReliefWeb events receive coordinates (0, 0) (Null Island) when no country code can be resolved.

Confidence Score: 5/5

Safe to merge; all findings are P2 quality/observability improvements that do not block the primary data path.

No P0 or P1 issues found. The three P2 findings are: a monitoring gap in MCP staleness tracking (doesn't break serving), hardcoded zero for ReliefWeb affected population (data enrichment gap, not a correctness failure), and Null Island placement for uncoded multi-country events (edge-case data quality). All core logic — seeder resilience, handler pagination, Redis shape, health wiring, and test coverage — is sound and follows established project patterns.

api/mcp.ts (staleness threshold mismatch), scripts/seed-climate-disasters.mjs (affectedPopulation and Null Island coordinate issues)

Important Files Changed

Filename Overview
scripts/seed-climate-disasters.mjs New 467-line seeder: fetches ReliefWeb + natural-events cache, deduplicates, and writes climate:disasters:v1 to Redis. Three P2 issues: affectedPopulation always 0 for ReliefWeb; multi-country events silently placed at (0,0); filtered-row error message misleading.
api/mcp.ts Adds climate:disasters:v1 to get_climate_data cache keys but retains the anomalies-only _seedMetaKey and 120-min staleness threshold; staleness monitoring gap for disaster feed.
server/worldmonitor/climate/v1/list-climate-disasters.ts New RPC handler: reads climate:disasters:v1 from Redis via getCachedJson, applies offset-based pagination, normalises camelCase/snake_case duality. Clean implementation; cursor/limit clamping correct.
tests/climate-disasters-seed.test.mjs Good test coverage: type filter shape, appname validation, source filtering, severity mapping, coordinate-based country lookup, partial-source failures, and Redis output shape.
api/health.js Adds climateDisasters to BOOTSTRAP_KEYS and SEED_META with correct 720 min maxStaleMin (2x the 6h interval). Consistent with seeder TTL.
server/gateway.ts Adds list-climate-disasters to static cache tier, matching the pattern for list-climate-anomalies. Correct.

Sequence Diagram

sequenceDiagram
    participant Cron as Railway Cron 6h
    participant Seed as seed-climate-disasters.mjs
    participant RW as ReliefWeb API
    participant NatCache as Redis natural:events:v1
    participant Redis as Redis climate:disasters:v1
    participant Handler as ListClimateDisasters Handler
    participant Client as API Client

    Cron->>Seed: trigger
    Seed->>RW: POST /v1/disasters types FL TC DR HT WF
    RW-->>Seed: disaster rows
    Seed->>NatCache: getCachedJson natural:events:v1
    NatCache-->>Seed: events array
    Seed->>Seed: filter GDACS and FIRMS then dedup sort slice 300
    Seed->>Redis: SET climate:disasters:v1 TTL 21600s
    Seed->>Redis: SET seed-meta:climate:disasters

    Client->>Handler: GET /api/climate/v1/list-climate-disasters
    Handler->>Redis: getCachedJson climate:disasters:v1
    Redis-->>Handler: disasters array
    Handler->>Handler: normalise and paginate
    Handler-->>Client: disasters with pagination
Loading

Comments Outside Diff (4)

  1. api/mcp.ts, line 54-55 (link)

    P2 MCP staleness check doesn't cover the disasters feed

    The tool now serves climate:disasters:v1, but _seedMetaKey still points only to seed-meta:climate:anomalies with _maxStaleMin: 120. The disasters seeder runs every 6 hours (360 min interval, 720 min max-stale per api/health.js), so the MCP health check will never reflect disaster-data staleness — the anomalies key can be perfectly fresh while the disasters feed has been stale for hours.

    At minimum, consider bumping _maxStaleMin to 720 to match the health.js entry for this domain so the anomaly-based check doesn't generate false-positive staleness alerts for what is now a combined tool.

  2. scripts/seed-climate-disasters.mjs, line 540-556 (link)

    P2 affectedPopulation is never populated for ReliefWeb events

    The fields.include list in buildReliefWebRequestBodies doesn't request any population data from the ReliefWeb API (e.g. affected, country.affected, or vulnerable), so affectedPopulation is hardcoded to 0 for every ReliefWeb event. The proto comment says "Affected population when available" — but for ReliefWeb it will never be available as currently wired.

    ReliefWeb's v1/v2 disasters endpoint does return affected as a top-level field. If population data is useful, consider adding it to fields.include and mapping it in mapReliefItem.

  3. scripts/seed-climate-disasters.mjs, line 559-603 (link)

    P2 Multi-country / uncoded ReliefWeb events are silently placed at (0, 0)

    When resolveCountryInfo returns an empty countryCode (e.g. for multi-country events like "Floods - Southern Africa" where no single ISO-2 code is resolved), getCountryCenter('') returns { lat: 0, lng: 0 }. That coordinate (Null Island, Gulf of Guinea) is then written to Redis and served to clients.

    These events are not filtered out — they land in the output with a valid country / name but wrong coordinates. Consider either filtering them out or explicitly flagging them with a sentinel so consumers can tell the coordinate is unknown rather than confidently wrong.

  4. scripts/seed-climate-disasters.mjs, line 605-647 (link)

    P2 Misleading error when rows are returned but all filtered by type

    When all ReliefWeb rows are present but every one is rejected by mapReliefItem (e.g. unrecognized disaster type), mapped.length === 0 falls through without a continue. The loop then tries the v2 endpoint with the same body. If that also yields only filtered items, the final error thrown is 'ReliefWeb returned no climate disaster rows' — which doesn't indicate that rows were returned from the API but none matched the type filter.

    Consider logging a warning or adjusting the error message so operators can distinguish "API returned empty" from "API returned rows but all were filtered".

Reviews (1): Last reviewed commit: "feat(climate): add climate disasters see..." | Re-trigger Greptile

@koala73 koala73 merged commit e2dea94 into main Mar 30, 2026
9 checks passed
@koala73 koala73 deleted the feat/add-climate-disaster-alerts-seeder branch March 30, 2026 08:23
@koala73
Copy link
Copy Markdown
Owner

koala73 commented Mar 30, 2026

Merged by mistake ! will revert & review @FayezBast

koala73 added a commit that referenced this pull request Mar 30, 2026
…ers RPC with snake_case Redis payload (#2535)"

This reverts commit e2dea94.
@koala73
Copy link
Copy Markdown
Owner

koala73 commented Mar 30, 2026

Pls resubmit this PR @FayezBast

koala73 added a commit that referenced this pull request Mar 30, 2026
…ers RPC with snake_case Redis payload (#2535)" (#2544)

This reverts commit e2dea94.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants