perf(aviation): halve seed cadence to 30min + extend Redis TTLs#3073
Conversation
AviationStack upstream call volume has been climbing over the past few days. Halving the seed-aviation Railway cron from every 15 min to every 30 min cuts this seeder's contribution from ~576/day to ~288/day. With the longer interval, the Redis TTLs must span the new cron gap with buffer — otherwise keys expire between runs and the panel goes empty. Bumping OPS_TTL and NEWS_TTL from 300/900s to 2100s (35 min) gives a 5-minute buffer over the 30-min interval. The Railway cron schedule itself is changed on the dashboard (service a8e49386-64c1-4e1e-9f82-4eb69a55fce3), out of band from this PR.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
Greptile SummaryThis PR halves the Confidence Score: 5/5Safe to merge — changes are confined to TTL constants and a one-line doc update with no logic risk. Both findings are P2 (style/hardening suggestions). The TTL math is correct (2100 s > 1800 s interval), the documentation is consistent, and the motivation is clearly stated. No correctness or data-integrity issues present. No files require special attention. Important Files Changed
Sequence DiagramsequenceDiagram
participant Railway as Railway Cron
participant Seed as seed-aviation.mjs
participant AviStack as AviationStack API
participant RSS as RSS Feeds (×9)
participant Redis as Upstash Redis
Note over Railway,Redis: Every 30 min (1800 s)
Railway->>Seed: trigger
par Airport ops (serial)
loop 6 airports
Seed->>AviStack: GET /flights?dep_iata=XXX
AviStack-->>Seed: flight data
Seed->>Seed: sleep 300ms
end
and Aviation news (parallel)
Seed->>RSS: fetch all feeds
RSS-->>Seed: XML
end
Seed->>Redis: SET aviation:ops-summary:v1:... TTL=2100s
Seed->>Redis: SET aviation:news::24:v1 TTL=2100s
Note over Redis: Keys live 35 min (5-min buffer over 30-min interval)
Reviews (1): Last reviewed commit: "perf(aviation): halve seed cadence to 30..." | Re-trigger Greptile |
Greptile review flagged the 5-min buffer on 2100s TTL as tight. Worst case: serial AviationStack calls for all default airports can take ~63s, and the Railway cron can fire slightly late — a run that starts 29 min after the previous could write a key only seconds before the old one expires, risking a brief empty-panel window. Bump OPS_TTL and NEWS_TTL from 2100s to 2400s (40 min) to give a 10-minute buffer over the new 30-min cron interval.
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Workflows to automatically generate PRs for you. |
…on-quiet-traffic alarm) (#3334) * fix(aviation): seeder writes delays-bootstrap aggregate (close EMPTY-on-quiet-traffic alarm) api/health.js BOOTSTRAP_KEYS.flightDelays points at aviation:delays-bootstrap:v1, but no seeder ever produced it — the key was only written as a 1800s side-effect inside list-airport-delays.ts. Quiet user-traffic windows >30 min let the bootstrap expire, tripping EMPTY (CRIT) even with healthy upstream FAA + intl + NOTAM seeds. PR #3073 (Apr 13) doubled the cron cadence to 30 min, putting the bootstrap TTL right at the failure edge. Make seed-aviation.mjs the canonical writer: - New writeDelaysBootstrap() reads FAA + intl + NOTAM from Redis, applies the same NOTAM merge + Normal-operations filler the RPC builds, writes aviation:delays-bootstrap:v1 with TTL=7200 (~4 missed cron ticks of cushion). - Called pre-runSeed (last-good intl, covers intl-fail tick) AND inside afterPublishIntl (this-tick intl, happy-path overwrite). - Bump RPC's incidental write TTL 1800 → 7200 so a user-triggered RPC doesn't shorten the seeder's expiry and re-create the failure mode. NOTAM merge logic + filler shape are now mirrored in two files (seeder + RPC's _shared.ts). Both carry comments pointing at the other to surface drift risk. Verified: typecheck (both tsconfigs) clean; node --test tests/aviation-*.test.mjs green; full test:data 6590/6590 green. * fix(aviation): seeder writes restrictedIcaos + bootstrap unwraps intl envelope PR #3334 review (P1 + P2): P1 — bootstrap silently dropped NOTAM restrictions seedNotamClosures() only tracked NOTAM_CLOSURE_QCODES; the live RPC's classifier in server/worldmonitor/aviation/v1/_shared.ts also derives restrictions via NOTAM_RESTRICTION_QCODES (RA, RO) + restriction code45s + restriction-text regex. Seeded NOTAM payload only had `closedIcaos`, so restrictedIcaos was always empty in Redis — both the new bootstrap aggregate AND the RPC's seed-read path silently dropped every NOTAM restriction. Mirror the full classifier from _shared.ts:438-452; side-car write now includes restrictedIcaos and seed-meta count reflects closures + restrictions. P2 — pre-runSeed bootstrap built with no intl alerts on intl-fail tick runSeed wraps the canonical INTL_KEY in {_seed, data} when declareRecords is enabled. writeDelaysBootstrap()'s upstashGet only JSON.parsed — no envelope unwrap — so intlPayload.alerts was undefined on the pre-runSeed bootstrap-build path, and an intl-fail tick would publish a bootstrap with all intl alerts dropped instead of preserving the last-good snapshot. Add upstashGetUnwrapped() (delegates to unwrapEnvelope from _seed-envelope-source.mjs); use it for all three reads (FAA/NOTAM bare values pass through unchanged via unwrapEnvelope's permissive path). Verified: typecheck (both tsconfigs) clean; aviation + edge-functions tests green; full test:data 6590/6590 green. * fix(aviation): bootstrap iterates union of seeder + RPC airport registries PR #3334 review (P2 ×2): P2 — AIRPORTS vs MONITORED_AIRPORTS registry drift Today the two diverge by ~45 iata codes (29 RPC-only, 16 seeder-only). Pre-fix the bootstrap iterated the seeder's local AIRPORTS list for Normal-operations filler and NOTAM airport lookup, so 29 monitored airports never appeared in the bootstrap aggregate even though the live RPC included them. Fix: parse src/config/airports.ts as text at startup (regex over the static const), memoise the parse, build a by-iata Map union (seeder wins on conflict for canonical meta), and iterate that for both NOTAM lookup and filler. First-run divergence summary logged to surface future drift in cron logs without blocking writes. Degrades to seeder AIRPORTS only with a warning if parse fails. P2 — afterPublishIntl receives raw pre-transform data runSeed forwards the RAW fetchIntl() result to afterPublish, NOT the publishTransform()'d shape. Today publishTransform is a pass-through wrapper so data.alerts is correct, but coupling is subtle — added an inline CONTRACT comment so a future publishTransform mutation doesn't silently drift bootstrap from INTL_KEY. Verified: typecheck (both tsconfigs) clean; aviation + edge-functions tests green; full test:data 6590/6590 green; standalone parse harness recovers all 111 MONITORED_AIRPORTS rows.
Summary
seed-aviationRedis TTLs to 2100s (35 min) so keys span the new 30-min cron interval with buffera8e49386-64c1-4e1e-9f82-4eb69a55fce3(15min → 30min), done out of bandTest plan
npm run typecheck+npm run typecheck:apinpm run lint+npm run lint:mdnode --test tests/edge-functions.test.mjsseed-meta:aviation:ops-newsupdates every 30 min andaviation:ops-summary:v1:...key never TTL-expires between runs