Skip to content

perf(aviation): halve seed cadence to 30min + extend Redis TTLs#3073

Merged
koala73 merged 2 commits into
mainfrom
perf/aviation-seed-30min
Apr 13, 2026
Merged

perf(aviation): halve seed cadence to 30min + extend Redis TTLs#3073
koala73 merged 2 commits into
mainfrom
perf/aviation-seed-30min

Conversation

@koala73
Copy link
Copy Markdown
Owner

@koala73 koala73 commented Apr 13, 2026

Summary

  • Bump seed-aviation Redis TTLs to 2100s (35 min) so keys span the new 30-min cron interval with buffer
  • Update architecture doc to reflect the new 30-min cadence
  • Paired with a Railway dashboard cron change on service a8e49386-64c1-4e1e-9f82-4eb69a55fce3 (15min → 30min), done out of band
  • Motivation: AviationStack upstream call volume has been climbing over the past few days; this cuts the seeder's contribution from ~576/day to ~288/day

Test plan

  • npm run typecheck + npm run typecheck:api
  • npm run lint + npm run lint:md
  • node --test tests/edge-functions.test.mjs
  • After Railway cron is switched, confirm seed-meta:aviation:ops-news updates every 30 min and aviation:ops-summary:v1:... key never TTL-expires between runs

AviationStack upstream call volume has been climbing over the past few
days. Halving the seed-aviation Railway cron from every 15 min to every
30 min cuts this seeder's contribution from ~576/day to ~288/day.

With the longer interval, the Redis TTLs must span the new cron gap with
buffer — otherwise keys expire between runs and the panel goes empty.
Bumping OPS_TTL and NEWS_TTL from 300/900s to 2100s (35 min) gives a
5-minute buffer over the 30-min interval.

The Railway cron schedule itself is changed on the dashboard (service
a8e49386-64c1-4e1e-9f82-4eb69a55fce3), out of band from this PR.
@vercel
Copy link
Copy Markdown

vercel Bot commented Apr 13, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
worldmonitor Ready Ready Preview, Comment Apr 13, 2026 6:13pm

Request Review

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented Apr 13, 2026

Greptile Summary

This PR halves the seed-aviation cron cadence from 15 min to 30 min and raises both Redis TTLs to 2100 s (35 min) so keys survive the longer interval with a 5-minute buffer. The documentation table in architecture.mdx is updated to match.

Confidence Score: 5/5

Safe to merge — changes are confined to TTL constants and a one-line doc update with no logic risk.

Both findings are P2 (style/hardening suggestions). The TTL math is correct (2100 s > 1800 s interval), the documentation is consistent, and the motivation is clearly stated. No correctness or data-integrity issues present.

No files require special attention.

Important Files Changed

Filename Overview
scripts/seed-aviation.mjs OPS_TTL bumped 300 → 2100 s, NEWS_TTL bumped 900 → 2100 s; both now exceed the new 30-min (1800 s) cron interval by a 5-minute buffer, preventing key expiry between runs.
docs/architecture.mdx One-line doc update: seed-aviation cadence column changed from "15 min" to "30 min" — accurate and consistent with the code change.

Sequence Diagram

sequenceDiagram
    participant Railway as Railway Cron
    participant Seed as seed-aviation.mjs
    participant AviStack as AviationStack API
    participant RSS as RSS Feeds (×9)
    participant Redis as Upstash Redis

    Note over Railway,Redis: Every 30 min (1800 s)
    Railway->>Seed: trigger
    par Airport ops (serial)
        loop 6 airports
            Seed->>AviStack: GET /flights?dep_iata=XXX
            AviStack-->>Seed: flight data
            Seed->>Seed: sleep 300ms
        end
    and Aviation news (parallel)
        Seed->>RSS: fetch all feeds
        RSS-->>Seed: XML
    end
    Seed->>Redis: SET aviation:ops-summary:v1:... TTL=2100s
    Seed->>Redis: SET aviation:news::24:v1 TTL=2100s
    Note over Redis: Keys live 35 min (5-min buffer over 30-min interval)
Loading

Reviews (1): Last reviewed commit: "perf(aviation): halve seed cadence to 30..." | Re-trigger Greptile

Comment thread scripts/seed-aviation.mjs Outdated
Greptile review flagged the 5-min buffer on 2100s TTL as tight. Worst
case: serial AviationStack calls for all default airports can take
~63s, and the Railway cron can fire slightly late — a run that starts
29 min after the previous could write a key only seconds before the
old one expires, risking a brief empty-panel window.

Bump OPS_TTL and NEWS_TTL from 2100s to 2400s (40 min) to give a
10-minute buffer over the new 30-min cron interval.
@koala73 koala73 merged commit ced3e70 into main Apr 13, 2026
9 checks passed
@koala73 koala73 deleted the perf/aviation-seed-30min branch April 13, 2026 18:13
@mintlify
Copy link
Copy Markdown

mintlify Bot commented Apr 13, 2026

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
WorldMonitor 🔴 Failed Apr 13, 2026, 6:22 PM

💡 Tip: Enable Workflows to automatically generate PRs for you.

koala73 added a commit that referenced this pull request Apr 23, 2026
…on-quiet-traffic alarm) (#3334)

* fix(aviation): seeder writes delays-bootstrap aggregate (close EMPTY-on-quiet-traffic alarm)

api/health.js BOOTSTRAP_KEYS.flightDelays points at aviation:delays-bootstrap:v1,
but no seeder ever produced it — the key was only written as a 1800s side-effect
inside list-airport-delays.ts. Quiet user-traffic windows >30 min let the
bootstrap expire, tripping EMPTY (CRIT) even with healthy upstream FAA + intl
+ NOTAM seeds. PR #3073 (Apr 13) doubled the cron cadence to 30 min, putting
the bootstrap TTL right at the failure edge.

Make seed-aviation.mjs the canonical writer:
  - New writeDelaysBootstrap() reads FAA + intl + NOTAM from Redis, applies
    the same NOTAM merge + Normal-operations filler the RPC builds, writes
    aviation:delays-bootstrap:v1 with TTL=7200 (~4 missed cron ticks of cushion).
  - Called pre-runSeed (last-good intl, covers intl-fail tick) AND inside
    afterPublishIntl (this-tick intl, happy-path overwrite).
  - Bump RPC's incidental write TTL 1800 → 7200 so a user-triggered RPC
    doesn't shorten the seeder's expiry and re-create the failure mode.

NOTAM merge logic + filler shape are now mirrored in two files (seeder + RPC's
_shared.ts). Both carry comments pointing at the other to surface drift risk.

Verified: typecheck (both tsconfigs) clean; node --test tests/aviation-*.test.mjs
green; full test:data 6590/6590 green.

* fix(aviation): seeder writes restrictedIcaos + bootstrap unwraps intl envelope

PR #3334 review (P1 + P2):

P1 — bootstrap silently dropped NOTAM restrictions
  seedNotamClosures() only tracked NOTAM_CLOSURE_QCODES; the live RPC's
  classifier in server/worldmonitor/aviation/v1/_shared.ts also derives
  restrictions via NOTAM_RESTRICTION_QCODES (RA, RO) + restriction code45s
  + restriction-text regex. Seeded NOTAM payload only had `closedIcaos`,
  so restrictedIcaos was always empty in Redis — both the new bootstrap
  aggregate AND the RPC's seed-read path silently dropped every NOTAM
  restriction. Mirror the full classifier from _shared.ts:438-452;
  side-car write now includes restrictedIcaos and seed-meta count
  reflects closures + restrictions.

P2 — pre-runSeed bootstrap built with no intl alerts on intl-fail tick
  runSeed wraps the canonical INTL_KEY in {_seed, data} when declareRecords
  is enabled. writeDelaysBootstrap()'s upstashGet only JSON.parsed — no
  envelope unwrap — so intlPayload.alerts was undefined on the pre-runSeed
  bootstrap-build path, and an intl-fail tick would publish a bootstrap
  with all intl alerts dropped instead of preserving the last-good
  snapshot. Add upstashGetUnwrapped() (delegates to unwrapEnvelope from
  _seed-envelope-source.mjs); use it for all three reads (FAA/NOTAM bare
  values pass through unchanged via unwrapEnvelope's permissive path).

Verified: typecheck (both tsconfigs) clean; aviation + edge-functions
tests green; full test:data 6590/6590 green.

* fix(aviation): bootstrap iterates union of seeder + RPC airport registries

PR #3334 review (P2 ×2):

P2 — AIRPORTS vs MONITORED_AIRPORTS registry drift
  Today the two diverge by ~45 iata codes (29 RPC-only, 16 seeder-only).
  Pre-fix the bootstrap iterated the seeder's local AIRPORTS list for
  Normal-operations filler and NOTAM airport lookup, so 29 monitored
  airports never appeared in the bootstrap aggregate even though the
  live RPC included them. Fix: parse src/config/airports.ts as text at
  startup (regex over the static const), memoise the parse, build a
  by-iata Map union (seeder wins on conflict for canonical meta), and
  iterate that for both NOTAM lookup and filler. First-run divergence
  summary logged to surface future drift in cron logs without blocking
  writes. Degrades to seeder AIRPORTS only with a warning if parse fails.

P2 — afterPublishIntl receives raw pre-transform data
  runSeed forwards the RAW fetchIntl() result to afterPublish, NOT the
  publishTransform()'d shape. Today publishTransform is a pass-through
  wrapper so data.alerts is correct, but coupling is subtle — added an
  inline CONTRACT comment so a future publishTransform mutation doesn't
  silently drift bootstrap from INTL_KEY.

Verified: typecheck (both tsconfigs) clean; aviation + edge-functions
tests green; full test:data 6590/6590 green; standalone parse harness
recovers all 111 MONITORED_AIRPORTS rows.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant