Skip to content

Backend Services and Event Processing

Jump to bottom
GiZano edited this page Aug 31, 2026 · 2 revisions

Backend Services & Event Processing

The backend is engineered to handle massive telemetry spikes (firehosing) during seismic swarms by decoupling ingestion from processing via Redis.

API Gateway & Asynchronous Ingestion

  • Rate Limiting: Sliding-window rate limiter in Redis (ZADD/ZREMRANGEBYSCORE, 50 req/s per IP).
  • Queue Offloading: Validated payloads are XADDed to a Redis Stream (readings:stream), returning 202 Accepted in milliseconds (no blocking DB write).

Redis Streams Ingestion Substrate (v1.2.1)

The queue is a Redis Stream (not a List), enabling consumer groups:

  • Horizontal scale: Any number of producers XADD; any number of workers XREADGROUP (docker compose scale worker=N).
  • At-least-once: XAUTOCLAIM reclaims pending entries after crashes.
  • Dead Letter Stream: readings:dlq parks poisoned messages.
  • Backpressure: MAXLEN ~200000, batch 64 with 500 ms block, single db.commit() per batch.
  • Multi-tenancy: All keys (stream, group, DLQ) are env-repointable.

Background Worker & Persistence

A decoupled worker.py loops XREADGROUP + XAUTOCLAIM:

  • Shares an optimized SQLAlchemy pool for PostgreSQL.
  • Stages a whole batch in-memory; on DB error routes the batch to the DLQ.

TimescaleDB Hypertable (v1.2.1)

readings is a TimescaleDB hypertable on recorded_at (PostGIS still enabled via unified postgres-timescale image). timescale.py DDL is idempotent and fails closed on plain PostGIS. Continuous aggregate readings_minute serves dashboard rollups; compression + retention policies.

Geographic Zoning & Zone-Scoped Data (v1.2.1)

  • Zone management: GET /zones, POST /zones/; seed of 8 global macro-regions + Unknown Region.
  • Auto-assignment: resolve_zone first checks Redis fast-path zoneindex:<geohash> (prec 3), falls back to PostGIS ST_Contains ordered by ST_Area ASC. The cache is purely an optimization.
  • Zone detection: GET /zones/locate?latitude=&longitude= for mobile “Detect my zone”.
  • Zone-scoped retrieval: GET /zones/{zone_id}/readings, DELETE /zones/{zone_id}/readings, GET /zones/{zone_id}/alerts.

Magnitude Estimation & Per-Area Deduplication

$$ M_{IoT} = \log_{10}(PGA_{calib}) + b $$

PGA_calib uses K_CALIBRATION=1.6, b=B_OFFSET=3.0 (shared with mobile).

  • Thresholding: Triggers Alert if M ≥ 4.5.
  • Per-area cooldown (v1.2.1): Atomic SET nx=True ex=60 keyed by area — alert_cooldown:<geohash> prec 4 (~39×19 km) when lat/lon present, else zone. Stops overlapping macro-regions from silencing independent quakes. Reading.lat/lon captured at ingestion.
  • Outbox: First worker acquiring the lock persists Alert and publishes quake_alerts to Redis Pub/Sub.
  • AI enqueue: When AI_REPORT_ENABLED=true, confirmed alerts push context to ai_report_queue — see AI Emergency Report Service.

Clone this wiki locally