Skip to content

openfantasymap/geomqtt

Repository files navigation

geomqtt

A Redis-compatible proxy + embedded MQTT broker that turns Redis GEO sets into a live, tile-keyed topic tree β€” so a web or game client can follow a moving viewport by subscribing to the tiles it can see.

ci tests release npm pages ghcr demo license

Quick start Β· Architecture Β· Clients Β· Observability Β· Protocol Β· Roadmap


What it does

  • Proxies Redis (RESP). Every standard command forwards to an upstream Redis; GEOADD / ZREM / HSET / HDEL / DEL on obj:* keys are intercepted to trigger MQTT fanout.
  • Embeds an MQTT broker. QoS 0, clean session, over both raw TCP (1883) and WebSocket (8083). Browser clients just connect over WS β€” no extra bridge.
  • Projects GEO sets onto slippy-map tiles. Topic tree is geo/<set>/<z>/<x>/<y>; a map viewport is literally a set of tile subscriptions.
  • Serves snapshots on subscribe. Each new subscriber gets the current tile contents as a per-session burst (GEOSEARCH) followed by the live stream.
  • Exposes GeoJSON over HTTP. /tiles/<set>/<z>/<x>/<y>, /viewport/<set>?bbox=…, /objects/<obid> for non-live callers.
  • Reports cheap Prometheus metrics. /status emits in-process counters (sessions, tile fanouts, RESP commands) plus process_resident_memory_bytes. Every increment is a single atomic; rendering walks atomics + one /proc/self/status read.
  • Scales horizontally. Cross-node fanout rides on Redis pub/sub with a node-id envelope so nodes don't echo their own publishes.
  • Ships five clients. TypeScript (Leaflet, MapLibre, core), Unity UPM, Unreal plugin.
  • Has a live demo. openfantasymap.github.io/geomqtt shows a public geomqtt instance tracking the ISS, with the active MQTT subscription set rolling in the corner.

πŸ“ Architecture

                β”Œβ”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”
 writers ──RESP──                                 β”œβ”€β”€β–Ά Upstream Redis
                β”‚     geomqtt-server (Rust)       β”‚    (GEO sets, obj:* hashes)
 browsers ──WS───  β”Œβ”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”¬β”€β”€β”€β”€β”€β”€β”€β”  β”‚
 native  ──TCP───  β”‚RESP β”‚ MQTT β”‚ HTTP β”‚ Redis │◀─┼──▢ Cross-node pub/sub
 scrapers──HTTP──  β”‚proxyβ”‚brokerβ”‚GeoJSNβ”‚coord. β”‚  β”‚    (node-id envelope)
                β”‚  β””β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”΄β”€β”€β”€β”€β”€β”€β”€β”˜  β”‚
                β””β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”€β”˜

A single Rust binary hosts four listeners (RESP, MQTT/TCP, MQTT/WS, HTTP) and a background bridge that relays cross-node publishes.

πŸš€ Quick start

docker compose up --build

Then, in another shell:

# write a point through the proxy
redis-cli -p 6380 GEOADD vehicles 11.34 44.49 veh-42
redis-cli -p 6380 HSET  obj:veh-42 icon truck color red

# read that tile as GeoJSON
curl http://localhost:8080/tiles/vehicles/10/544/370

# discover server tile-size + effective zooms
curl http://localhost:8080/config

# Prometheus-format counters (uptime, sessions, fanout volumes, …)
curl http://localhost:8080/status

# subscribe to live updates (raw TCP)
mosquitto_sub -h localhost -p 1883 -t 'geo/vehicles/10/544/370'

The compose file also runs examples/iss-demo, a tiny sidecar that polls the open-notify ISS position endpoint every 5 s and feeds it into geomqtt β€” handy for watching live tile traffic without any local data:

mosquitto_sub -h localhost -p 1883 -t 'geo/iss/#' -v
curl http://localhost:8080/objects/iss

A matching static web UI lives in examples/web-iss β€” a MapLibre page that subscribes to the iss set and is published to GitHub Pages by .github/workflows/pages.yml. The bottom-left panel shows the active MQTT subscription set (and a rolling sub/unsub log) so you can watch the tile-keyed protocol in action as you pan the map.

β–Ά Open the live demo β€” points at a public deployment at wss://geomqtt.fantasymaps.org/mqtt. Override ?url=wss://your-host:8083 to point it at your own.

πŸ“¦ Install

Pick the channel that matches how you're going to run or talk to geomqtt:

Channel Address
Docker (multi-arch) docker pull ghcr.io/openfantasymap/geomqtt:latest
Docker β€” ISS demo docker pull ghcr.io/openfantasymap/geomqtt-iss-demo:latest
Binaries GitHub Releases β€” Linux / macOS / Windows, x86_64 + aarch64
npm β€” core library npm install @openfantasymap/geomqtt-core (published to GitHub Packages by npm.yml on tag push or manual dispatch β€” see install note below)
npm β€” Leaflet adapter npm install @openfantasymap/geomqtt-leaflet
npm β€” MapLibre adapter npm install @openfantasymap/geomqtt-maplibre
Unity (UPM) Add https://github.com/openfantasymap/geomqtt.git#upm/v0.1.0 to manifest.json
Unreal (UE 5.3+) Drop clients/geomqtt-unreal/ into your project's Plugins/Geomqtt/ and rebuild

GitHub Packages install note. The npm packages live on GitHub Packages, which requires authentication even for public packages. Add a .npmrc at your project root (or ~/.npmrc):

@openfantasymap:registry=https://npm.pkg.github.com
//npm.pkg.github.com/:_authToken=YOUR_GITHUB_PAT

Any GitHub personal access token with the read:packages scope works.

βš™οΈ Configuration

All config is via environment variables:

Variable Default Purpose
GEOMQTT_REDIS_URL redis://127.0.0.1:6379 Upstream Redis connection string
GEOMQTT_RESP_ADDR 0.0.0.0:6380 RESP (Redis-compatible) listener
GEOMQTT_MQTT_ADDR 0.0.0.0:1883 MQTT TCP listener
GEOMQTT_MQTT_WS_ADDR 0.0.0.0:8083 MQTT WebSocket listener (browser clients)
GEOMQTT_HTTP_ADDR 0.0.0.0:8080 HTTP listener (GeoJSON + /healthz + /config)
GEOMQTT_ENRICH_ATTRS (empty) CSV of attribute keys embedded in tile payloads
GEOMQTT_ENRICH_ZOOMS 6-12 Zoom levels that receive tile-topic publishes. Accepts ranges (6-12) and mixed lists (4,6-10,14)
GEOMQTT_TILE_SIZE 256 Tile edge in pixels, power of 2 in 1..=256. Smaller = finer granularity (128 doubles tile count per zoom)
GEOMQTT_OBJECT_KEY_PREFIX obj: Prefix for the per-object attribute hash
RUST_LOG info tracing-subscriber filter

GEOMQTT_TILE_SIZE shifts every configured zoom upward by log2(256 / tile_size). For example, GEOMQTT_ENRICH_ZOOMS=6-12, GEOMQTT_TILE_SIZE=128 publishes on effective zooms 7-13. The effective list is returned by GET /config so clients can mirror it.

πŸ“Š Observability

GET /status returns Prometheus-format text. Counters increment at hot paths via AtomicU64::fetch_add(Relaxed); gauges resolve from the broker session table or one /proc/self/status read. No background ticker, no allocator hooks, no Redis round-trips on a scrape.

geomqtt_build_info{version="0.1.2",node_id="…"} 1
geomqtt_uptime_seconds 1234.5
geomqtt_mqtt_sessions 4
geomqtt_mqtt_subscriptions 71
geomqtt_mqtt_connections_total{transport="ws"} 42
geomqtt_mqtt_packets_received_total 318
geomqtt_resp_commands_total 1024
geomqtt_resp_geo_writes_total 220
geomqtt_tile_fanouts_total 1540
geomqtt_object_fanouts_total 12
geomqtt_redis_bridge_messages_total 87
geomqtt_http_requests_total 60
process_resident_memory_bytes 12345678
process_virtual_memory_bytes  2861531136
process_resident_memory_max_bytes 14000000

Point a Prometheus scrape config at http://<host>:8080/status (use the canonical metrics path of your scraper β€” geomqtt does not enforce the /metrics convention so it can keep /healthz / /config / /tiles / /viewport / /objects cohesive on :8080).

🧭 Clients

Four client packages under clients/. Same protocol, different runtimes:

@openfantasymap/geomqtt-core TypeScript. MQTT transport, tile math, viewport-diff subscribe loop, state map. Runs in Node and the browser.
@openfantasymap/geomqtt-leaflet L.LayerGroup adapter. Wires moveend/zoomend to the core client; default L.circleMarker rendering with a markerFor hook.
@openfantasymap/geomqtt-maplibre MapLibre / Mapbox GL adapter. Keeps a GeoJSON source fed from the current state; debounced via updateThrottleMs.
com.geomqtt.unity Unity UPM package. Pure-C# GeomqttClient plus a GeomqttWorld3D MonoBehaviour that projects lat/lng β†’ local ENU meters for 3D world-anchored scenes. 2D overlay also supported.
geomqtt (UE 5.3+) Unreal Engine plugin. Built-in MQTT v3.1.1 codec over UE's WebSockets module β€” no third-party MQTT plugin required. UGeomqttClient for protocol logic, AGeomqttMarkerSpawner for drag-and-drop 3D world-anchored scenes, UGeomqttSubsystem for Blueprint access.

Build the TypeScript workspace:

cd clients
npm install
npm run build

πŸ—‚ Repository layout

.
β”œβ”€β”€ Cargo.toml                  # workspace root
β”œβ”€β”€ crates/
β”‚   └── geomqtt-server/         # the single Rust binary
β”‚       └── src/
β”‚           β”œβ”€β”€ main.rs         # orchestration, signal handling
β”‚           β”œβ”€β”€ config.rs       # GEOMQTT_* env var parsing + zoom-range syntax
β”‚           β”œβ”€β”€ coord.rs        # slippy-map XYZ tile math
β”‚           β”œβ”€β”€ resp.rs         # RESP proxy with command interception
β”‚           β”œβ”€β”€ mqtt.rs         # embedded MQTT broker (TCP + WS)
β”‚           β”œβ”€β”€ broker.rs       # in-memory session registry + fanout
β”‚           β”œβ”€β”€ fanout.rs       # point β†’ tile events (add/move/remove)
β”‚           β”œβ”€β”€ http.rs         # axum router: GeoJSON + /config + /healthz + /status
β”‚           β”œβ”€β”€ metrics.rs      # AtomicU64 counters + Prometheus text rendering
β”‚           β”œβ”€β”€ payload.rs      # JSON payloads (mirrored in client packages)
β”‚           └── redis.rs        # fred client + cross-node pub/sub bridge
β”œβ”€β”€ clients/
β”‚   β”œβ”€β”€ geomqtt-core/           # @openfantasymap/geomqtt-core β€” TypeScript
β”‚   β”œβ”€β”€ geomqtt-leaflet/        # @openfantasymap/geomqtt-leaflet
β”‚   β”œβ”€β”€ geomqtt-maplibre/       # @openfantasymap/geomqtt-maplibre
β”‚   β”œβ”€β”€ geomqtt-unity/          # com.geomqtt.unity (UPM)
β”‚   └── geomqtt-unreal/         # Unreal Engine plugin (UE 5.3+, built-in MQTT codec over WS)
β”œβ”€β”€ examples/
β”‚   β”œβ”€β”€ iss-demo/               # alpine sidecar polling api.open-notify.org β†’ RESP
β”‚   └── web-iss/                # static MapLibre demo (esbuild β†’ GitHub Pages)
β”œβ”€β”€ .github/workflows/
β”‚   β”œβ”€β”€ ci.yml                  # Rust fmt + clippy, TS build + typecheck
β”‚   β”œβ”€β”€ tests.yml               # Rust unit + integration (Redis service), TS vitest
β”‚   β”œβ”€β”€ npm.yml                 # Publishes @openfantasymap/geomqtt-* to GH Packages
β”‚   β”œβ”€β”€ release.yml             # Binaries + Docker (geomqtt + geomqtt-iss-demo) + UPM split
β”‚   └── pages.yml               # Builds + deploys the web demo to GitHub Pages
β”œβ”€β”€ Dockerfile
β”œβ”€β”€ docker-compose.yml
β”œβ”€β”€ PROTOCOL.md                 # wire contract
└── CLAUDE.md                   # developer context

πŸ›£ Roadmap

  • Workspace scaffold, four listeners binding their ports
  • Env-driven configuration (with range + tile-size syntax)
  • Protocol spec (PROTOCOL.md)
  • RESP parsing + upstream forwarding via fred::custom_raw
  • GEOADD / ZREM / HSET / HDEL / DEL interception and MQTT fanout
  • Embedded MQTT broker (hand-rolled on mqttbytes β€” QoS 0, clean session)
  • Per-subscriber snapshot burst on SUBSCRIBE
  • Cross-node Redis pub/sub with node-id envelope
  • HTTP GeoJSON endpoints + /config
  • @openfantasymap/geomqtt-{core,leaflet,maplibre} TS packages on GH Packages
  • Unity UPM package with GeomqttClient + GeomqttWorld3D
  • Unreal Engine plugin (UGeomqttClient + AGeomqttMarkerSpawner, built-in MQTT codec)
  • CI + release automation (binaries, Docker, npm, UPM, GitHub Pages)
  • /status Prometheus endpoint + process memory metrics
  • ISS demo (examples/iss-demo) and static MapLibre web demo (examples/web-iss) with live subscription panel
  • CORS on the HTTP API β€” fetchServerConfig() works cross-origin
  • Tile-side attr fanout (attribute-only updates also reach tile topics)
  • Lua-scripted atomic GEOADD + old-pos capture
  • SPUBLISH / SSUBSCRIBE for Redis Cluster sharded pub/sub

πŸ“„ License

Dual-licensed under either:

at your option.

About

a REDIS wrapper with MQTT output for real time geospatial information management

Topics

Resources

License

Apache-2.0, MIT licenses found

Licenses found

Apache-2.0
LICENSE-APACHE
MIT
LICENSE-MIT

Stars

Watchers

Forks

Packages

 
 
 

Contributors