v2.0.0

Beacon Server 2.0.0

A huge thank you to n30nex for helping get Beacon to this point. Many of the dashboards and features in 2.0.0, from observer monitoring and the analytics endpoints to channel scopes and the MeshMapper work, are there because of his help.

Pair this release with Beacon Web 2.0.0.

Breaking change: 2.0.0 needs a fresh database

There were so many changes to the backend database over this cycle that it made sense to start clean. 2.0.0 starts from a new schema baseline and will not run against a 1.x database. It refuses to start with:

migrations failed error="database schema predates Beacon 2.0.0; 2.0.0 needs a fresh database"

1.x history does not carry over. The database user and database name also change from tower to beacon, and several config settings that used to be tolerated now stop startup. Follow the upgrade guide step by step before you pull the new image.

Built for big instances

A lot of this release is about load. MeshMapper's instance at analyzer.meshmapper.net handles around 400,000 packets every 15 minutes on 2.0.0 and barely sweats. MQTT bursts are buffered into bounded per-observer workers, route reconfirmation runs in batches that no longer stall ingest, retention deletes are batched with tuned autovacuum, and every historical stat now reads from hourly rollups instead of materialized views or live scans.

MeshMapper integration

MeshMapper has expanded its API and Beacon now ingests it. With the meshmapper config block turned on, Beacon can:

• Tag scoped traffic using the transport scope catalogue MeshMapper publishes for each IATA.
• Set up regions automatically from MeshMapper zone groups, so you do not have to hand-write regions:.
• Draw geo borders from MeshMapper zone outlines, used for the map border layer and the foreign repeater flag.
• Import each IATA's public hashtag channels and decrypt them like configured ones.

Everything is off by default, needs no API key, and respects MeshMapper's per-endpoint rate limits. Details are in the configuration guide.

Versioning from here on

Starting with 2.0.0, Beacon Server and Beacon Web share the same major and minor version. Only the patch number moves on its own, so server 2.0.3 runs with web 2.0.1. Keep both on the same 2.X. The latest image tag is now reserved for stable releases from main; pin :2.0 on both images if you want to choose when to move to the next minor.

Everything since v1.6.0

Breaking changes and configuration

• Fresh database required. All 1.x migrations are flattened into a single 2.0.0 baseline, and migrations now run under an advisory lock inside a transaction so a crash mid-migration cannot strand a database.
• The 1.6.0 materialized views are gone. Historical stats read hourly rollup tables instead (see Stats and analytics).
• packets.retention now defaults to 7 days (down from 30) and also covers observations and channel messages. Hourly stats outlive raw packets under analytics.rollup_retention, default 90 days.
• Every manual scopes: entry must set region: to one of your regions: slugs. A missing or unknown region stops startup.
• REST rate limiting is on by default: 300 requests per minute per client IP on /api/v1/*, answered with 429 and Retry-After. Behind a proxy, set server.trusted_proxies (CIDRs only) and have the proxy send X-Real-IP, or all visitors share one budget.
• WebSocket upgrades are limited to websocket.max_connects_per_minute (default 10) per client IP, and only the page's own host may open /ws unless other origins are listed in websocket.allowed_origins.
• Admin endpoints under /api/v1/admin/ need BEACON_API_KEY or auth.api_key. The key must be at least 16 characters with no whitespace.
• Observer telemetry fields airtimeTxPct and airtimeRxPct are renamed to airtimeTxSecs and airtimeRxSecs. The values were always seconds.
• Startup now rejects negative durations, retention under 24h, non-CIDR trusted proxies, invalid log.level or log.format, MeshMapper refresh intervals out of range, and nodes.mark_foreign: true with no border source.
• meshmapper.scopes.sources is removed and ignored with a warning. Scope catalogues are found through MeshMapper's zone list.
• Go 1.26.6.

MeshMapper

• Scope catalogue import (meshmapper.scopes.enabled): imports each IATA's transport scope names so TRANSPORT_FLOOD traffic is tagged without listing scopes by hand. Manual scopes stay authoritative and match first. Catalogues and ETags persist in PostgreSQL, and a failed refresh keeps the known names and retries.
• Zone boundaries (meshmapper.zones.enabled): imports each IATA's outline for GET /iatas/{iata}/border and for the foreign repeater flag, falling back to borderFile when MeshMapper has none. Listed IATAs also pick up a display name and location unless iatas: sets them.
• Zone groups as regions (meshmapper.zones.import_groups): MeshMapper zone groups become regions with the group code as slug and the members' average location as centre. Hand-written regions with the same slug win, and hand-written regions: are now optional.
• Regional channels (meshmapper.channels.enabled): imports each IATA's public hashtag channels. Keys and hashes are checked against the name, history is backfilled when a key first appears, and imported channels list under their IATA when /channels is filtered by region.
• Configured channels can be regional too: channel_keys.hashtags entries accept {name, region} and channel_keys.keys.<hash> accepts region:.
• GET /scopes and GET /stats/scopes filtered by iata, iatas, region or regionId now list the scopes configured or catalogued for that region instead of inferring membership from traffic.
• Every known IATA is looked up, including ones auto-created from traffic. Requests honour MeshMapper's per-endpoint limits, use ETags, and allow MeshMapper's 60 second response time.

Stats and analytics

• Hourly rollups replace materialized views. One advisory-locked analytics_rollup task fills rollup tables for each closed UTC hour. Windows snap to UTC hours, responses report the effective window, and an hour whose raw rows were deleted first is reported as partial. IATA-set tables give exact distinct counts for any IATA set or region, so there is no more cross-IATA overcount.
• New GET /stats/series: hourly network activity with a per-hour status, observation, packet, observer, IATA, scope, path and signal values, and a summary over complete hours.
• New GET /stats/signal: SNR and RSSI histograms, weighted means, sample coverage and hourly trend, capped at 30 days.
• New GET /stats/paths: path header entry counts, 1/2/3-byte hash width distribution, hourly trend and coverage categories. Each hour carries maxEntries, the longest path heard that hour.
• New GET /stats/observer-comparison: distinct flood packets heard by observer A only, B only, and both, over a window.
• New GET /stats/clock-drift: out-of-sync repeaters and room servers, worst first.
• GET /stats/scopes accepts iata, iatas, region and regionId, and each scope carries an hourly array of packets, observers and nodes.
• GET /stats/top-advertisers splits advertCount into floodAdvertCount and directAdvertCount.
• GET /stats/overview spans the 24 newest rolled hours and reports windowHours.
• Heavy stats endpoints return 503 when a query times out instead of hanging.

Admin and operations

• Admin subtree /api/v1/admin/* protected by a bearer key, with 503 when unconfigured and 401 for a missing or wrong header. Public reads and /ws stay open.
• GET /admin/config returns the running CORS settings, whether auth is configured and the broker count, never credentials or keys. PUT /admin/config replaces cors.allowed_origins at runtime without a restart.
• Operator accounts: GET, POST and DELETE under /admin/accounts for name-only records.
• Backup download: GET /admin/backup streams a private .tar.gz with a consistent pg_dump, your saved config.yaml and a checksum manifest. Opt in with backup.enabled: true.
• New beacon-backup CLI exports the same bundle offline, and beacon-backup -verify <archive> checks structure, sizes and SHA-256 without extracting or connecting.
• Structured logging with log.level and log.format (text or json), overridable with LOG_LEVEL and LOG_FORMAT. Every record carries a component, HTTP records carry route, status, duration and request ID, and query strings are never logged.
• ratelimit block: enabled, requests_per_minute and burst.
• server.trusted_proxies: only listed peers may set the client IP via X-Real-IP. Forwarding headers from anyone else are stripped, and a one-time warning fires when an untrusted peer sends them.
• Bounded CPU profiling with BEACON_CPU_PROFILE_DIR and BEACON_CPU_PROFILE_UNTIL: 30 second captures every 30 minutes plus route-maintenance triggers, with file and directory budgets and no HTTP pprof endpoint.
• Debug-level cache diagnostics for hits, misses and Redis bypasses, without keys or values.
• observers.delete_after ages out observers with no retained data (off by default).
• nodes.mark_foreign adds possiblyForeign to repeaters whose position is outside the union of your borders.
• Image tags: latest only from stable releases on main.

Ingest and database

• MQTT burst buffering: packet processing moves out of the paho callback into bounded per-observer-ordered workers, and paho auto-acks deliveries so the broker inflight window no longer caps throughput.
• Random MQTT client IDs per worker, so two Beacon deployments on one broker no longer collide.
• Route reconfirmation runs in batches of 1000 routes with a timeout per batch, skips busy rows, and computes the ambiguous-prefix set once per run, so it no longer blocks ingest.
• Known route retention: routes.retention (14 days), routes.grace (7 days for routes seen fewer than routes.min_observations, default 3). The route ambiguity check now actually matches rows, so expect route counts to settle lower.
• Retention deletes are batched and autovacuum is tuned on high-churn tables.
• Source and destination endpoints are resolved at read time in at most two batched queries, so older packets show current node names.
• Route evidence: GET /routes/{iata}/{pathKey}/observations returns retained reports matching a saved route, with a range up to 720h or since and until, and cursor paging.
• Node directory, clock drift and trace reads do less work per page. Route pages are bounded to the requested IATA.
• Ingest hardening: topic public keys must be 32 bytes (a short key used to panic a worker), lane workers recover from panics, non-finite radio values are rejected, and NUL bytes are stripped from advert names, neighbour scopes and observer status.
• Timestamps accept RFC3339 offsets and fractional precision, space-separated forms and colonless offsets.
• Advert endpoints resolve after the node update so first or renamed adverts broadcast the current name.
• Bucketed observer airtime and receive-error counters are summed from per-row increments, with dips, stale replays and impossible jumps rejected.
• Hearing paths are recorded right after insert, and observer presence is separated from packet arrival in the coalescer.
• /neighbors MQTT messages are parsed for observer and node region_scope.
• IATAs are validated on ingest and in path parameters.

WebSocket

• websocket.allowed_origins accepts exact origins or scheme://*.example.com wildcards. Unlisted origins get 403.
• configure {includeObserverKey: true} adds observation.observerPublicKey to packetObservation events.
• configure {includeRepeats: true} streams later hearings of a packet over a new path as packetObservation with packet.isRepeat: true. Repeats are never stored and are dropped first under load.
• Connect rate limit returns 429 with Retry-After: 60 before upgrade. When max_connections_per_ip (default 5) is full, the socket closes with code 1013 so browsers back off without evicting existing clients.
• A full send buffer now evicts the oldest event and still delivers the newest. Previously both were dropped.
• Subscribe routeTypes and observerIds filters are now applied. They were accepted and ignored before.
• nodeUpdate sends explicit null coordinates when an advert clears a position.

Packets, channels, observers and routes

• Packet summary on lists, backfill and live events: the advert's stored name, or ACK <checksum>, TRACE <tag> and PING <tag>.
• Channel history, catch-up and live channelMessage carry the packet's transport scope and a scopeStatus of matched, unscoped, unknown or unavailable.
• GET /channels?keyKnown=true|false filters channels Beacon can decrypt from hash-only ones. Channel paging uses an opaque pageCursor that preserves timestamp ties.
• New GET /observers/{observerId}/activity: bucketed observation counts, LoRa airtime computed at ingest, SNR and RSSI aggregates and a payload-type breakdown. Intervals of 1h and up read hourly rollups with a raw tail; 5m and 15m read raw rows up to 48h.
• GET /routes takes cursorId and GET /traces takes cursorTag for tie-safe paging. /traces reads a per-tag summary kept at ingest and accepts region filters.
• An explicit 0/0 advert clears a node's stored position instead of storing a literal 0/0.
• Observer statusMetadata is emitted as raw JSON.
• Packet rows carry pathLength and pathBytes on latestObserver.
• List limit is handled the same on every list endpoint: malformed or nonpositive is 400, over 200 is clamped to 200.
• Operator docs moved to beacon-docs.

Fixes

• Trace list pages group whole trace tags, so a tag no longer reappears on a later page with a smaller count.
• GET /observers/{id}/adverts no longer 500s on an advert with no origin key.
• Observer telemetry at 6h and 24h intervals returns a JSON 500 instead of a bare panic when the query fails.
• Lookup endpoints return 404 only for missing rows and 500 for database failures. /regions and /iatas return [] rather than null when empty.
• Non-finite SNR and RSSI samples are kept out of observer JSON.
• Channel messages cascade on packet delete.
• A failed background refresh is reported as failed rather than complete.
• Backups abort on shutdown and stale staging directories are swept at startup. The verifier reports trailing archive data as corruption.
• Profiling captures are offset from task tickers and the recorder tolerates subdirectories.