Skip to content

Reference

Jump to bottom
joergsflow edited this page Jun 11, 2026 · 1 revision

🛰 Sun-Moon Transit Predictor wiki — Home · Setup · Usage · Advanced · Reference · ↩ README

📚 Reference — config, API, limits, trivia

Contents

Configuration

config/observer.json (see observer.example.json)

{
  "name": "Rheine",
  "latitudeDeg": 52.2833,
  "longitudeDeg": 7.4406,
  "elevationM": 50.0,
  "geoidUndulationM": 46.0
}

elevationM is the observer's WGS84 ellipsoidal height (a local MSL value within ~50 m is fine). geoidUndulationM is the EGM2008 N at the observer location — used only when an aircraft reports alt_baro (pressure altitude, ≈MSL); the offset is added so the geometric comparison happens in the right reference frame. Look up your local N at e.g. unavco.org/software/geodetic-utilities. Default 0 is fine if you only see GPS-equipped aircraft (alt_geom).

config/service.json (see service.example.json)

{
  "adsb":     { "url": "http://localhost:8080/data/aircraft.json", "pollIntervalMs": 2000 },
  "tracker":  { "horizonS": 300, "stepS": 0.5, "thresholdDeg": 0.3, "looseThresholdDeg": 2.0, "bodies": ["Sun", "Moon"] },
  "pushover": { "token": "...", "user": "...", "enabled": true },
  "server":   { "port": 8081, "host": "0.0.0.0", "publicUrl": "" },
  "store":    { "path": "./data/history.db" },
  "routes":   { "enabled": true, "ttlMs": 3600000, "negativeTtlMs": 300000 },
  "display":  { "enabled": false, "sourceUrl": "", "quickRefreshS": 2, "longRefreshS": 60 }
}

display configures the optional e-paper panel client — edit it from the web Settings panel rather than by hand; the Python client reads it live from /api/config.

For the complete list of currently-supported fields (sharpcap.targets[], tracker.minAltitudeM, tracker.minBodyElevationDeg, iss.*, predictor.*, lifecycle.*, update.*, driftPersist.*, lifecyclePersist.*, display.*, etc.) refer to config/service.example.json — that file is kept in sync with the installer defaults.

thresholdDeg (default 0.3°) is the maximum line-of-sight separation that triggers a candidate — the Sun's angular radius is ~0.27°, so 0.3° catches near-misses too. stepS (default 0.5 s) is the sample step the tracker walks across the look-ahead horizon; the closest-approach time is then sub-step refined with a parabolic vertex fit, so this only sets the lower bound on detection coverage, not the time precision of the alert.

HTTP API

The service exposes a small JSON API and serves the web UI on the same port (default 8081, bind host 0.0.0.0). Replace <host> below with the Pi's hostname or IP address — for example http://raspberrypi.local:8081/ or http://192.168.1.42:8081/.

Method & path Description
GET / Web UI (live state + history table).
GET /api/state Current observer, Sun/Moon Az/El + observability, aircraft count, lifecycle[] (unified per-(icao, body) tracking list with status enum, M11 — primary feed for the new UI), plus candidates[] (live tracker output, backward compat), expected[] (history-based 24 h watchlist, backward compat) and optics (current FOV setup). Refreshed every poll.
GET /api/history?limit=… Past notifications (radio / candidate / imminent stages) from SQLite, newest first. Default 100, max 500. Each row now also carries outcome (graduated / faded / surprise / null) computed across the episode it belongs to — see Alert learning below.
GET /api/config Sanitised view of the runtime config used by the Settings panel: observer, masked Pushover credentials (incl. the notification url), optics, tracker, AirNav. Pushover token + user key come back as ••••<last4> so the page never echoes the secret in plaintext.
POST /api/config Apply a partial config update ({ observer, pushover, optics, tracker, airnav }). Hot-reloads the running service in place and persists changes back to config/observer.json + config/service.json. Masked secret placeholders (••••…) are ignored so a no-op resave never overwrites the real token.
GET /api/learning?windowDays=… Rolling alert-effectiveness stats over the requested window (default 14 days, capped at 90). Returns aggregates (radioFired, radioGraduated, surprises, hitRatePct, surpriseRatePct, …) plus the last 20 classified episodes.
GET /api/hourstats?sepDeg=…&minElevationDeg=…&windowDays=… "Best hours" — 24-bin hour-of-day histogram of the usable hits (imminent-confirmed, sep < sepDeg default 0.5°, elevation ≥ minElevationDeg default 30°), split per body in observatory-local time (hour of closest_at_ms). Returns perBody.{Sun,Moon}[24], total[24], n and peak.{Sun,Moon,all} ({hour,count}/null). Retrospective; windowDays default 3650, capped at 3650.
GET /api/health Liveness probe — always returns { ok: true, time: <ISO> }.

Responses are Cache-Control: no-store; no authentication, so keep the service on a trusted LAN or front it with a reverse proxy if you need to expose it publicly.

Example calls

# liveness
curl -s http://<host>:8081/api/health
# → {"ok":true,"time":"2026-05-11T12:00:00.000Z"}

# current sky + active candidates
curl -s http://<host>:8081/api/state | jq

# last 20 dispatched notifications
curl -s 'http://<host>:8081/api/history?limit=20' | jq '.events[]'

# open the live UI in a browser
xdg-open http://<host>:8081/        # Linux
open     http://<host>:8081/        # macOS

Sample /api/state response (abbreviated)

{
  "observer":     { "name": "Rheine", "latitudeDeg": 52.2833, "longitudeDeg": 7.4406, "elevationM": 50 },
  "nowMs":        1762870000000,
  "lastUpdateMs": 1762869998000,
  "aircraftCount": 17,
  "bodies": {
    "Sun":  { "azimuthDeg": 178.4, "elevationDeg": 42.1, "rangeM": 1.5e11, "observable": true  },
    "Moon": { "azimuthDeg":  65.2, "elevationDeg": -8.7, "rangeM": 3.8e8,  "observable": false }
  },
  "candidates": [
    {
      "icao":                "3c6589",
      "callsign":            "DLH4PV",
      "body":                "Sun",
      "minSeparationDeg":    0.18,
      "closestApproachAtMs": 1762870042000,
      "transitDurationS":    1.4,
      "altitudeFt":          37000,
      "groundSpeedKt":       454,
      "route":               { "iataFlight": "LH123", "origin": "FRA", "destination": "JFK", "airline": "Lufthansa" }
    }
  ]
}

observable: false on a body means it is below the 20° horizon floor — any aircraft passing in front of it is not reported, by design.

Where files live

Path Purpose Tracked in git?
<repo>/config/observer.json Observer location (lat / lon / elevation, geoid undulation). Personal. no — gitignored
<repo>/config/observer.example.json Schema reference / template for observer.json. yes
<repo>/config/service.json Runtime config (ADS-B URL, intervals, Pushover keys, server, DB, routes). Personal. no — gitignored
<repo>/config/service.example.json Schema reference / template for service.json. yes
<repo>/data/history.db SQLite history of all recorded transit-stage events (created on first run). no — gitignored
<repo>/data/lifecycle.json Tracking-panel snapshot so a restart doesn't empty the list. no — gitignored
<repo>/data/iss.tle ISS two-line elements for the offline SGP4 (written by refresh-tle.js). Feature inactive until present. no — gitignored
<repo>/data/update.request Transient click-to-update trigger; consumed by stp-update.path. no — gitignored
<repo>/web/ Static frontend served at http://<host>:<port>/. yes
<repo>/display/ Optional e-paper panel client (Python) + its README. yes
<repo>/bin/stp.js Service entry point. yes
<repo>/scripts/install-pi5.sh Idempotent Pi installer (interactive or --non-interactive). yes
<repo>/scripts/auto-update.sh Pull + install-deps + restart-on-change. Backs up local config first. yes
<repo>/scripts/refresh-tle.js Opt-in ISS TLE fetcher (Celestrak); run by stp-tle.timer. yes
<repo>/scripts/test-push.js One-shot Pushover sanity check. yes
<repo>/systemd/stp.service Template for the main systemd unit. yes
<repo>/systemd/stp-display.service Template for the optional e-paper display client unit. yes
<repo>/systemd/stp-update.{service,timer,path} Auto-update + click-to-update watcher templates. yes
<repo>/systemd/stp-tle.{service,timer} Daily ISS TLE refresh templates. yes
/etc/systemd/system/stp.service Generated unit (paths and user templated by the installer). n/a (system)
/etc/systemd/system/stp-update.{service,timer,path} Generated auto-update + click-watcher units. n/a (system)
/etc/systemd/system/stp-tle.{service,timer} Generated ISS-TLE refresh unit + timer. n/a (system)
/etc/sudoers.d/stp-update Narrow rule: <user> NOPASSWD: /bin/systemctl restart stp.service. n/a (system)

The main service runs sandboxed: ProtectSystem=strict, ProtectHome=read-only, and the only writable path is <repo>/data/. The SQLite history file therefore must live inside data/ (the default) — pointing store.path outside that directory will fail at write time when running under systemd.

Config preservation contract. observer.json and service.json are gitignored from the first commit that contains this README. Neither git pull nor auto-update.sh will ever touch them. The installer only rewrites them when run with --overwrite. If you ever need to roll back, copy from the matching *.example.json and re-edit.

Project layout

.
├── package.json                  src deps + npm scripts
├── vitest.config.js              test runner config
├── bin/stp.js                    service entry point
├── config/
│   ├── observer.example.json     schema reference (real observer.json is gitignored)
│   └── service.example.json      schema reference (real service.json is gitignored)
├── src/
│   ├── geometry.js               topocentric Az/El + great-circle separation
│   ├── adsb.js                   fetch + normalise dump1090 aircraft.json
│   ├── tracker.js                extrapolation + transit detection (sub-step refined)
│   ├── pushover.js               Pushover REST client
│   ├── notifier.js               3-stage dispatch (radio/candidate/imminent) + dedup
│   ├── adsbdb.js                 callsign → route + hex → airframe, in-memory TTL cache
│   ├── airnav.js                 AirNav On-Demand API v2 client (server-side, cached)
│   ├── sharpcap.js               imminent-transit → SharpCap TCP capture trigger (opt-in)
│   ├── sgp4.js                   dependency-free SGP4 (ISS), TLE parse, TEME→ECEF
│   ├── iss.js                    offline ISS transit + visible-pass prediction
│   ├── store.js                  SQLite history (node:sqlite) + episode stats
│   ├── server.js                 HTTP server (built-in, no framework)
│   ├── service.js                orchestrator (the polling loop)
│   ├── predictor.js              history-based 24 h watchlist (M10)
│   ├── opensky.js                OpenSky Network REST client (M10, opt-in)
│   ├── lifecycle.js              candidate state machine: planned→radio→candidate→imminent→stale (+coasting)
│   ├── config.js                 loadObserver()
│   └── index.js                  public re-exports
├── web/
│   ├── index.html                Sky-now + LIVE-TRACKING-SIGNALS + History + FOV UI
│   ├── app.js                    vanilla-JS poller
│   ├── sketch.js                 FOV transit sketch (SVG, incl. ISS glyph)
│   ├── aircraft-types.js         offline ICAO-type → specs table
│   └── style.css                 dark theme
├── scripts/
│   ├── bootstrap-pi5.sh          bare-image one-liner: apt deps + clone + install-pi5.sh
│   ├── install-pi5.sh            idempotent installer (interactive or --non-interactive)
│   ├── auto-update.sh            git pull → npm install → restart, with config backup
│   ├── refresh-schedule.js       OpenSky daily fetcher (M10, opt-in)
│   ├── refresh-tle.js            ISS TLE fetcher (Celestrak, opt-in / stp-tle.timer)
│   ├── test-push.js              one-shot Pushover sanity check
│   └── sharpcap/                 Windows SharpCap trigger: listener, bootstrap, PS installer, README
├── systemd/
│   ├── stp.service               main service unit template
│   ├── stp-update.service        auto-update oneshot template
│   ├── stp-update.timer          nightly schedule (03:30 ±15 min)
│   ├── stp-update.path           click-to-update trigger watcher
│   ├── stp-tle.service           ISS TLE refresh oneshot template
│   └── stp-tle.timer             daily ISS TLE schedule (05:40 ±20 min)
└── test/                         17 vitest files, ~205 cases

Manual run (development / non-Pi)

Useful for hacking on the code, testing config changes, or running on a non-Pi machine that already has dump1090-fa (or an equivalent feed) reachable on the network.

npm install
cp config/service.example.json config/service.json   # then edit
node --experimental-sqlite bin/stp.js

The process logs the listening URL, the resolved ADS-B URL, and whether Pushover is enabled. Stop it with Ctrl+C — it traps SIGINT / SIGTERM, closes the HTTP server, flushes SQLite, and exits cleanly.

--experimental-sqlite is needed on Node 22; on Node 24+ the flag becomes a no-op since node:sqlite is stable.

Environment variables

Variable Default Purpose
STP_OBSERVER <repo>/config/observer.json Override the observer-config path.
STP_CONFIG <repo>/config/service.json Override the service-config path.

Useful for running multiple observer locations from a single checkout, or for keeping production credentials out of the repo:

STP_OBSERVER=/etc/stp/observer-rheine.json \
STP_CONFIG=/etc/stp/service.prod.json     \
  node --experimental-sqlite bin/stp.js

Tests

npm test

~205 vitest cases across 17 files cover geometry, ADS-B parsing, tracker (including the ADS-B latency back-stamp, sub-step vertex refinement, barometric geoid offset, and the level=candidate/radio split), Pushover client, notifier (3-stage pipeline with minStage filter), route lookup with TTL cache, history store (with stage-rename migration), the HTTP server, the history-based predictor, the OpenSky REST client, the SGP4 propagator (validated against the official Vallado 88888 verification vectors), the SharpCap trigger (dedup / re-arm / busy-vs-network-fail distinction), and the lifecycle state machine (planned / radio / candidate / imminent / stale + coasting + the four stale-reason classifications).

Assumptions and limitations

  • Geometry: 0° = N, 90° = E. WGS84 → ECEF → ENU for aircraft Az/El. Observer ECEF is computed once per tick and reused for every aircraft × body.
  • Reference frame for the comparison: both aircraft and body are compared in geometric (un-refracted) coordinates. /api/state still exposes the refracted body position via the regular bodyAzEl for display. Differential refraction along two near-coincident lines of sight is well below the search noise.
  • Observability: isObservable(azEl, minElevationDeg) returns true only above the threshold (default 20°). Since v0.30.37 the tracker auto-widens this down to the lowest enabled rig's minElevationDeg (hard-floored at 5° for refraction sanity), so a clear-horizon site with a 10°-tolerant main rig will see candidates with the body between 10° and 20° elevation that the old hardcoded floor would have hidden.
  • Aircraft altitude: prefers alt_geom, falls back to alt_baro. alt_geom is GPS height above WGS84 ellipsoid (DO-260) and is fed straight in. alt_baro is pressure altitude (≈MSL on standard atm.) and is converted to HAE by adding observer.geoidUndulationM (default 0; ≈+46 m at Rheine).
  • Extrapolation: linear, locally-flat tangent plane, 300 s horizon (default — clamped to [10, 1800]). Error versus geodesic is well under 1 m at our typical speeds within ~60 s; reasonable through ~5 min in stable cruise. Aircraft are projected from their fix time (receivedAtMs), not from now, so a seen_pos lag of several seconds does not bias the predicted position.
  • Sub-step time precision: after the discrete minimum is located, a parabolic vertex is fitted through the three samples around it. With the default stepS = 0.5 s this gives sub-100-ms closest-approach time.
  • ADS-B liveness: aircraft with seen_pos > 30 s are dropped during parsing — stale fixes are not extrapolated.
  • No camera trigger: explicitly out of scope. We push, you arm the camera.

Status

Production-ready and in daily use; ~78 named milestones (M1 → M78, covering v0.0 → v0.30.39). Detailed history with per-milestone scope lives in MILESTONES.md. For "what changed in release X" see git log --oneline.

Trivia & statistical insights

A mix of physical-geometry math, real prediction statistics from a month-plus of live operation, astronomical curiosities, and the amusing-in-hindsight bugs that shaped the architecture. Numbers are from the running site (Rheine, Germany) unless noted; your data will look broadly similar but specific values drift with traffic patterns and weather.

Physical geometry — how forgiving is an "on-disc" transit, really?

The Sun and Moon disc both span about 0.53° of sky — angular radius ~0.27°. That radius, multiplied by your slant range to the aircraft, is how many METRES of lateral track error pushes the plane out of the disc:

Body elevation Slant range @ 12 km alt Lateral tolerance for disc-edge
90° (overhead) 12.0 km ~57 m (≈ half an A320)
60° elev 13.9 km ~65 m
45° elev 17.0 km ~80 m
30° elev 24.0 km ~113 m
20° elev 35.1 km ~165 m
10° elev 69.1 km ~326 m

Counterintuitive result: at LOWER body elevation the lateral tolerance GROWS (longer slant range = same 0.27° spans more metres). But low- elevation traffic is overwhelmingly descent / approach aircraft that get ATC-vectored constantly, so they drift 500-1000 m in the last minute — which overwhelms the geometric tolerance. Cruise traffic at high elevation has < 50 m of late-stage drift (autopilot + LNAV + GPS is mathematically exact at this scale), so the system records very tight bullseye transits when the prediction holds.

Prediction accuracy — what the postmortem table actually shows

After ~90 finished episodes the prediction-error stats break down roughly like this (the live panel auto-updates, these are typical ranges):

Lead at sample High-elev (≥ 30°, cruise) Low-elev (< 30°, descent / approach)
> 90 s p50 ~0.20° · p95 ~1.5° p50 ~0.30° · p95 ~2.5°
30–60 s p50 ~0.08° · p95 ~1° p50 ~0.10° · p95 ~2°
< 10 s p50 ~0.04° · p95 < 0.3° p50 ~0.13° · p95 ~2°

The clean read: high-elev cruise predictions converge to <0.05° median in the last 10 seconds — well inside the 0.27° disc radius. Low-elev approach traffic plateaus at ~0.13° median (still on-disc on average), but the long-tail (P95 ~2°) reflects ATC-vectoring drift that mathematically cannot be predicted from ADS-B alone. The predictor reports it honestly — it does not hide the long tail.

Field calibration (n=1014 episodes). A larger run confirms the shape and sharpens the takeaways:

  • Lead time barely matters. Aggregate P50 is 0.09–0.18° and P95 ~0.93° across all lead buckets — waiting from 90 s out to 10 s out buys only ~0.09°. There is no reason to hold out for a late lock-in; the prediction does not meaningfully tighten as the ETA approaches.
  • Elevation is the real lever. At ≥ 30° the > 90 s bucket is already 0.09° (n=293); below 30° it sits at 0.21° (n=664) and is what drags the aggregate down. Treat transits ≥ 30° as reliable; keep more margin (or skepticism) below that. This validates the default 30° Pushover gate — it is gating on the genuine accuracy cliff, not an arbitrary line.
  • Aim for the best, not the last. The prediction drifts back out after its tightest point — median 0.34°, P95 0.97° (final − best). The best projected moment is more trustworthy than the final one, which is exactly why the live table and SharpCap arming track bestSepDeg rather than the latest value.

Sun vs Moon — no per-body difference, by design. The error budget is entirely the aircraft trajectory; both body positions are known to arcsecond precision (topocentric, parallax-corrected), so there is no physical reason for prediction accuracy to differ between the two discs. predictionAccuracy() therefore blends both bodies on purpose — any Sun/Moon gap you could split out would be a traffic-mix / time-of-day confound (Moon transits cluster at different hours → different elevations and traffic), not a property of the body. The elevation split above is the cut that actually carries signal.

Wind-drift as a detector

The drift-bias sampler runs every tick over every aircraft above 20° elevation, comparing actual position vs. a constant-velocity extrapolation of the previous fix. The mean residual across many flights = systematic wind / ATC bias for the day. Two observed days worth contrasting:

  • Day 1 (light Ostwind): mean drift 0.9 m/s @ 97° E across n=1407 samples. σ_E = 7.5 m/s, SE = 0.20 m/s → 4.5σ above zero, statistically significant.
  • Day 2 (calm): mean drift 0.1 m/s @ 108° E across n=4000 samples. σ_E = 8.0 m/s, SE = 0.13 m/s → 0.8σ, indistinguishable from no wind.

The system actually detects real-world wind conditions — and is honest when there isn't a meaningful signal. Standard Error (σ/√n), not raw σ, is the correct test for "is the mean different from zero" (see v0.30.39 fix).

Disc-crossing duration — a transit is over before you can react

Angular speed of an airliner across the sky is roughly v / range:

Airliner speed Slant range ω (deg/s) Time to cross 0.53° Sun disc
250 m/s (≈ 900 km/h) 12 km 1.19°/s ~0.45 s
250 m/s 25 km 0.57°/s ~0.93 s
250 m/s 60 km 0.24°/s ~2.2 s

For an overhead cruise jet you have less than half a second of disc contact — explains why SharpCap's own Sequencer is too slow and why this whole project exists.

Sun-disc radius isn't quite constant

The Sun's angular diameter varies between 0.524° (Aphelion ~July 4) and 0.541° (Perihelion ~January 3) — about 3 % swing. The disc radius therefore ranges from 0.262° to 0.270°. The Moon swings wider: 0.490° (Apogee) to 0.564° (Perigee, a "supermoon"). The predictor uses fixed nominal disc sizes in the search threshold; the real variation rounds inside the 0.3° candidate band so it doesn't affect detection.

Hit rate at a mid-European urban site

From a few days' worth of stats at Rheine, NW Germany (well-served by Amsterdam approach traffic + Frankfurt overflights):

  • ~5-15 confirmed-imminent transits per day, mixed Sun + Moon.
  • Of those, ~15-20% are on-disc bullseyes (< disc radius). The rest are near-misses 0.3-2° off-centre.
  • So about 2-3 actual photographable transits per day for the site — multiply by a typical clear-sky fraction in NW Germany and you get the realistic photographic output.

Architectural war stories (one-liners)

A few bugs that shaped the current design. All in MILESTONES.md if you want the full reconstruction.

  • The → arrow that broke a Windows install (v0.30.15): a single U+2192 in a Python comment crashed the SharpCap bootstrap's cache writer because Windows defaults to cp1252 encoding for text-mode file writes. Forced the listener body to pure ASCII forever after.

  • subprocess.run is too new (v0.30.17): some SharpCap builds embed Python 3.4. subprocess.run arrived in 3.5, capture_output=True in 3.7. Had to drop to Popen + communicate to stay portable.

  • The console window that flashed for 1 second (v0.30.18): CREATE_NO_WINDOW flag is Python 3.7+. Older Python silently fell back to 0 and a CMD window briefly popped on every robocopy invocation. Combine with STARTUPINFO + STARTF_USESHOWWINDOW + SW_HIDE (in subprocess since Python 2) to cover all versions.

  • The 120 s outcome wait (v0.30.34): a 20 GB SSD → NAS copy takes many minutes. The predictor's _probempty / _confirmed verdict arrives ~60 s after the lifecycle settles, so the LOCAL rename on SSD has to happen FIRST, then the long upload uses the already- tagged filename. Listener blocks on threading.Event[captureId] for up to 120 s before starting the NAS phase — so the final filename never races the upload.

  • TCP-storm on busy listener (v0.30.3): the per-rig dedup was releasing the slot on EVERY failed send. A "busy" response (listener recording for another ICAO) would clear dedup, next tick fired again, repeat 25 times in 60 s. Fix: distinguish JSON-reply-but- ok:false (= listener said no, keep dedup) from no-reply (= network failed, release dedup).

  • Drift sampler's "low signal" flag was using the wrong test (v0.30.39): comparing the mean magnitude to within-population σ is not a significance test — that ratio never converges to <1 even with millions of samples. The right comparison is Standard Error of the mean (σ / √n), which shrinks with n. With n=1407 samples a 0.9 m/s mean against σ=7.5 is 4.5 standard errors above zero — highly significant, even though the population variance suggests noise.

Headline numbers at a glance

  • ~205 vitest cases across 17 files cover the whole pipeline.
  • 5-15 confirmed-imminent transits per day at a mid-European site.
  • 57-326 m of lateral track-error pushes an aircraft off the disc, depending on body elevation.
  • 0.5 - 2 seconds is a typical disc-crossing time for an airliner.
  • ~120 s is the worst case for the outcome-verdict pipeline to settle on the final filename (lifecycle-stale + outcome-wait).
  • 78 named milestones, M1 → M78, in MILESTONES.md.

🛰 Transit Predictor

Clone this wiki locally