Skip to content

Operations Runbook

Jump to bottom
Chris Nighswonger edited this page Mar 26, 2026 · 2 revisions

Operations Runbook

This runbook is for day-to-day operation of a deployed Kanfei instance.

Core Health Checks

Use these checks first when diagnosing issues:

  1. App reachable in browser (/)
  2. API health by querying key endpoints:
    • /api/station
    • /api/current
  3. Live stream functioning:
    • WebSocket /ws/live

If /api/station reports disconnected, investigate the hardware connection first (serial device, network reachability, or station power).

Startup Sequence (Manual)

  1. Ensure station connection is available:
    • For serial stations: verify serial device is present
    • For network stations: verify station is powered and reachable on the network
  2. Start logger + web stack (or python station.py run for single-process launch mode used in this repo workflow).
  3. Verify /api/station becomes connected.
  4. Verify dashboard receives live updates.

Common Commands

python station.py status
python station.py run
python station.py dev
python station.py test

Service Management

Linux (systemd)

sudo systemctl status kanfei-logger kanfei-web
sudo systemctl restart kanfei-logger kanfei-web
sudo journalctl -u kanfei-logger -f       # tail logger logs

Windows

python station.py service-status           # quick status check

net stop KanfeiWeb && net stop KanfeiLogger
net start KanfeiLogger && net start KanfeiWeb

sc query KanfeiLogger                      # detailed Windows service state
sc query KanfeiWeb
services.msc                               # GUI

Service logs on Windows are in <repo>/logs/ (auto-rotated, 10 MB x 5 files).

Routine Maintenance

Backups

Enable automatic backups in Settings > Backup or use the CLI:

python station.py backup

The built-in backup system archives the database and custom backgrounds with automatic rotation. Also back up your configuration file (.env or /etc/kanfei/kanfei.conf) separately.

See Backup and Restore Playbook for full details.

Database Housekeeping

Use DB admin APIs (or Settings UI) for long-lived instances:

  • GET /api/db-admin/stats
  • POST /api/db-admin/compact
  • DELETE /api/db-admin/purge/{table}
  • DELETE /api/db-admin/purge-all

Always back up before purge/compact actions.

Upgrade Workflow

  1. Stop services/processes.
  2. Back up DB and config.
  3. Deploy updated code/package.
  4. Start services.
  5. Validate:
    • /api/station
    • /api/current
    • dashboard live updates
    • optional services (nowcast/uploads)

Incident Playbooks

Logger unreachable from web app

Symptoms:

  • degraded responses from station endpoints
  • no live sensor updates

Actions:

  1. Check logger process logs.
  2. Confirm IPC port alignment (default 6514).
  3. For serial stations: confirm no other process owns the serial device.
  4. For network stations: confirm station is reachable on the network.
  5. Restart logger, then web app if needed.

No new sensor rows

Symptoms:

  • /api/current timestamp stale

Actions:

  1. Check station hardware connection (cable/device/power for serial; network/power for WiFi stations).
  2. Confirm configured connection parameters match the station.
  3. Use setup probe/auto-detect to validate hardware path.
  4. Reconnect station from setup API/UI.

External integrations failing (NWS/WU/CWOP/LLM)

Actions:

  1. Verify internet egress and DNS.
  2. Validate feature flags and credentials in config.
  3. Inspect relevant service logs and /api/usage/status for budget gating.

Capacity Notes

  • SQLite is appropriate for single-node self-host use; monitor file growth.
  • Compaction and retention strategy should be part of normal operations for long-running deployments.

Related Pages

Clone this wiki locally