Changelog

Generated from CHANGELOG.md in the repository. Edit it there; this wiki is rebuilt on every change.

Changelog

All notable changes are documented here. Format loosely follows Keep a Changelog.

[1.17.0] - 2026-10-19

Changed

• The heartbeat agent is an app (ADR 0018): on nodes with systemd it is installed by esapp like any app, in /opt/edgeshell/apps/heartbeat with its own Python venv (no more apt packages installed system-wide), the service and user es-heartbeat, the node's keys and login in its private data/ folder. Updates switch atomically and go back to the previous version by itself when the new one does not stay up; the newest 3 stable versions stay; the Apps page shows its versions and can roll it back. Its package is built from the agent in the image and kept in the Apps bucket, so every agent sent to a node is in the audit.
• Nodes move over by themselves: the routine sees agent 1.2 as older and updates it. The classic agent stops only for the switch and is removed (folder, service, user hbagent) once the new one runs; if the new one does not run, the classic one starts again. Nodes without systemd keep the classic install. The apt packages the classic install used stay (other software may use them).
• Agent 1.3.0 bundled: setup.py --files-only writes only the node's settings; its folder, group and service come from HB_HOME (or APP_DATA), HB_GROUP, HB_UNIT.

Fixed

• esapp: reinstalling the version that already runs (a repair) left the app's folder root-only, so its service could not start (found by the new end-to-end test on a real node). Folders are locked only when esapp creates them, and their modes are repaired on every install.
• esapp packages are reproducible: the same files always give the same bytes.

Tests

• CI runs the heartbeat agent as an app on a real Debian 12 systemd node with a real broker (18 checks): moved from the classic install in about 31 s, a broken first version leaves the classic agent running, the agent's own venv, private settings, the agent's own test, a settings push hot reloaded, a bad update rolled back in about 38 s with heartbeats continuing, the failed version refused next time, a repair.

[1.16.0] - 2026-10-19

Added

• Apps on the edge (ADR 0017, docs): one way to ship any service or app to the fleet. A folder with a manifest.json, esapp.py pack, upload on the new Apps page, roll out. On each node esapp (/opt/edgeshell/bin/esapp) installs it into /opt/edgeshell/apps/NAME only: its own systemd service es-NAME (starts at boot, restarts on failure, logs in the journal, hardened), its own user, its own Python venv (nothing installed system-wide), code read-only, data in APP_DATA kept across versions.
• Versions side by side, an atomic switch, a way back by itself: a new version is built next to the running one and switched in with one rename; if it does not stay up for its health window (or fails its health check) the previous version comes back by itself, and a version that failed on a node is never installed there again. After a while without a restart a version is stable.
• Rollouts with a canary and an automatic stop: canary nodes first, one at a time (any failure leaves the rest of the fleet untouched), then the rest 4 at a time, stopping when more than a set share failed. The package's checksum is checked before anything is sent; offline nodes are skipped.
• On the nodes: every node's apps, the running version, the service and every version with its state. Roll back (one click: the newest stable version before, the failed one removed), Switch to any version on the node, Clean up (pick versions; the running one and the newest 3 stable versions are always kept), Remove (service, versions, OTA copies, user and data, or keep the data). Every version also carries its own install.sh, rollback.sh and uninstall.sh on the node.
• Bucket and audit: every package uploaded stays on the server with its checksum until an administrator removes it after the audit; every upload, rollout and node action is listed with the result per node.
• CI: a new required job runs esapp on a real Debian 12 systemd node: install, a crashing version that rolls itself back (about 22 s), one-click rollback (about 12 s), cleanup rules, full removal that leaves the system exactly as before. An example app in examples/apps/hello-sensor.

Changed

• Deploy Files writes only to ~/Downloads/edgeshell on each node (a folder inside it may be given), as the SSH user, under a temporary name then renamed. Paths elsewhere (/etc, /opt, /usr/local/bin...) are refused: anything that must run or live elsewhere becomes an app.

[1.15.2] - 2026-10-19

Fixed

• Upgrading an existing database to 1.15.x. The three heartbeat routine columns (hb_way, hb_routine_at, hb_routine_note) were not added to databases made by an earlier version, so every page and job that reads nodes failed after an upgrade. They are added at start now, and a safety net adds any other nullable model column an older database lacks, so a forgotten column can never break an upgrade again (tests drop more than 50 columns across every table and check they come back).

[1.15.1] - 2026-10-19

Changed

• The heartbeat routine tries once, then pauses. A node the routine worked on (install, update, repair or check, whatever the result) is left alone for 24 hours; before, only a failure held a node. Offline and login-paused nodes stay skipped, and a node whose heartbeat arrives is never touched.
• No pass on stale data. When SenseHive is not reachable (or not set up) a routine pass changes nothing, because every node would look silent; the pass summary in Fleet Setup says why.

[1.15.0] - 2026-10-18

Added

• The heartbeat agent on every node, kept there by a routine (ADR 0016). About 10 minutes after a start and then every 6 hours, for each node SSH reaches: install the agent where it is missing, update it where it is older than this EdgeShell's, and repair it where SSH works but no heartbeat arrived for 15 minutes (the agent's own test first, a reinstall only if it fails). At most 25 nodes per pass (HEARTBEAT_ROUTINE_MAX), a node that failed is left alone for a day, every action is in the activity log, the pass summary in Fleet Setup (Keep every node covered: on or off, every 1 to 168 hours, Run now). The heartbeat is what lets the SLA tell a down node from a VPN or SSH glitch, so it is no longer optional; OpenRemoteGUI, Docker and Node-RED stay optional.
• Four ways to the broker (agent 1.2.0): MQTT over TLS, WebSocket over TLS, plain MQTT, plain WebSocket. The install tries each with a real login (the job log shows the table) and keeps the first that works; the node card's heartbeat badge shows it. An agent whose way in stops working finds another by itself, and tries the preferred way again every 10 minutes. This fixes agents stuck retrying a blocked TLS port.
• Broker login for every agent: HEARTBEAT_BROKER_USERNAME / HEARTBEAT_BROKER_PASSWORD (default admin / admin123; change them on your broker and here) and Fleet Setup, Push to every node. Each node checks a new login against the broker before it saves it.

Changed

• Installs go through the agent's setup.py with a join token (the node's keys and the broker, never the master key), the same way SenseHive's Add node works.
• Save and push to fleet merges the new broker, port, TLS and interval into each node's settings and keeps the way in the node found, unless the broker itself changed.
• Agent 1.2.0 bundled (from 1.1.0); SenseHive 1.8.0 by default (SENSEHIVE_TAG): Add node with join tokens, every SenseHive its own fleet, bulk tokens, broker login.

[1.14.2] - 2026-10-18

Changed

• SenseHive 1.7.0 by default (SENSEHIVE_TAG): secure topics (devices publish encrypted and signed messages with a key SenseHive issues, and their topics appear by themselves) and a heartbeat recycle bin for retired nodes. A node in SenseHive's bin leaves its read API; EdgeShell keeps its last known state, as for any node SenseHive no longer lists.
• The sandbox checks that the bundled SenseHive serves secure topics through the production nginx and that their API needs a SenseHive sign-in.

[1.14.1] - 2026-10-17

Security

• Hardware and webhook errors shown to people and API clients are fixed texts looked up by a code, never an exception's own text, as everywhere else in EdgeShell (two CodeQL stack-trace-exposure alerts from 1.13.0). Messages no longer repeat node names. A test checks every code has its text and that no route returns str(e).

[1.14.0] - 2026-10-17

Added

• Field guide inside the app (/guide/): an animated, full-screen tour for site technicians and operators with Nodi, a small robot guide. Report a fault, start and close a job, confirm a swapped SD card or board, replace a dead node, what the signs on a node card mean, how serious is serious, three golden rules. A phone taps through the real screens; one headline per scene; plays by itself, swipe, scroll, arrow keys or a topic menu; every scene fits any screen and zoom (checked by the browser test from a phone to a 200 % zoomed laptop). It replaces the text guide on the Hardware page.
• An (i) button in the top bar of every page and a Field guide link in the sidebar open it; its close button returns to the page it was opened from (only ever an EdgeShell page).

Changed

• The account button in the top bar now looks like a button: a border, a down arrow, a hover and focus state, and a label for screen readers.

Fixed

• The browser test's Recycle Bin purge check waits for the keyboard focus before typing (it failed now and then on the release run).

[1.13.0] - 2026-10-17

Added

• Hardware lifecycle (docs, ADR 0015). A node is a position; the parts serving it (board, SD card or disk, network adapter, OS install, power supply) have service periods.
  • Every check reads each node's hardware identity with the metrics (no extra connection, no root). A swapped part shows as Hardware changed, named for people ("SD card or disk replaced", "Board replaced (the same SD card in new hardware)", "Operating system reinstalled"). Operators confirm it with the reason (a fault is counted) or reject it as unexpected, which flags the node until its hardware matches the record again.
  • Replacement nodes: adding a node that looks like the replacement of a broken one (same SD card, location, similar name) asks "Is this a replacement?". Yes retires the old node, kept with its history, and joins both histories.
  • Fault reports by anyone, including monitors, from the node card, several selected nodes, the Hardware page or the API. Operators and administrators mark them in progress, add notes and resolve them (replaced, repaired, reseated, no fault found), which records the time to fix and the downtime.
  • Low power: Raspberry Pi under-voltage shows on the card and marks power supply faults reported soon after.
  • Hardware page with open faults, detected changes, parts in service and the numbers: faults per component and month, faults per 100 nodes, mean time to fix, downtime, replacements and service life per kind of part; CSV of faults, parts and changes; a guide. Fault categories are editable by administrators.
  • API: hardware endpoints, and a narrow report permission (every role) for field forms and ticketing tools. Hardware events in the event feed.
  • Signed webhooks (HMAC-SHA256, secret shown once and stored encrypted) for fault and change events and node replacements, refusing addresses inside the server's own networks.

Changed

• Earlier saved API policies gain the report permission unless an administrator turns it off.

[1.12.1] - 2026-10-16

Fixed

• Heartbeat history always shows an outage that is still going on, with SenseHive 1.6.1: an offline or silent node shows how long it has been down since its last heartbeat (and when its last will arrived), also when it went down before the chosen range or before heartbeat history was kept, and on the 7 and 30 day views. A last will inside an outage no longer splits it into two breaks.

Changed

• SenseHive 1.6.1 by default (SENSEHIVE_TAG): in-app guides, status filters and the history fix above.

[1.12.0] - 2026-10-16

Added

• Remote desktop through EdgeShell at https://<domain>/remoteGUI/<node name>/: the Remote button opens the node's OpenRemoteGUI (noVNC) through EdgeShell's own address, behind its sign-in and roles, with no IP or port shown and no VPN needed on the user's side. The desktop WebSocket (keyboard and mouse) needs a 2-minute ticket from the Remote button; every opening is logged (ADR 0014).

Security

• The same-origin isolation now covers remote desktop pages too: EdgeShell refuses their requests, and neither node app can drive the other or another node's.
• The OpenRemoteGUI status check and the node page no longer contain the node's address.

[1.11.0] - 2026-10-16

Added

• Node-RED through EdgeShell at https://<domain>/nodered/<node name>/: the Node-RED button now opens the node's editor through EdgeShell's own address, behind its sign-in and roles (operators and administrators). The node's IP is never shown and no VPN is needed on the user's side; every opening is in the activity log (ADR 0014).

Security

• Node-RED pages share EdgeShell's address, so EdgeShell now refuses browser requests made by a Node-RED page or by a script without a Referer, is never shown in a frame (frame-ancestors 'none'), gives Node-RED pages a different opener policy, and keeps its cookies away from Node-RED. The SenseHive gate refuses the same requests, one node's Node-RED page cannot drive another's, and no service worker can be installed under /nodered/.
• The node terminal needs a short-lived signed ticket (60 seconds, one user, node and session) that only EdgeShell's own pages can get; a WebSocket without it is refused.

Changed

• Up to 5 administrators (was 3).
• The primary administrator is permanent: the first administrator (owner, super administrator) can no longer step down, and ownership never passes to anyone else. Other administrators can still step down.

[1.10.0] - 2026-10-16

Added

• Heartbeat history on every node: click the heartbeat badge for a chart of the last minute, hour, day, 7 or 30 days (gaps between heartbeats, breaks, last wills; hourly uptime for 7 and 30 days), with uptime, longest break, breaks list and delivery delay. Every role.
• Heartbeat CSV export: from the history (one node, every role) or Export heartbeats in the Node Fleet bulk-action bar (selected nodes, operators and administrators): every heartbeat, online minutes or status changes. Exports are in the activity log.
• SenseHive 1.5.0 in the stack (SENSEHIVE_TAG): heartbeat history, dashboard view cards, dialogs that fit the screen, hostile-input hardening.

Fixed

• Monitoring, History showed "No history available yet" for every node. The page and the server disagreed on the data format, so stored samples were never drawn. The history now has periods (1 hour, 6 hours, 1 day from every sample; 7 and 30 days from the hourly rollups), CPU, memory, disk and temperature lines, down minutes shaded red and hourly uptime bars, and the dialog fits the screen.

[1.9.2] - 2026-10-16

Changed

• SenseHive 1.4.1 in the stack (SENSEHIVE_TAG): EdgeShell's look (light by default), a layout that fits phones to desktops at 67 to 200 percent zoom, and no CDN: its browser code is served by SenseHive itself under a strict Content Security Policy (script-src 'self'). It also refuses topics that could break its SQL, and never sends exception text to the browser.
• nginx sends one copy of each security header on /sensehive/ and keeps SenseHive's CSP.

[1.9.1] - 2026-10-16

Fixed

• Fleet Setup no longer fails with "Unexpected token '<'" on large fleets. The scan checked every node over SSH inside one request and ran past the proxy timeout, which returned an HTML error page.
  • Fleet Setup now opens instantly from the stored results.
  • Each item (Docker, Node-RED, OpenRemoteGUI, heartbeat agent) has its own Check, a background job (16 nodes at a time) with progress on its card.
• The heartbeat agent card in Fleet Setup now shows its count; it stayed empty before.

Added

• Install on selected in the bulk-action bar (administrators): tick nodes, pick Heartbeat agent, OpenRemoteGUI, Node-RED or Docker, and the install starts at once in the background (8 at a time, live progress); no fleet scan needed.
• Heartbeat button on Node Fleet (operators and administrators): fresh heartbeat state from SenseHive in about a second, no SSH.
• Every capability install runs as a background job, so a page never waits for the fleet.

[1.9.0] - 2026-10-16

Added

• MQTT heartbeat: a second, VPN-independent liveness signal per node (docs/HEARTBEAT.md, ADR 0013).
  • The heartbeat agent on each node (/opt/edgeshell-heartbeat, hardened systemd service) sends an encrypted, authenticated heartbeat every 10 s and a signed last will over a public broker (default broker.hivemq.com, TLS). Per-node keys, rotating random topics, no data but the node name and a time.
  • SenseHive joins the Docker stack (SQLite in ./data/sensehive): it verifies the heartbeats and serves them read-only to EdgeShell. Administrators open it from the sidebar (new tab) at /sensehive/, after nginx checks with EdgeShell that they are signed-in administrators.
  • Node cards show heartbeat, LWT offline, silent or "no heartbeat agent" for every role; status changes are events (heartbeat_changed).
  • Uptime counts a node as up when SSH or a verified heartbeat says so: dashboard availability (SLA) and the hourly history; VPN gaps are recorded separately (heartbeat_only).
  • Fleet Setup: install the agent (background job, 8 nodes at a time) and push heartbeat settings (config.yaml) to every node, hot reloaded by the agents without a restart; administrators only.
  • Sandbox end to end: agent on an emulated node, local broker, SenseHive, EdgeShell, settings push and last will.
• SenseHive behind a gate (docs/HEARTBEAT.md): administrators only.
  • The sidebar button opens it directly (a CSRF-protected POST that issues a pass).
  • A direct visit to /sensehive/ asks for a fresh 2FA code first; bots never see SenseHive's login.
  • The pass is HttpOnly, SameSite=Strict, valid only for /sensehive/, and bound to the EdgeShell session.
  • nginx never forwards EdgeShell's cookies to SenseHive and rate limits SenseHive's sign-in.
  • SenseHive 1.3.0 adds salted password hashes, a lockout, CSRF refusal and strict cookies; SENSEHIVE_ADMIN_PASSWORD sets its first password.
• CI renders the real nginx configuration and runs nginx -t; the sandbox runs the production nginx and tests the gate end to end.
• log_activity records who started a background job.

[1.8.5] - 2026-10-15

Security

• The runtime image has no pip: nothing uses it at run time, and it bundled its own older copies of urllib3, msgpack and setuptools that scanners reported (the app's own packages were already current). CI checks that it stays out.

[1.8.4] - 2026-10-15

Security

• Log Viewer: shell injection fixed. Browsing and viewing logs put the typed path into a sudo command on the node, and these pages are open to every signed-in role, so a monitor account could run commands as root on a node. Every path sent to a node (Log Viewer browse, view, download and new folder, and file deploy) is now validated (absolute, no control characters, at most 1024 characters) and shell-quoted, and the line count is a bounded number. Tests send injection payloads as a monitor.
• Names from nodes can no longer become script in the browser: file and folder names, sizes and errors in the Log Viewer, and custom commands in the terminal, go into escaped data attributes read by event listeners instead of inline handlers (CodeQL js/xss-through-dom, js/incomplete-sanitization).
• Error messages never carry exception text (CodeQL py/stack-trace-exposure, 13 places): one helper (app/services/errors.py) maps a failure to a fixed description (authentication failed, timed out, connection refused...) and writes the details to the server log. /healthz no longer shows database error text.
• Sign-in redirects accept only a path on this site, checked with a URL parser (CodeQL py/url-redirection).
• HTML stripping for email text cannot backtrack (CodeQL py/polynomial-redos).
• SSH connection pool keys no longer hash the password with a fast hash (CodeQL py/weak-sensitive-data-hashing).
• run.py never turns on the Werkzeug debugger (CodeQL py/flask-debug).
• Native installer: generated passwords are hidden in every console message, and files are created with their final permissions (no moment where a secret is readable).
• Image: the base image's own pip and setuptools are removed (the app runs from its virtualenv); CI rebuilds the dependency layer every week so the scanned image never runs on stale packages; urllib3>=2.8.0.
• The Tailscale key placeholder in .env.example is empty (secret scanning flagged the old placeholder text).

Added

• GitHub Wiki generated from the docs (tools/build_wiki.py, workflow wiki.yml): every doc, decision record, the security policy, contributing guide and changelog as wiki pages with a sidebar, rebuilt on every documentation change on main. CI fails on broken wiki links.

Changed

• Password reset by link is one page with 2FA: the new password and a code from the authenticator app (or a backup code). The link proves the mailbox and the code proves the device, so a stolen inbox alone cannot change a password. Wrong codes count toward the account lockout; the password is checked first so a typo never spends a backup code.

[1.8.3] - 2026-10-15

Added

• Activity log: filter by user (administrators): a user list on the Activity Log (everyone who appears in it, including deleted accounts and system), a click on any username shows only that user's entries, and GET /api/v1/activity?user=<username>. Indexed, so it stays quick on a long log.

• Email guide (docs/EMAIL.md): what EdgeShell sends, free providers compared (Resend, Brevo, useSend, Gmail, Amazon SES), DNS and key setup for each, putting the secret on the server without it appearing anywhere, testing before users depend on it, troubleshooting, and keeping credentials safe.

Fixed

• Startup schema upgrades also run with mariadb:// database URLs (they were skipped; mysql:// URLs, as in the Docker stack, were not affected).

[1.8.2] - 2026-10-15

Added

• Verified email for every account when outgoing mail is set up: after 2FA at the first sign-in, a link is sent to the address and the app opens once it is confirmed. A new address is used only after its link is opened (older links stop working). Accounts from before confirm their address at the next sign-in. Without mail, nothing changes.
• Password reset by email for every role (to the verified address only; 2FA stays on). Accounts without a verified address still go to the administrators.
• Admin alerts in the sidebar: open tickets on Issues; reset requests and sign-ups waiting on User Management.
• Ticket notifications are sent over SMTP when useSend is not configured.

Security

• Email address checks use a pattern that cannot backtrack (CodeQL py/polynomial-redos), with a length cap.

Changed

• The activity log is for administrators only: the page, its sidebar link, the dashboard's Recent Activity and Recent Errors panels, and GET /api/v1/activity (403) are no longer available to operators and monitors. Node logs in the Log Viewer are unchanged.
• The login page's username field no longer suggests an account name.
• Installing software on nodes is for administrators only: Fleet Setup, Install on Fleet and the OpenRemoteGUI install from a node card. Operators see the same as monitors there; they can still open OpenRemoteGUI where it already runs.
• API page in the sidebar (Intelligence, API) for every role: your sessions and service keys, what your role allows on this server, the API address, a quick start, and links to the OpenAPI spec and guide (administrators also get a link to Administration, API Access). It replaces the API section on My account.
• API guide as a readable page: the API page's "API guide" opens docs/API.md in a new tab as one formatted page with a table of contents (built by tools/build_api_guide.py; CI fails if it drifts from the doc). The raw OpenAPI JSON stays available for tools.

[1.8.1] - 2026-10-15

Added

• Administration, API Access: turn the API or parts of it (read, write, control, admin) off, stop new service keys, block any user's API access (every credential of theirs is revoked), see every credential of every user (purpose, permissions, addresses, expiry, last use) with Revoke, and a usage log of every API request (user, credential, endpoint, result, time, address; never bodies or values), read from the data lake.
• Service keys need a purpose, shown to administrators and in the logs.
• The Docker Hub description is pushed whenever docker/DOCKERHUB.md changes on main (as well as with every release), and CI fails if it or the README falls behind VERSION. It now opens with the project banner and badges, like the README.
• API request log table, kept for good in the data lake (API_LOG_RETENTION_DAYS, default 30, on the hot database).

Changed

• Nothing in the app removes data from the data lake: clearing the activity or node log, permanently deleting a node and deleting or rejecting a user now act on the hot database only (refused while the lake is behind), and the lake keeps the rows. Deleted nodes and users are renamed first so names can be reused without breaking replication.
• The app's database account on the lake is SELECT only (lake-init.sh).
• Deleting a user with support tickets is refused with a hint to deactivate (it used to fail on MariaDB).
• Session and expired-credential housekeeping, and sign-out, leave the lake's copy in place; db-import --wipe is refused on a database with data while a lake is configured.

Security

Fixes from an end-to-end review before release:

• Only the owner changes the API policy, and an administrator cannot lift an API block on their own account.
• Revoking another user's service key through the API needs the admin part of the API turned on and follows the owner rules (other administrators' keys only by the owner); listing every key needs the same.
• The API usage log no longer records requests to unknown paths (they bypass rate limits), stores a failed sign-in's username only when the account exists (never a password typed into the field), and is written in its own transaction.
• A full-length node or user name no longer loses its unique suffix when deleted, which could stop replication when the name was reused.
• Deletes stay off the lake with mariadb:// database URLs too, and the unlogged connection is discarded rather than reused if resetting it fails.
• A delete refused because the lake is behind changes nothing (no rename, no revoked sessions, no 2FA removal).
• The Docker Hub description action is pinned to a commit.

Docs

• Decision records 0008 to 0012: why the data lake, the API control model, the fleet safety guards, CI as the release gate, and the CIA triad security model.

[1.8.0] - 2026-10-15

Added

• Public API (/api/v1, docs/API.md, OpenAPI at /api/v1/openapi.json generated from the code). Sign in with password and 2FA (/auth/login, /auth/verify) for an API session, or create a service key for unattended integrations (on My account or POST /keys; needs the password and a fresh 2FA code; chosen scopes up to the role, lifetime, optional IP allowlist; shown once, stored as a hash). Every user has their own credentials, and every API action is audited with the user and the credential.
• Data points for other systems: latest metrics per node, time series (raw, hourly, daily), fleet summary by category, and an event feed (status changes, paused and resumed logins) read with a cursor, so alerting and automation tools can build thresholds and triggers. Node management, bulk login updates, exports and job progress through the API; shell commands only when API_ALLOW_COMMANDS=true.
• API protection: per-credential rate limits by role, sign-in limits and lockout shared with the web app, strict validation (unknown parameters and fields refused, 1 MB body limit), JSON errors, no-store and nosniff headers, no cookies for token requests, browser cookies read-only (no CSRF exposure).
• Data lake: mariadb-lake, a live replica of the hot database that keeps every row (2FA data excluded). Exports, history, events and activity read from it when caught up. Raw-metric and event clean-up on the hot database now skips the lake, so all history is kept.
• Hot retention policy (Settings, Data lake and hot retention): off by default; an administrator can keep only the last week, month, quarter, semester or year on the hot database. Nothing is deleted unless the lake is replicating, caught up and holds every row.
• Command policy enforced in the fleet terminal and the API: built-in rules (wiping the root filesystem, writing over disks, mkfs, fork bombs) plus the administrators' blocked commands.
• Stricter CI: Bandit, pip-audit and a no-SQL-from-input check (Security scans), CodeQL (Python and JavaScript), a coverage floor of 74 percent and at least 170 unit tests, the data lake replication test on real MariaDB, and an API step with real 2FA in the sandbox. New pull request checklist with a security section.
• Docs: docs/API.md, docs/TROUBLESHOOTING.md (every production issue met so far, with cause and fix), data lake in the EC2 guide and architecture, accepted risks and social engineering in SECURITY.md.

Changed

• Breaking for API clients: GET /api/v1/nodes and GET /api/v1/activity return {"nodes": [...], "pagination": {...}} and {"activity": [...], ...} instead of a bare list. /api/v1/status needs sign-in; use /api/v1/health for unauthenticated checks.
• Settings, Security shows the limits the server enforces (read only) instead of fields that were saved but never used.
• Settings, System Info shows the real version, Python, Flask and database engine (it was hardcoded as 1.0.0 and SQLite).
• Clearing the activity log is itself recorded.

Upgrade

• Download lake-init.sh next to compose.yaml and the new compose.yaml, then docker compose up -d. The hot database restarts once to turn on its binary log; lake-init copies it to the lake and starts replication.

Documentation

• EC2 guide: one canonical address (why a domain added later shows "Not secure" and jumps to the IP while the server is still set to the IP), and how to move from the IP to a domain without downtime (certificate first, then switch; the IP keeps redirecting). Troubleshooting entry added.

[1.7.4] - 2026-10-14

Documentation

• Tailscale SSH on the nodes (docs/VPN-NETWORK.md): why every node can show "login failed" while ping, Node-RED and the VPN page are fine; how to recognise it; the default check rule; that a matching check rule overrides accept whatever the order; three fixes (a non-overlapping accept rule, tags, or Tailscale SSH off); how to change the policy safely and verify it (tailscale debug netmap, a test login, root still checked); and that sudo still uses the node's password. Linked from the EC2 guide's troubleshooting table, the README's Tailscale step, and the messages on node cards and Node Fleet.

Changed

• CI runs in two stages. The gate (lint, secret scan, unit tests on Python 3.11 to 3.13, and the browser UI test with 1500 nodes on every screen size and zoom level) runs first; the image build and the amd64 and arm64 sandbox start only when it passed. The browser UI test is now a required check on main.
• Test counts are enforced. Unit tests fail the gate unless every test passed, none was skipped and at least 110 ran (tools/test_gate.py); the browser test needs at least 375 checks (--min-checks). Counts appear on each run's summary page.
• The cd-persistence unit test is skipped on Windows only (it needs a POSIX shell); CI runs it on Linux.

[1.7.3] - 2026-10-14

Fixed

• Recycle Bin: "Delete all permanently" accepted no typing. The open side panel kept the keyboard focus inside itself, so DELETE could not be typed in the confirmation. Dialogs now close the panel first and put the cursor in their first field. The browser UI test types DELETE with real key presses and empties the bin.
• Monitoring polled every node live, one request per node, every 15 seconds. An older second pollAll on the page replaced the fast one, so each refresh opened an SSH collection on every node from every open Monitoring tab (and, before 1.7.1, retried failing logins that often). With a large fleet the browser ran out of connections and dropped other requests. The page now reads the background collector's latest results in one request, as designed; a test fails the build if a page defines a function twice.
• Monitoring "Auto" refresh is remembered. Switching it off used to last only until the page was left. The choice is now saved on the account and kept until changed: across pages, sign-outs, sign-ins and devices (each user has their own), also when switched while the page is still loading.

[1.7.2] - 2026-10-14

Added

• Bulk actions on Node Fleet. Selecting nodes shows a bar with Edit login and Delete. Edit login sets the SSH username, password and/or port of every selected node (empty fields keep each node's value), clears paused logins and checks each node once in the background with a progress card; the password is never logged. Delete moves the nodes to the Recycle Bin (monitoring and SSH stop, nothing on the nodes changes). Select All and every bulk action use only the nodes shown by the current filters.
• Recycle Bin: Restore all, and Delete all permanently (administrators, type DELETE), which also removes the nodes' monitoring history.
• Re-adding deleted nodes by import: a node whose name is in the Recycle Bin comes back with the imported details (same node, history kept) instead of being skipped as already present.

Fixed

• Permanent delete failed on MariaDB for a node with monitoring history (rows still referenced it); the history is now deleted with it.
• Node names in the delete confirmations are escaped for JavaScript.

[1.7.1] - 2026-10-14

Added

• Node login policy. EdgeShell no longer keeps logging in to a node whose login fails: after 2 rejected passwords in a row (LOGIN_PAUSE_AFTER_FAILURES), or one refusal by Tailscale SSH, automatic and bulk logins to that node pause, and the pause survives restarts. Recovery without guessing: every 30 minutes (LOGIN_RECHECK_MINUTES) the SSH banner of each paused node is read without logging in, a changed SSH server gets one new attempt, and one canary login tests the nodes paused by Tailscale SSH (when it works, they all resume). Fix login, Test on a card and "Retry paused logins" try once. The Node Fleet page lists paused nodes, cards show a badge, Test All skips them, and every pause and resume is in the activity log. Nodes already failing when 1.7.1 starts are paused at once instead of retried.
• A CI test that fails the build if any code opens SSH without the guarded helper.

Fixed

• File handle leak in SSH connections. A login that failed (wrong password, time-out, refused) left its half-open connection and transport thread behind. With a few dozen nodes failing every 10 minutes the web process ran out of file handles after about three hours, then crashed and restarted, and new connections failed. Every SSH connection now goes through one helper that closes it on any failure.
• Tailscale SSH is named as the cause. When port 22 of a node is answered by Tailscale SSH and the login is refused or held for a browser check (an SSH policy rule with action check), the card says so instead of "check the SSH username and password", which no password change can fix. See the EC2 guide's troubleshooting table.

[1.7.0] - 2026-10-14

Added

• Stats export on the dashboard. Choose a period (last 24 hours, 7 days, 30 days, quarter, semester or year), a category (or all) and the detail (one row per node per hour, per day, or for the whole period). The CSV is built in the background with a progress bar, then downloads; the job card on every page also offers the download. Columns: uptime percent, samples online, offline and with a login failure, average and peak CPU, memory, disk, temperature and load, reboots, network traffic, and each node's category, type, location and environment, ready for a data lake and health-prediction models.
• Long-term history. One sample per node per minute whatever its state, so uptime is measured, not guessed (offline and login-failed minutes used to leave no trace), now with temperature and network counters. Hourly rollups are kept for good.
• VPN / Tailscale peer filters: status, operating system, link (direct, relayed, idle, none) and last seen (last hour, 24 hours, 7 days, or not for 7+ days), clickable column sorting, a "showing X of Y" count and Clear; choices are remembered in the browser.
• Browser UI test (tools/ui_test.py, a CI job): 1500 nodes, every page on phone, tablet, laptop and desktop sizes and at 67 to 200 percent zoom. It checks that nothing sticks out of its card, no page scrolls sideways, no word or number is broken over lines and no script fails, then drives the filters, Select All, dialogs, the fleet terminal, the VPN filters, the export and a real SSH terminal on an emulated Raspberry Pi.
• A menu button on phones and narrow windows: the sidebar was unreachable below 768 px.

Fixed

• Monitoring: the Restart button was hidden from everyone (a role check compared against capitalised role names); node names in its buttons are now escaped for JavaScript, and long names truncate.
• OpenRemoteGUI install runs sudo bash install.sh (./install.sh fails when the clone has no execute bit).
• Node Fleet card actions wrap inside the card (Delete no longer sticks out).
• Phone and zoomed layouts: the top bar, the Node Fleet and Monitoring header buttons, the fleet terminal (its output pane was squeezed to 36 px on phones), Monitoring and Users stat cards (numbers and words broke mid-word), table columns (emails and IPs broke letter by letter; wide tables now scroll inside their card), dashboard metric values and the import sample.

[1.6.0] - 2026-10-14

Added

• Interactive SSH terminal per node. A terminal button on each Node Fleet card opens a real shell on that node in a window (xterm.js over a WebSocket, a real PTY on the node): colors, top, nano, tab completion, Ctrl+C, and sudo prompting like any SSH session. Closing the window ends the SSH session on the node; it also closes after 30 minutes without input or as soon as the EdgeShell sign-in ends. Operators and administrators only, same-origin only, at most 4 per user and 12 per server, and every open and close is in the activity log.
• Guided OpenRemoteGUI install with live steps and log, one node at a time: safety checks (OS, free disk, network to GitHub, desktop session, sudo) that skip unsuitable nodes without touching them, then git clone, then cd OpenRemoteGUI && sudo ./install.sh, then a check that the page answers. "Install on Fleet" stops at the first failed install so a problem cannot spread across production nodes. The confirmation says what the installer changes (it turns off Raspberry Pi Connect's own VNC).
• A live job window (steps per node and the log) for long jobs, opened from the progress card's Details link.

Changed

• Fleet terminal runs in parallel and streams. Offline and login-failed nodes are skipped up front (and listed), the rest run 16 at a time over the collector's open SSH connections, and each node's output appears as soon as it finishes instead of after the slowest one.

Fixed

• sudo hung until the timeout on nodes that ask for a password (the OpenRemoteGUI install showed no progress for 10 minutes and the terminal's sudo switch did the same): there was no way to answer the prompt. sudo now gets the node's stored password on stdin, never on a command line and never echoed.

[1.5.1] - 2026-10-13

Fixed

• OpenRemoteGUI was reported as installed on every reachable Raspberry Pi. The check counted any wayvnc or websockify process, but Raspberry Pi OS runs its own wayvnc (and Raspberry Pi Connect's rpi-connectd) whenever VNC is enabled, and pgrep -f also matched the checking shell's own command line. OpenRemoteGUI now counts only when its install (/opt/openremotegui, /etc/openremotegui or a ~/OpenRemoteGUI checkout) is present and its web gateway port (6080 by default) is listening; "installed but not running" is reported separately. On the live 73 online nodes this changed the result from 73 installed to the 2 nodes that really serve the OpenRemoteGUI page.
• The metrics collector now refreshes every node's OpenRemoteGUI state on each pass, so the Remote button is accurate without a manual scan. Earlier wrong results are cleared once on upgrade.
• Node-RED "running" now matches the process name (pgrep -x node-red) or the nodered service instead of a command-line search that could match the checking shell.

[1.5.0] - 2026-10-13

Added

• Device categories on the dashboard. A row of cards, one per category (default: Pi Nodes, Edge Gateways, Mini PCs, matched by node type, plus an automatic "Other"), each with online/total, availability against its own SLA target, average CPU and RAM, and login-failed and offline counts. Clicking a card scopes the SLA bar, stat cards, charts and node grid to that category.
• Administrators edit categories on the dashboard: add, rename, remove and reorder them, choose which node types belong to each, an icon and an SLA target. The editor lists the node types found in the fleet with their counts.
• Node Fleet has a Category filter (also /nodes/?category=<name>), and the status filter has "login failed".

Fixed

• The last-seen time could overflow node cards on Node Fleet; the badge row now wraps.
• Node tags separated by semicolons (pi-node;ts-online;tailscale) showed as one tag; commas and semicolons both separate tags now.

[1.4.0] - 2026-10-13

Fast fleet monitoring. Measured on a live 122-node Tailscale fleet, one full refresh went from about 11 minutes to under 30 seconds cold and 5 to 8 seconds warm.

Added

• Background metrics collector. One SSH connection per node is kept open and reused; metrics come from a single shell command reading /proc (no Python started on the node); the fleet is polled in parallel (COLLECTOR_WORKERS, default 48) every COLLECTOR_INTERVAL seconds (default 30), outside any web request. Pages read the stored results, so the dashboard and Monitoring load instantly; "Refresh All" collects live, in parallel.
• Login problems show on the card. Failures are classified: wrong SSH username or password shows "Login failed" with a Fix login button on the dashboard, Node Fleet and Monitoring cards, which saves the new username, password and port and re-checks the node at once. Unreachable nodes show the reason.
• Failing nodes back off instead of being retried every cycle; a wrong password is retried only every 10 minutes, so a node's own fail2ban does not lock EdgeShell out. Saving new credentials retries immediately.
• Adding a node returns at once; its login is checked in the background and the card updates within seconds.
• tools/perf.py: times every page, API and node action against a large simulated fleet. CI fails if any read endpoint's median exceeds 300 ms with 300 nodes.

Changed

• Node password encryption derived its key (PBKDF2, 100,000 rounds) on every encrypt and decrypt, 17 ms each: a 500-node import or metrics pass spent over 8 seconds on it. The derived key is now cached per process (ciphertext unchanged): 0.01 ms per call. A 500-row import commit went from 8.6 s to 0.1 s.
• Bulk import checks the new nodes in batches through the collector (parallel SSH, one database writer): 500 nodes in about 4 s at 300 ms per node. It no longer runs the OpenRemoteGUI check per node; use "Check OpenRemoteGUI" on Node Fleet.
• Metrics history is stored at most once a minute per node, and history older than METRICS_RETENTION_DAYS is purged automatically.

[1.3.5] - 2026-10-13

Ships the 1.3.4 changes below; the 1.3.4 tag was never published because a timing-sensitive test stopped its release.

Fixed

• The bulk import test read job progress through the app while the job was writing; with the in-memory test database that could roll a write back. It now watches the job registry directly.
• Terminal commands, Select All and other page actions failed on 1.3.3 servers with Unexpected token '<' ... is not valid JSON: the shared page script did not run, so requests lacked their CSRF token (fixed in the changes below).

[1.3.4] - 2026-10-13 (not published)

Fixed

• Buttons and actions driven by JavaScript stopped working on every page in 1.3.0 (Select All, toasts, the session watcher, the CSRF header on background requests): an apostrophe in the "still there?" prompt ended a JavaScript string early, so the shared page script never ran. Fixed, and a test now renders every main page and checks each inline script with Node.js.
• Bulk import returned 502 for larger files. Every parsed row was stored in the session cookie, which outgrew nginx's header buffer (and browsers' 4 KB cookie limit). Rows now wait on the server under a single-use token; nginx also has larger header buffers.
• Bulk import waited for SSH checks of the whole fleet inside the request, one node at a time. It now creates the nodes at once and checks only the new ones in the background (8 in parallel on MariaDB), with a progress card at the bottom of every page; you can keep working or refresh.
• Long node names pushed status badges out of the dashboard cards and under the checkbox on Node Fleet: a missing min-w-0 utility meant flex items could not shrink. Names now truncate (full name on hover), and cards, badges and table cells are guarded against overflow app-wide.
• The Terminal node list grew past the screen with a large fleet. The list and the output now scroll inside their panes, and on phones the panes stack.

[1.3.3] - 2026-10-13

Fixed

• Log Viewer, Activity Log and node log pages returned 500 (Internal Server Error) in the published image. A .gitignore rule meant for runtime logs/ folders also matched app/templates/logs/, so those three templates were never committed; local builds had them, the CI-built image did not. The rule now applies only to /logs/ at the project root, the templates are in the repository, and a test fails CI if any template the code renders is missing from git.

[1.3.2] - 2026-10-13

Fixed

• Browsers logged a 404 for /favicon.ico on every page. EdgeShell now has an icon, linked from every page, and /favicon.ico redirects to it.
• nginx reported itself unhealthy in HTTPS mode after the server's address changed: its health check followed the HTTP to HTTPS redirect to the public address. /healthz is now answered directly on port 80.
• privacyIDEA signed its internal session tokens with a random 24-character key and warned about the key length. The stack now passes a dedicated MFA_SECRET_KEY (add it to an existing .env with echo "MFA_SECRET_KEY=$(openssl rand -hex 32)" >> .env).

[1.3.1] - 2026-10-13

Fixed

• 2FA setup QR code could not be scanned. The page sized the code with CSS but the SVG had no viewBox, so browsers cropped it instead of scaling it, cutting off two of its three corner markers. It now scales, with the standard quiet zone; verified by decoding a browser screenshot of the page. The manual setup key was always correct.
• The VPN Status page never showed data: it read field names the service did not return, and in Docker there is no tailscale CLI. It now reads tailscaled's LocalAPI through the host's socket (mounted read-only; tailscaled grants non-root users status only), falls back to the CLI on native installs, and escapes peer names.

Added

• VPN Status refreshes every 10 seconds: connection state, this server's Tailscale IPs and DNS name, tailnet, version, health warnings, exit node, and every peer with online state, direct or relayed link, last seen and traffic, with a filter.
• docker/host-hardening.sh: fail2ban (SSH, repeat offenders, nginx rate-limit trips and vulnerability scanners, with web bans in Docker's DOCKER-USER chain), a key-only SSH policy, log rotation and automatic security updates.
• HTTPS without a domain: set EDGESHELL_DOMAIN to the server's public IPv4 address and certbot gets a free Let's Encrypt certificate for the IP (short-lived profile, renewed automatically when a third of its life is left).
• nginx TLS follows Mozilla's intermediate profile (TLS 1.2 and 1.3, forward-secret AEAD ciphers, ECDSA keys), with HSTS and Cross-Origin-Opener-Policy. Access and error logs are also written to data/nginx-logs for fail2ban.
• CI fails if an em dash appears anywhere in code, docs or the UI.

[1.3.0] - 2026-10-13

Accounts release: mandatory two-factor authentication, account recovery, administrator governance, sign-up approval and device-aware sessions.

Added

• Mandatory two-factor authentication for every account: an authenticator app (TOTP, any standard app) plus 10 one-time backup codes shown once to save privately. Set up at first sign-in; nothing is signed in until the second factor passes, and failed codes count toward the account lockout.
• In the Docker stack the codes are checked by privacyIDEA (open-source MFA server) in its own container on the internal network, with no published port and no internet route. EdgeShell stores only token serials; the secrets stay in privacyIDEA, encrypted with its own key in ./data/privacyidea. Installs without that container (run.py, single container) use a built-in verifier with the secret encrypted by CREDENTIAL_KEY and backup codes hashed.
• My account page: 2FA status, backup codes left, new backup codes, reset 2FA for a new phone, personal email, signed-in devices with "Sign out" for each.
• Password recovery. Users send a reset request from the sign-in page; an administrator sets a temporary password, which also removes that user's 2FA so it is set up again. Administrators recover by an emailed reset link (any SMTP server, SMTP_* and PUBLIC_URL in .env) or with one of their backup codes. Reset links are single-use, expire after 30 minutes and are never built from the request's Host header.
• Administrator governance. At most 3 administrators. The first administrator is the owner (primary administrator) with full control: the only one who can grant or remove the administrator role, and able to change, reset, sign out, disable or delete any other account. Other administrators manage monitors and operators only, and nobody can change the owner. Administrators step down themselves with their password, a 2FA code and a typed confirmation; the last administrator cannot. When the owner steps down, ownership passes to the longest-serving administrator. The owner is prompted to add more administrators.
• Administrators must add a personal email (where reset links go) before using the platform. The first sign-in goes: 2FA, personal email, new password.
• nginx replaces Caddy in the Docker stack and is the only container with published ports. Plain HTTP on the server's IP until EDGESHELL_DOMAIN is set; then a certbot container gets a Let's Encrypt certificate and nginx switches to HTTPS by itself, redirecting HTTP and the bare IP. Security headers, HSTS with TLS, a rate limit on sign-in and recovery form posts, and client IPs that cannot be spoofed through X-Forwarded-For.
• Self sign-up (SIGNUP_ENABLED, on by default): requests wait for an administrator, who approves with a role (monitor or operator; administrator only by the owner) and a device limit, or rejects them.
• Device-aware sessions. Each session records its device type (Desktop, Mobile, Tablet) and the time of the last real interaction (click, key press, page load; background polls do not count). Users and administrators see and close individual devices. The activity log records which device did what.
• Inactivity sign-out. After IDLE_MINUTES (10) without interaction a session shows "Are you still there?" with a countdown, and signs out unless answered within IDLE_WARNING_MINUTES (5). The server enforces the same deadline, so a closed laptop or forgotten tab is signed out too. Tabs of the same session share the timer.

Security

• The default admin / admin123 account is created only in an empty database. Previously it was re-created whenever no user was named admin.
• Passwords are at least 10 characters (temporary ones included) and cannot be reused on change.
• Admin password resets in User Management had no password field and always failed; resets now ask for a temporary password (a random one is suggested).

Fixed

• python3 run.py stopped at the Nginx step on Ubuntu 26.04 LTS: its nginx.conf sets server_tokens itself and nginx rejects the duplicate. The installer now switches that line to off (backup kept) instead. Verified on Ubuntu 26.04.1 (Python 3.14, MariaDB 11.8, PHP 8.5): fresh install, a re-run over a half-finished install, a repeat run and a reboot.

[1.2.0] - 2026-10-12

Container-first release: multi-arch image, a full sandbox with Pi Fleet Emulator, and stronger session security.

Added

• Official multi-arch image devprincekumar/edgeshell (also ghcr.io/855princekumar/edgeshell) for linux/amd64 and linux/arm64, so it runs on x86 clouds, AWS Graviton, Ampere Altra and Raspberry Pi 4/5. Non-root, read-only code, tini as PID 1, SBOM and provenance attached.
• sandbox/: EdgeShell plus a 6-node Pi Fleet Emulator fleet (32-bit and 64-bit across Buster, Bullseye and Bookworm) with an end-to-end suite covering import, SSH, architecture detection, fleet commands, monitoring, file push and pull, an OTA-style rollout, Ansible, session limits, brute-force protection, RBAC and restart persistence. Run it with sandbox/run.sh.
• CI on every push: lint, secret scan of the full history, unit tests on Python 3.11 to 3.13, amd64 and arm64 image builds with a hardening audit and vulnerability scan, and the sandbox suite on native amd64 and arm64 runners. Tagged releases publish only after all of it passes.
• Bulk import accepts JSON, including Pi Fleet Emulator's pi-fleet-nodes.json, plus a default SSH password and a host.docker.internal replacement option.
• Node addresses may be hostnames (container names, host.docker.internal, pi.local), not only IPv4.
• run.py enables a host firewall (ufw) on EC2: inbound only for the sshd port(s), 80, 443, Tailscale 41641/udp and the tailscale0 interface. The SSH port is read from sshd -T, so the admin cannot be locked out. EDGESHELL_FIREWALL=off skips it.
• Production Docker Compose (docker/compose.yaml): EdgeShell, MariaDB 11.4 on an internal network and Caddy with automatic HTTPS; all state in a host ./data folder; services start in health order and restart on reboot. Documented on the Docker Hub page.
• Exposure audit at the end of run.py: every listening socket is checked and anything reachable beyond loopback or the tailnet is reported.
• README screenshots, CODE_OF_CONDUCT, issue forms, pull request template, and private vulnerability reporting.
• Per-user session limits: each account allows 1 device by default (admins included). Administrators can set 1 to 4 when creating a user or later in User Management. Signing in past the limit asks to sign out the least recently active device.
• Sessions are stored server-side in a new user_sessions table; the MariaDB view v_user_sessions now lists every session.
• "Sign out everywhere" action per user in User Management.

Security

• An admin password reset now signs the user out of every session. Previously an existing session stayed valid until it expired.
• Changing your own password signs out your other sessions.
• Disabling a user deletes their sessions, so re-enabling the account does not revive an old cookie.
• Logging out invalidates the session server-side even when SINGLE_SESSION=false.
• Login failures (unknown user, wrong password, locked account) show one identical message and take the same time, so usernames cannot be discovered from the login form.

Fixed

• Enabling or disabling a user from User Management showed a raw JSON page; it now returns to the list with a message.
• Ansible fleet operations failed in the container: ansible-core was missing and node names with spaces broke the inventory. Both fixed; without Ansible, EdgeShell now falls back to plain SSH instead of failing.
• The login rate limit counted page loads, not just password attempts.
• /api/v1/status reported version 1.0.0; it now reads VERSION.
• Monitoring summary counted Docker and Node-RED as 0 because it ran before the node results arrived.
• Dashboard "Fleet Avg Temp" always showed n/a: the fleet summary never computed it.
• Getting Started, Migration and Architecture docs described the old installer, the wrong MariaDB driver and an outdated data model.
• The Docker image ran two Gunicorn workers, which split the in-memory rate limits; it now matches the systemd deployment (one worker, threads).

Verified

• Full run.py install on a fresh Ubuntu 24.04 with systemd: MariaDB, Nginx, phpMyAdmin, ufw, all services enabled; services return after a reboot and the app restarts within seconds after being killed.
• Production Compose stack with the published image: HTTPS login, HSTS, HTTP to HTTPS redirect, and all data surviving removal and re-creation of every container.

Upgrade notes

• Docker users: the image name is now devprincekumar/edgeshell. Existing volumes keep working.
• The schema upgrades itself on start (users.max_sessions, user_sessions table). The old users.session_* columns are no longer used.
• Everyone is signed out once after upgrading and signs in again.

[1.1.0] - 2026-09-30

Production deployment on EC2.

Added

• python3 run.py on Linux installs or updates production end to end: MariaDB, Nginx reverse proxy with Let's Encrypt, phpMyAdmin, systemd, watchdog. Idempotent.
• MariaDB via mysql-connector-python (unix socket); one-time automatic SQLite migration.
• Single active session per account with a confirm-before-takeover prompt; replaced devices are signed out with the reason, idle tabs within 20 s.
• service_events table: every service start/stop classified (host boot, crash restart, watchdog, deploy, shutdown) and mirrored to the Activity Log.
• phpMyAdmin bound to the Tailscale IP only, with basic auth, cookie login, root disabled, dedicated PHP-FPM pool.
• Local-time MariaDB views: v_activity_local, v_service_events_local, v_user_sessions.
• docs/DEPLOY-EC2.md.

Fixed

• Timestamps displayed as UTC. DB still stores UTC; pages render DISPLAY_TZ (default Asia/Kolkata) and APIs emit ISO-8601 with Z.
• db-export wrote node SSH passwords in plaintext (read the decrypting property). Snapshots now carry ciphertext.
• Heartbeat timer was silently rejected by login and CSRF checks.
• Default rate limit (80/hour) was below the dashboards' own polling rate.
• Client IPs behind a proxy were all 127.0.0.1 (lockout, rate limits, audit); ProxyFix added.
• Health endpoint returned 503 on low disk, which would make the watchdog restart-loop.

[1.0.0] - 2026-08-28

First deployment-focused release. Previous iterations were local proofs of concept.

Fixed

• OpenRemoteGUI detection now recognizes noVNC/wayvnc installs (port, directory, or process), so installed nodes are no longer reported as missing.
• Adding a user with a blank email no longer 500s; blank emails get a unique placeholder and duplicate emails/usernames show a friendly message.
• Terminal and Node-RED actions in the dashboard node popup are hidden from monitors.
• Docker detection uses sudo, so nodes where the login user is not in the docker group now report Docker correctly.
• Role checks are now case-insensitive and legacy capitalized roles are normalized, so operators are no longer treated as monitors.
• Monitoring cards were blank because the poll wrapper was not unwrapped and metric keys did not match the backend contract.
• Node-RED button opened localhost with a simulator-derived port; it now opens the node's real IP on its Node-RED port.
• SQLite WAL was assumed but never enabled; now set with busy_timeout for concurrent multi-user use.

Added

• Add Node has a 'quick test node' toggle: SSH details only when on, full identification fields required when off.
• Node fleet: 32/64-bit architecture auto-detected on connection test and capability scan, shown as a badge and a filter.
• User creation now requires a unique username and unique email (case-insensitive), with a live 'already taken' check as you type.
• Admins can change any user's role inline from User Management.
• Fleet Setup: scan Docker, Node-RED, and OpenRemoteGUI across the fleet and install on nodes missing each.
• Issue portal: users report issues visible only to themselves and admins; admins see all. Threaded reply conversation per issue, status workflow, and email notifications through useSend (open-source, Resend-compatible).
• Central role-based access control with login lockout.
• CSRF protection across all state-changing requests.
• Fail-hard on missing production secrets; config-driven credential encryption.
• OpenRemoteGUI integration: per-node detect/install/open, fleet scan with counts, fleet-wide install via Ansible with per-host results.
• Per-node Node-RED and OpenRemoteGUI port configuration.
• Bulk node import from CSV/TXT with dynamic schema inference and a verify step, followed by ping, SSH test, and OpenRemoteGUI check.
• Backend-agnostic VPN status (Tailscale now, WireGuard later) with heartbeat.
• One-command database export/import for SQLite to Postgres/MariaDB migration.
• Health endpoint, watchdog, systemd units, and manifest-based update/rollback.
• Docker image (non-root, healthcheck) and compose with VPN/network options.
• Documentation: README with diagrams, getting started, architecture, RBAC, VPN/network, migration, and ADRs.

Changed

• Removed the committed database and stray build directories from the tree.