Post-backlog hardening + feature batch (26.18–26.30)

README.md

@@ -11,12 +11,12 @@ The system is designed to run as **two cooperating agent instances** — named *
 ## Features
 
 - **Active discovery** — concurrent TCP-probe scanning across configurable CIDR ranges to find live hosts. Optional deep TCP and UDP probe passes per profile.
-- **Asset fingerprinting** — banner-grab on SSH, FTP, SMTP, POP3, IMAP, HTTP, HTTPS (with TLS cert peek), MySQL handshake, Telnet. Stored per-port in `Port.Service`.
-- **Device-type classifier** — heuristic rules over (vendor, OS banner, open ports) tag hosts as printer / router / hypervisor / windows-host / windows-dc / database (mysql|postgres|…) / mail-server / linux-host / appliance / iot-broker / embedded.
-- **MAC + vendor enrichment** — `/proc/net/arp` lookup on Linux + embedded OUI prefix table for ~80 common vendors.
+- **Asset fingerprinting** — banner-grab on SSH, FTP, SMTP, POP3, IMAP, HTTP, HTTPS (with TLS cert peek), MySQL handshake, PostgreSQL (SSLRequest probe), Redis (`INFO`), Memcached (`version`), VNC (RFB greeting), Telnet, plus UDP DNS and NTP (stratum) probes. Stored per-port in `Port.Service`.
+- **Device-type classifier** — heuristic rules over (vendor, OS banner, open ports) tag hosts as printer / router / hypervisor / windows-host / windows-server / windows-dc / nas / database (mysql|postgres|…) / mail-server / dns-server / kubernetes-node / container-host / camera / linux-host / appliance / iot-broker / embedded.
+- **MAC + vendor enrichment** — neighbour-cache lookup on Linux (`/proc/net/arp`) and macOS (routing socket, no shell) + embedded OUI prefix table for ~90 common vendors, including major IP-camera (Hikvision/Dahua/Axis), NAS (Synology/QNAP/WD), networking (Ubiquiti), and IoT (Espressif) brands that also drive device classification.
 - **Per-subnet scan profiles** — aggressive hourly deep scans on critical infra, lazy daily liveness on guest networks, all in one config.
 - **Change detection + alerts** — diffs host inventory each cycle; fires `host.discovered` / `host.vanished` events to HTTP webhook and/or RFC 5424 syslog.
-- **JSON query API** — `/api/v1/hosts` with filters (vendor, device type, hostname, subnet, port) and pagination; `/api/v1/hosts/{ip}` with nested ports.
+- **JSON query API** — `/api/v1/hosts` with filters (vendor, device type, hostname, subnet, port) and pagination; `/api/v1/hosts/{ip}` with nested ports; `/api/v1/scans` paginated scan history (optional `subnet` filter).
 - **Continuous monitoring** — periodic re-scans detect new devices, removed devices, and configuration changes over time.
 - **Mutual watchdog** — two agent instances cross-check each other for liveness, scan freshness, and inventory consistency. Optional mTLS between peers.
 - **Web admin console** — dark-themed browser UI with dashboard, host inventory, per-host port detail, scan history, watchdog peer status; auto-starts alongside each agent.
@@ -241,9 +241,9 @@ Each agent automatically starts a browser-based admin console alongside the scan
 | Page | URL | Description |
 |------|-----|-------------|
 | Dashboard | `/` | Summary cards and latest 10 scans and hosts; auto-refreshes every 30 s |
-| Host inventory | `/hosts` | Full list of all discovered hosts with metadata |
+| Host inventory | `/hosts` | List of discovered hosts with metadata; paginated (`?limit=`, `?offset=`, default 100) |
 | Host detail | `/hosts/{ip}` | Per-host metadata and open port table |
-| Scan history | `/scans` | All subnet sweeps with duration and status |
+| Scan history | `/scans` | Subnet sweeps with duration and status; paginated (`?limit=`, `?offset=`, default 100) |
 
 ### Terminal UI console
 
@@ -337,15 +337,16 @@ Each agent reads a JSON config file and then applies environment variable overri
 | `scanner.subnets` | `[]` | Legacy flat CIDR list. Mutually exclusive with `scanner.profiles`. |
 | `scanner.profiles` | `[]` | Per-subnet override list (see below). |
 | `scanner.scan_interval` | `5m` | How often to re-scan; default for any profile that doesn't set its own. |
-| `scanner.timeout` | `2s` | Per-host TCP probe timeout. |
+| `scanner.timeout` | `2s` | Per-host TCP probe timeout; also bounds reverse-DNS (PTR) lookups. |
 | `scanner.workers` | `50` | GLOBAL concurrent probe cap across every subnet (not per-subnet). |
 | `scanner.max_hosts` | `65535` | Maximum usable addresses per subnet; larger subnets are rejected. |
 | `scanner.probe_ports` | `[22, 80, 443, 8080]` | TCP liveness ports — host alive if any answer. |
 | `scanner.deep_probe` | `false` | Second-pass scan of `deep_probe_ports` on every live host. |
 | `scanner.deep_probe_ports` | `top-services list` | TCP ports for the deep pass when `deep_probe` is on. |
 | `scanner.udp_ports` | `[]` | UDP ports to probe per live host. Empty disables UDP probing. |
-| `scanner.enrich_arp` | `false` | Populate Host.MACAddress + Vendor from `/proc/net/arp` (Linux). |
+| `scanner.enrich_arp` | `false` | Populate Host.MACAddress + Vendor from the OS neighbour cache (Linux `/proc/net/arp`, macOS routing socket). No-op on other platforms. |
 | `scanner.host_ttl` | `0` (disabled) | Hosts not seen within this duration are deleted at the end of each cycle. |
+| `scanner.scan_history_ttl` | `0` (disabled) | Scan-history rows older than this duration are deleted at the end of each cycle, bounding the `scans` table and `/scans` view. |
 | **Scanner — per-subnet profile (each item in `scanner.profiles`)** | | |
 | `subnet` | required | CIDR for this profile. Must be unique. |
 | `scan_interval` | inherits global | Per-profile scan cadence. |
@@ -366,6 +367,7 @@ Each agent reads a JSON config file and then applies environment variable overri
 | `health.client_ca_path` | — | When set, requires mTLS (clients must present a cert signed by this CA). |
 | **Admin console** | | |
 | `admin.addr` | `127.0.0.1:9090` | Listen address for the admin console + `/api/v1/*`. |
+| `admin.auth_token` | — | Shared secret gating the whole console. Required when `admin.addr` is off-loopback. Clients send `Authorization: Bearer <token>` or HTTP Basic with the token as the password. |
 | **Watchdog** | | |
 | `watchdog.peer_addr` | — | Base URL of the partner agent's health server. |
 | `watchdog.peer_token` | — | Bearer token sent to the peer. Must match peer's `health.auth_token`. |
@@ -379,9 +381,9 @@ Each agent reads a JSON config file and then applies environment variable overri
 | **Tracing** | | |
 | `tracing.endpoint` | — | OTLP/HTTP collector URL. Empty = no-op exporter (instrumentation active, spans discarded). |
 | **Alerts** | | |
-| `alerts.webhook.url` | — | HTTP POST target for host.discovered / host.vanished events. |
+| `alerts.webhook.url` | — | HTTP POST target for host.discovered / host.vanished events. Must be `http`/`https`; scheme-validated at startup. |
 | `alerts.webhook.auth_header` | — | Verbatim `Authorization` header (e.g. `Bearer abc123`). |
-| `alerts.syslog.addr` | — | `udp://host:514` or `tcp://host:514`. RFC 5424. |
+| `alerts.syslog.addr` | — | `udp://host:514` or `tcp://host:514`. RFC 5424. Scheme-validated at startup. |
 | `alerts.syslog.tag` | `network-inventory` | APP-NAME field. |
 | `alerts.syslog.facility` | `16` (local0) | RFC 5424 facility number 0..23. |
 
@@ -421,6 +423,7 @@ fails fast if both are set.
 | `INVENTORY_ADMIN_ADDR` | `admin.addr` |
 | `INVENTORY_AUTH_TOKEN` | `health.auth_token` |
 | `INVENTORY_PEER_TOKEN` | `watchdog.peer_token` |
+| `INVENTORY_ADMIN_TOKEN` | `admin.auth_token` |
 
 ## Health endpoints
 
@@ -434,19 +437,20 @@ Both agents expose two HTTP endpoints used by the watchdog and for external moni
 | `/status` | GET | JSON-encoded status snapshot (see below) |
 | `/metrics` | GET | Prometheus text exposition format — counters for scans, probes, DB, watchdog, alerts; gauges for host count + peer-up state |
 
-**Admin console** (default `127.0.0.1:9090`, unauthenticated — keep loopback unless on a trusted segment):
+**Admin console** (default `127.0.0.1:9090`). Unauthenticated on the loopback default; set `admin.auth_token` (or `INVENTORY_ADMIN_TOKEN`) to gate every route below. A token is **required** when binding off-loopback — the agent refuses to start otherwise. Authenticate with `Authorization: Bearer <token>` or HTTP Basic auth using the token as the password (browsers get a native login prompt):
 
 | Endpoint | Method | Response |
 |----------|--------|----------|
 | `/` | GET | HTML dashboard |
-| `/hosts` | GET | HTML host inventory |
+| `/hosts` | GET | HTML host inventory (paginated: `?limit=`, `?offset=`) |
 | `/hosts/{ip}` | GET | HTML host detail (with ports) |
-| `/scans` | GET | HTML scan history |
+| `/scans` | GET | HTML scan history (paginated: `?limit=`, `?offset=`) |
 | `/watchdog` | GET | HTML watchdog peer-status panel |
 | `/export.json` | GET | Full inventory snapshot as JSON |
 | `/export.csv` | GET | Full inventory snapshot as CSV |
 | `/api/v1/hosts` | GET | Filterable JSON list — `?vendor=`, `?device_type=`, `?hostname=`, `?subnet=`, `?port=`, `?limit=`, `?offset=` |
 | `/api/v1/hosts/{ip}` | GET | Single-host JSON with nested ports |
+| `/api/v1/scans` | GET | Paginated JSON scan history — `?subnet=`, `?limit=`, `?offset=` |
 | `/scan` | POST | Trigger an out-of-cycle scan (CSRF-gated) |
 
 ### `/status` response

SECURITY.md

@@ -43,16 +43,16 @@ The following table documents the project's posture against the [OWASP Top 10 (2
 
 | # | Category | Status | Notes |
 |---|----------|--------|-------|
-| A01 | Broken Access Control | ⚠️ Partial | `/health` and `/status` are intentionally unauthenticated for simplicity. The default bind address is `127.0.0.1` (loopback only). Operators who expose these endpoints on a wider interface accept responsibility for network-level access control. |
+| A01 | Broken Access Control | ⚠️ Partial | `/health` and `/status` are unauthenticated on the loopback default but require `health.auth_token` when bound off-loopback (enforced at startup). The admin console (full inventory, exports, JSON API, `POST /scan`) likewise requires `admin.auth_token` when bound off-loopback — the agent refuses to start otherwise. Both default to `127.0.0.1` (loopback only). |
 | A02 | Cryptographic Failures | ✅ Pass | Peer-to-peer watchdog traffic supports TLS (with optional mTLS) — set `watchdog.tls.ca_cert_path` and `health.tls_cert_path`/`tls_key_path` in the configs. TLS 1.2+ enforced. Database is stored unencrypted; operators should apply filesystem-level encryption where needed. |
 | A03 | Injection | ✅ Pass | All SQL queries use parameterized `?` placeholders. No shell commands are invoked; the scanner uses `net.Dialer` directly. |
 | A04 | Insecure Design | ✅ Pass | Health server binds to loopback by default. `peer_addr` is validated to `http`/`https` schemes only, preventing SSRF via alternate URI schemes. No user-controlled input reaches internal APIs without validation. |
 | A05 | Security Misconfiguration | ✅ Pass | Default `health.addr` is `127.0.0.1:8080` (loopback only). HTTP server has explicit read, write, and idle timeouts. Response bodies from peers are capped at 1 MiB. |
 | A06 | Vulnerable Components | ✅ Pass | All dependencies are pure Go (no C libraries). `go.sum` is committed and verified on every build. `govulncheck` is required before dependency PRs (see CONTRIBUTING.md). |
-| A07 | Auth Failures | ⚠️ Partial | No authentication on health endpoints by design. Mitigated by loopback-only default and operator guidance in this document. |
+| A07 | Auth Failures | ⚠️ Partial | Loopback-only defaults are unauthenticated by design. Off-loopback binds of both the health server and the admin console require a shared bearer/Basic token, enforced at startup; tokens are compared in constant time. |
 | A08 | Data Integrity | ✅ Pass | `go.sum` provides cryptographic verification of all module downloads. Config validation rejects malformed or unexpected values at startup. |
 | A09 | Logging & Monitoring | ✅ Pass | Structured `log/slog` output in text or JSON format. All three watchdog failure conditions (liveness, freshness, consistency) are logged at `WARN` or `ERROR` level with structured fields. |
-| A10 | SSRF | ✅ Pass | `peer_addr` is validated to `http`/`https` only at config load time. Response bodies from external HTTP calls are limited to 1 MiB via `io.LimitReader`. Scanner targets come from operator-controlled config, not external input. |
+| A10 | SSRF | ✅ Pass | All outbound targets are scheme-validated at config load: `watchdog.peer_addr` and `alerts.webhook.url` to `http`/`https`, `alerts.syslog.addr` to `udp`/`tcp`. This blocks scheme-confusion vectors (`file://`, `gopher://`, …) before the URL reaches a client. Response bodies from external HTTP calls are limited to 1 MiB via `io.LimitReader`. Scanner targets come from operator-controlled config, not external input. |
 
 ## OWASP AI Top 10
 
@@ -62,7 +62,9 @@ The OWASP AI Top 10 is **not applicable** to this project. NetworkInventoryAgent
 
 NetworkInventoryAgent is designed to run on a trusted internal network. Before deploying, consider the following:
 
-**Health endpoints are unauthenticated.** The `/health` and `/status` endpoints expose agent name, scan counts, host counts, and timestamps to anyone who can reach the listening address. The default bind address is `127.0.0.1` (loopback only). Do not change this to `0.0.0.0` unless the network segment is trusted or access is controlled at the firewall.
+**Health endpoints are unauthenticated.** The `/health` and `/status` endpoints expose agent name, scan counts, host counts, and timestamps to anyone who can reach the listening address. The default bind address is `127.0.0.1` (loopback only). Binding off-loopback requires `health.auth_token` (or `INVENTORY_AUTH_TOKEN`); the agent refuses to start without it.
+
+**The admin console is gated off-loopback.** The console at `admin.addr` (default `127.0.0.1:9090`) serves the full host/port inventory, `/export.json|csv`, the `/api/v1/*` query API, and the `POST /scan` trigger. On the loopback default it is unauthenticated for convenience; binding it off-loopback (e.g. `0.0.0.0` for Docker, or via `INVENTORY_ADMIN_ADDR`) requires `admin.auth_token` (or `INVENTORY_ADMIN_TOKEN`) and the agent refuses to start without it. Clients authenticate with `Authorization: Bearer <token>` or HTTP Basic auth using the token as the password.
 
 **Peer communication can use TLS.** Watchdog checks between Wintermute and Neuromancer default to plain HTTP for the loopback case. For off-loopback deployments, switch `watchdog.peer_addr` to `https://…`, set `watchdog.tls.ca_cert_path` to the CA that signs the peer's cert, and set `health.tls_cert_path` / `health.tls_key_path` on the peer. For full mutual auth, set `health.client_ca_path` on both sides and `watchdog.tls.client_cert_path` / `client_key_path` on the dialer side. Bearer tokens stack on top of TLS.
 

cmd/internal/runtime/runtime.go

@@ -148,6 +148,7 @@ func Run(opts Options) int {
 		cfg.Admin.Addr, opts.Name,
 		db.Hosts(), db.Ports(), db.Scans(),
 		tracker.Get, a.Trigger,
+		admin.ServerOptions{AuthToken: cfg.Admin.AuthToken},
 	)
 	if err != nil {
 		slog.Error("failed to create admin server", "err", err)

go.mod

@@ -11,6 +11,8 @@ require (
 	go.opentelemetry.io/otel v1.43.0
 	go.opentelemetry.io/otel/exporters/otlp/otlptrace/otlptracehttp v1.43.0
 	go.opentelemetry.io/otel/sdk v1.43.0
+	go.opentelemetry.io/otel/trace v1.43.0
+	golang.org/x/net v0.52.0
 	modernc.org/sqlite v1.50.0
 )
 
@@ -48,9 +50,7 @@ require (
 	go.opentelemetry.io/auto/sdk v1.2.1 // indirect
 	go.opentelemetry.io/otel/exporters/otlp/otlptrace v1.43.0 // indirect
 	go.opentelemetry.io/otel/metric v1.43.0 // indirect
-	go.opentelemetry.io/otel/trace v1.43.0 // indirect
 	go.opentelemetry.io/proto/otlp v1.10.0 // indirect
-	golang.org/x/net v0.52.0 // indirect
 	golang.org/x/sys v0.42.0 // indirect
 	golang.org/x/text v0.35.0 // indirect
 	google.golang.org/genproto/googleapis/api v0.0.0-20260401024825-9d38bb4040a9 // indirect

internal/admin/api.go

@@ -49,6 +49,15 @@ type hostDetailResponse struct {
 	Ports []*models.Port `json:"ports"`
 }
 
+// scansResponse is the JSON envelope for GET /api/v1/scans, mirroring
+// hostsResponse. Scans are newest-first (as the store returns them).
+type scansResponse struct {
+	Total  int            `json:"total"`
+	Limit  int            `json:"limit"`
+	Offset int            `json:"offset"`
+	Scans  []*models.Scan `json:"scans"`
+}
+
 // apiError sets standard headers and writes a JSON error body. The status
 // codes follow the standard REST conventions: 400 for bad input, 404 for
 // unknown resource, 500 for backend failure.
@@ -128,6 +137,54 @@ func (s *Server) handleAPIHosts(w http.ResponseWriter, r *http.Request) {
 	}
 }
 
+// handleAPIScans returns scan history as paginated JSON. Optional `subnet`
+// query param exact-matches Scan.Subnet. Mirrors handleAPIHosts.
+func (s *Server) handleAPIScans(w http.ResponseWriter, r *http.Request) {
+	q := r.URL.Query()
+	limit, offset, err := parsePagination(q)
+	if err != nil {
+		apiError(w, http.StatusBadRequest, err.Error())
+		return
+	}
+
+	all, err := s.scans.List(r.Context())
+	if err != nil {
+		slog.Error("api list scans", "err", err)
+		apiError(w, http.StatusInternalServerError, "list scans failed")
+		return
+	}
+
+	matched := all
+	if subnet := q.Get("subnet"); subnet != "" {
+		matched = make([]*models.Scan, 0, len(all))
+		for _, sc := range all {
+			if sc.Subnet == subnet {
+				matched = append(matched, sc)
+			}
+		}
+	}
+
+	total := len(matched)
+	start := offset
+	if start > total {
+		start = total
+	}
+	end := start + limit
+	if end > total {
+		end = total
+	}
+
+	w.Header().Set("Content-Type", "application/json")
+	if err := json.NewEncoder(w).Encode(scansResponse{
+		Total:  total,
+		Limit:  limit,
+		Offset: offset,
+		Scans:  matched[start:end],
+	}); err != nil {
+		slog.Error("api encode scans", "err", err)
+	}
+}
+
 func (s *Server) handleAPIHostDetail(w http.ResponseWriter, r *http.Request) {
 	ip := r.PathValue("ip")
 	host, err := s.hosts.GetByIP(r.Context(), ip)

internal/admin/api_test.go

@@ -249,3 +249,64 @@ func TestAPIHostDetail_NotFound(t *testing.T) {
 	assert.Equal(t, http.StatusNotFound, resp.StatusCode)
 	_ = resp.Body.Close()
 }
+
+func fixtureScans() []*models.Scan {
+	now := time.Now()
+	return []*models.Scan{
+		{ID: 1, Subnet: "10.0.0.0/24", HostsFound: 3, StartedAt: now},
+		{ID: 2, Subnet: "10.0.1.0/24", HostsFound: 1, StartedAt: now},
+		{ID: 3, Subnet: "10.0.0.0/24", HostsFound: 4, StartedAt: now},
+	}
+}
+
+type scansBody struct {
+	Total  int            `json:"total"`
+	Limit  int            `json:"limit"`
+	Offset int            `json:"offset"`
+	Scans  []*models.Scan `json:"scans"`
+}
+
+func TestAPIScans_ListsAllPaginated(t *testing.T) {
+	srv := newTestServer(t, &mockHostStore{}, &mockPortStore{}, &mockScanStore{scans: fixtureScans()})
+	resp := get(t, srv, "/api/v1/scans")
+	defer func() { _ = resp.Body.Close() }()
+	require.Equal(t, http.StatusOK, resp.StatusCode)
+
+	var body scansBody
+	decodeJSON(t, resp, &body)
+	assert.Equal(t, 3, body.Total)
+	assert.Equal(t, 100, body.Limit)
+	assert.Len(t, body.Scans, 3)
+}
+
+func TestAPIScans_Pagination(t *testing.T) {
+	srv := newTestServer(t, &mockHostStore{}, &mockPortStore{}, &mockScanStore{scans: fixtureScans()})
+	resp := get(t, srv, "/api/v1/scans?limit=2&offset=0")
+	defer func() { _ = resp.Body.Close() }()
+
+	var body scansBody
+	decodeJSON(t, resp, &body)
+	assert.Equal(t, 3, body.Total, "total reflects the full set, not the page")
+	assert.Equal(t, 2, body.Limit)
+	assert.Len(t, body.Scans, 2)
+}
+
+func TestAPIScans_SubnetFilter(t *testing.T) {
+	srv := newTestServer(t, &mockHostStore{}, &mockPortStore{}, &mockScanStore{scans: fixtureScans()})
+	resp := get(t, srv, "/api/v1/scans?subnet=10.0.0.0/24")
+	defer func() { _ = resp.Body.Close() }()
+
+	var body scansBody
+	decodeJSON(t, resp, &body)
+	assert.Equal(t, 2, body.Total, "only the two 10.0.0.0/24 scans match")
+	for _, sc := range body.Scans {
+		assert.Equal(t, "10.0.0.0/24", sc.Subnet)
+	}
+}
+
+func TestAPIScans_InvalidLimit(t *testing.T) {
+	srv := newTestServer(t, &mockHostStore{}, &mockPortStore{}, &mockScanStore{scans: fixtureScans()})
+	resp := get(t, srv, "/api/v1/scans?limit=0")
+	defer func() { _ = resp.Body.Close() }()
+	assert.Equal(t, http.StatusBadRequest, resp.StatusCode)
+}