NetInv v1.0.0

NetInv is a network asset monitoring platform — a modern successor to Cacti + Weathermap. SNMP collection, ICMP availability, inventory with change history, alerting, a weathermap editor, and flow collection, on Go + VictoriaMetrics + PostgreSQL.

It runs a real network. The reference pilot has been live continuously through development: four UniFi gateways in an SD-WAN mesh, a Ruckus Unleashed controller with a mesh AP, a published weathermap over six WireGuard tunnels with live traffic, and three gateways exporting NetFlow.

Read this before deploying

v1.0.0 ships with four release gates open. They were written at the start of the pilot and are not met. They are listed here rather than dropped, and the reasoning is in ADR-023.

1. Six of eight connectors have never been run against the hardware they target. ubiquiti, generic and ruckus are validated against real units and documented in doc 10 with what each device actually exposes. cisco-ios, juniper-junos, huawei-vrp, zte-zxr, fortinet-fortios and paloalto-panos are written from published vendor MIBs and have never received a packet from the platform they describe. If you run one of those, expect to find faults — and please report them.
2. No soak or chaos testing. There is no staging Kubernetes cluster behind this release. Behaviour under sustained load or component failure is untested rather than proven.
3. No TLS security review. The doc 20 §12 checklist has not been run against a TLS deployment.
4. No backup/restore drill. Restoring this system from backup has never been rehearsed. Do your own before you rely on it.

What is validated: a live single-site pilot running continuously, an automated suite covering both Go modules and the frontend, and every gate in CI — build, vet, golangci-lint, race detector, connector plugin contract, licence check, shellcheck, Helm lint, and Playwright end-to-end.

What it does

Collection. SNMP v2c/v3 for interface traffic, device health, inventory and topology, plus ICMP availability. Eight vendor connectors behind a compile-time plugin contract — adding a platform is one package and one registry line, with no core changes.

Weathermap — the flagship. A drag-and-drop editor with LLDP-based link suggestions, draft/publish revisions, and live utilization on the Cacti colour scale. Links show one band coloured for the heavier direction with the reverse rate still readable, name their destination so a crossing map stays legible, can be deleted without deleting their nodes, and parallel links between the same pair are drawn apart.

Flow collection (ADR-020/022). A dedicated netinv-flow service decodes NetFlow v5, v9 and IPFIX on UDP 2055 and 4739 — including v9/IPFIX templates, options-template sampling, IPv6, variable-length and enterprise fields. It aggregates at ingest into top-N talkers, conversations and applications per interface, so cardinality is bounded by construction and no per-flow record is stored. Surfaced on a device Flow tab and a dashboard panel, with an in-product setup guide carrying per-vendor snippets.

Dashboards and graphs. A per-user customisable dashboard including embedded weathermaps and arbitrary metric panels. One shared time range across every graph using Cacti's nineteen timespans. Retention configurable, defaulting to two years, with a Capacity page that measures whether the disk will actually hold it.

Alerting. Threshold, state and inventory rules evaluated against MetricsQL, with acknowledge/silence, and delivery by email, webhook or Slack.

Operations. RBAC with audit trail, per-site pollers, discovery, an SNMP OID browser for working out why a metric is empty, and a fleet-wide poll cadence settable from the UI.

Deploying

• One host: deploy/compose-app/quickstart.sh (doc 32).
• Proxmox LXC: deploy/proxmox/netinv-lxc.sh create|verify|destroy (doc 33), verified end to end on Proxmox VE 9.2.10 — including the fix for unprivileged-container ICMP that Docker's ping_group_range clamping breaks.
• Kubernetes: Helm charts for the app and for remote-site pollers.

Images for all seven services are published per release.

A note on the documentation

Thirty-four design documents, and they are kept honest rather than aspirational: where the code diverges, the doc changes in the same commit. Several record things this project got wrong and later corrected — most recently a flat claim that UniFi gateways cannot export flow, which was inferred rather than tested, and turned out to be false. The reasoning behind every architectural decision is in DECISIONS.md as numbered ADRs.

Contributing

The most useful contribution is a hardware validation report for any of the six unvalidated connectors — see CONTRIBUTING.md and the ask in Discussions. SECURITY.md covers vulnerability reporting and the deployment posture worth knowing before exposing this anywhere.

Apache-2.0.

v1.0.0-rc.2 — flow collection, enterprise firewalls, shared time range

The headline is flow collection: NetInv can now answer what is on a link, not only how much. Plus two firewall platforms, a shared time range across every graph, a customisable dashboard, and configurable retention.

87 commits since v1.0.0-rc.1. Still a release candidate — the v1.0 gates in CLAUDE.md are unchanged and listed at the bottom.

Flow collection (ADR-020, ADR-022 · doc 34)

A seventh service, netinv-flow, receives NetFlow v5, v9 and IPFIX on UDP 2055 and 4739 and reads the version from the datagram, so any format works on either port. sFlow is not decoded.

It aggregates at ingest rather than storing flows. A flow record is a wide event keyed by source, destination, port and protocol; stored as time series its cardinality is the number of host pairs on the network. So each interval is reduced to top-N talkers, conversations and applications per interface, and only those are written — netinv_flow_bytes and netinv_flow_packets. Raw per-flow retention stays deferred and needs its own datastore.

The cost is stated rather than hidden: a conversation outside the top N for an interval was never written. It did not go unqueried; it does not exist.

Where it surfaces:

• Flow tab on device detail — talkers / conversations / applications, an interface filter, and a chart of the heaviest few.
• Top flow dashboard panel, fleet-wide or per exporter, repeatable.
• In-product setup guide with a format selector and per-vendor snippets (Cisco IOS, Junos, Huawei VRP, FortiGate, PAN-OS, RouterOS, VyOS, softflowd, UniFi), the collector's address filled in, and the traps that produce silence stated before the commands.

v9 and IPFIX make the collector stateful: a data record is meaningless without the template describing it. A restart therefore loses every template and flow is missing until exporters resend — counted and reported as awaiting_template, deliberately not as "undecodable", so nobody hunts a fault that is not there. The template cache is capped and expiring, because it is attacker-influenced state on an unauthenticated UDP port.

Validated on real hardware

A pilot UniFi gateway exports v5 into the collector and it decodes cleanly — not one packet refused or undecodable. Sampling was verified against the wire, not assumed: at Hash 1-in-512 the header read mode=2 interval=512 and the decoder scaled by 512; with sampling off it reads mode=0 interval=0.

Two findings worth repeating, because both correct things this project previously believed:

• UniFi gateways do export flow. Earlier documentation said they could not. That claim was never tested — it was inferred from a missing sFlow MIB on the switches and generalised to the gateways. The feature is in UniFi Network under Traffic Logging → NetFlow (IPFIX), offering v5/v9/IPFIX. It is now documented with the click-path and its three traps: export is selected per network rather than per interface, sampling ships on, and the "Timeout Rate" is the active timeout.
• Flow totals are a fraction of the SNMP counters, by design. Measured at 3.3% and 10.5% on two interfaces with sampling off. The dominant cause was confirmed rather than reasoned: every interface sat at the ten-bucket top-N cap in every one-minute interval. This is why ADR-020 makes SNMP the authority for volume and flow the answer to composition — the rule now has a measurement behind it.

Flow is attributed by the datagram's source address, and routers routinely export from an uplink or loopback. Devices therefore carry a flow_exporters list, and the Flow tab's empty state names the unattributed addresses it can see and offers to claim one.

Enterprise firewalls (ADR-021 · doc 10)

fortinet-fortios and paloalto-panos connectors — inventory, interfaces and health like any connector, plus device-level session gauges (netinv_firewall_session_count, _session_max, _session_setup_rate). Anything policy-, rule-, NAT- or VPN-scoped remains deferred: the line is drawn where the data stops being a number about the appliance and becomes a table about its configuration.

FortiOS publishes no session ceiling, so no utilization is invented from one. PAN-OS does, so utilization is a query over count and max rather than a stored number that could drift. PAN-OS is also the only platform here whose CPU and memory come from HOST-RESOURCES-MIB.

Neither is validated against hardware, and neither has a fixture — every OID comes from the published vendor MIBs. Six of eight connectors are now unvalidated, which doc 10 states per connector.

Graphs, dashboard and retention

• One shared time range across every graph, using Cacti's nineteen timespans, persisted per browser. Presets beyond retention are shown disabled rather than hidden.
• Customisable dashboard: add, remove, reorder and widen panels, including an embedded weathermap and a chart of any published metric. Saved per user.
• Metrics retention is configurable and defaults to two years. The API's range ceiling reads the same setting, so a query limit cannot drift from what the store actually keeps.
• Platform → Capacity answers "will the disk hold this retention", measured from the running system rather than derived from configuration.
• Fleet-wide poll interval settable from the UI (1/5/10/15 min), with rate() windows sized from it.
• Weathermap: links can be deleted without deleting their nodes; a link shows one flow in the heavier direction with the reverse rate still readable; labels name their destination.

Deployment

• Proxmox LXC: deploy/proxmox/netinv-lxc.sh create|verify|destroy, and doc 33. Verified end to end on Proxmox VE 9.2.10, including a fix for unprivileged-container ICMP that Docker's ping_group_range clamping breaks.
• The Helm chart and image build now cover the flow service. It declares listen ports rendered as a Service — the first component that must be reachable from the device network — and is pinned to a single replica, because UDP has no replay and template state is per exporter.

Notable fixes

• A device can be re-addressed instead of deleted and recreated; PATCH /devices/{id} silently ignored mgmt_ip and snmp_port at both layers.
• The Flow tab hid real data between a bursty exporter's sends — a presence check on the instant lookback while the table summed over the selected range.
• Connectors reported the health capability twice.
• A recognised service port now beats the lower one when naming an application, so WireGuard on 51820 is not filed under a client's ephemeral port.

Still open before v1.0

Unchanged from rc.1 and not fudged: real-hardware validation of the six untested vendor connectors, a 72-hour soak plus chaos-lite on staging Kubernetes, the doc 20 §12 security checklist against TLS, and a backup/restore drill on pilot data. These need real infrastructure and time.

sFlow remains the one flow format NetInv does not read; it samples packet headers rather than exporting flow records and shares none of the template machinery.

Hardware validation reports remain the single most useful contribution — see CONTRIBUTING.md and the ask in Discussions.

v1.0.0-rc.1 — release candidate

NetInv v1.0.0-rc.1 — the release candidate for v1.0. Feature-complete for v1; the remaining gates before v1.0.0 final require real hardware and a staging cluster (see below).

What's in it

Centralized multi-vendor network monitoring, built across 19 sprints — each with a verified live exit demo (see the commit log). All four build milestones achieved:

• M1 — Collection pipeline: scheduler → RabbitMQ → site pollers → SNMP → ingester → VictoriaMetrics. IF-MIB traffic/errors/status (64-bit counters), ICMP availability (RTT/jitter/loss), inventory sync with change detection + LLDP topology.
• M2 — Alerting: MetricsQL rule evaluation, fingerprinted lifecycle with flap suppression and silences, Email/Webhook/Slack delivery with retries + per-channel delivery log.
• M3 — Product: the NOC dashboard (status strip, alerts with ack, Top-N, health heatmap, capacity watchlist, uPlot charts), inventory with search/filter/CSV+XLSX export, device detail where an alert deep-links to the explaining graph.
• M4 — Weathermap (flagship): React Flow editor with interface binding, autosave, undo, publish-with-validation; live viewer with the classic utilization color scale.

Plus: 5-vendor connector framework (Cisco/Juniper/Huawei health via real MIB maps; ZTE/Ubiquiti best-effort pending hardware), enrollment-based poller fleet with heartbeats + disk-buffer offline resilience, RBAC + full audit trail, envelope-encrypted credential vault, and the admin UI (users, audit, platform, settings).

Verified this cycle

• Chaos drills, run live: scheduler leader failover in 18.8s; broker-outage reconnect in 5s (forced the amqpx redial + supervised consumer loops now in place).
• Backup/restore: destructive drill passed — 73 devices, 292 schedules, 123 audit rows, and VM series all recovered (scripts/backup.sh / restore.sh).
• Security: govulncheck + gitleaks clean; no-secret-leak invariant test green; Vite bumped to clear the high npm advisory.
• Tests: unit + PG integration + connector tests + Playwright E2E smoke in CI (which caught and fixed a real session-restore/refresh bug).
• Mini-soak: clean — no goroutine or heap growth (docs/sprint-notes/soak-rc1.md).
• Multi-arch images for all 6 services + frontend publish to GHCR on tag.

Not yet done (needs your infrastructure — Sprint 20)

• Real-hardware validation of each vendor connector (ZTE/Huawei are the risk items, R-07).
• 72-hour soak + full chaos matrix on a staging Kubernetes cluster.
• Full doc-20 §12 security checklist sign-off on a TLS deployment.
• The pilot across the 4–5 production sites — follow docs/31-pilot-runbook.md.

Install

helm install the core chart (deploy/helm/netinv) against your data tier, plus deploy/helm/netinv-poller per remote site with an enrollment token. Full steps in the pilot runbook.

🤖 Generated with Claude Code