Repository navigation
Releases
Releases · agit8or1/m0nitor
v1.63
Fixed
- "Detector errors: new_dest_asn … TIMEOUT_EXCEEDED". The new-network and new-country rules compared the last hour
against 14 days of baseline with aNOT INsubquery that ClickHouse expanded twice. It took about 14 seconds
on a quiet server and went past the 45-second limit under load. Both rules now read the baseline and the window
in a single grouped pass, which takes about 3 seconds and gives the same findings (checked on real data). - Acknowledged findings came back. Acknowledging only held for the 6-hour repeat window, so a finding that
kept happening was raised again as a new alert. Now an acknowledged finding absorbs repeats (count and
last-seen time still update) until it has been quiet fordetector.acked_quiet_days, which defaults to 7. It is
raised again sooner if it gets more severe. - "This is normal" on findings that change subject every time. Rules that raise one finding per destination
network, country, scanner, beaconing target or port start a new finding for each new subject, so choosing "only
this behaviour" never stopped the next one. When the same device has had at least two other such findings in the
last 7 days, the card now says so and pre-selects "Any … finding for this device". With "close matching open
findings now", it also closes the ones already in the list.
Full changelog: v1.62...v1.63
v1.62
Fixed
- WAN bandwidth panel while loading. On the Overview, the "Loading…" overlay made the bandwidth chart taller and
pushed the WAN utilization meters below it out of the panel. The chart now keeps its normal size while loading,
so the meters stay visible.
Full changes: v1.61...v1.62
v1.61
Added
- "Loading…" indicator on panels still waiting for data. A spinner with "Loading…" now covers each map, list,
chart and table until its data arrives: Overview, Countries, a country's page, a server's page, Internal devices,
Connections and a finding's traffic map. Before, the map showed an empty world with no sign it was still loading.
It appears on first load and after changing site, time range or filter; the periodic refresh updates in place
without it.
Fixed
- Logged firewall blocks over 7 days. The query returned one row per source and destination pair, and over 7 days
that passed the 200,000-row result limit, so the 7-day Countries map and block statistics failed. Rows are now
grouped per external address inside ClickHouse, using the same protected-network and special-range rules as before:
about 100,000 rows for 7 days instead of more than 377,000. Per-country counts, addresses and rules are unchanged
(checked against the old query on 24 hours of real data).
Full changes: v1.60...v1.61
v1.60
Fixed
- Overview loads much faster. Under normal load the overview API took 2–17 s, and up to 40 s while ClickHouse
was busy. Three causes, all fixed:- Map markers for open findings looked up the country of every named address by scanning 7 days of flows,
formatting each row's address as text. The cache was dropped whenever any open finding changed, which happens on
nearly every detector run. Each address's country is now remembered (6 h; addresses not seen are re-checked after
10 min), and only new addresses are queried, on the raw address column (about 0.4 s instead of 4–30 s). - The collection state (newest record, flows in the last 5 minutes, oldest record) scanned the whole flow table
every 5 s. It now reads only the last day, and the oldest record is found from the table index (cached 10 min). - Background work (detector, cache warm-up for the 24-hour and 7-day views, health checks, resolvers) competed with
pages for ClickHouse query threads. When the 7-day warm-up ran, a small overview query got a single thread.
Queries made for a page or API request now run at a higher ClickHouse priority than background queries, so
ClickHouse pauses background scans while a page is waiting. - The DNS name resolver had fallen days behind. Each run re-read the whole backlog, timed out after 45 s, and so
never moved forward. It now never falls more than 15 minutes behind and reads at most 5 minutes of the query log
per run.
- Map markers for open findings looked up the country of every named address by scanning 7 days of flows,
Full changes: v1.59...v1.60
v1.59
Added
- Onboarding test per firewall (Settings → Status → Onboarding test; summary for every firewall you can see on
Coverage). It checks whether WAN flows, LAN return flows, firewall logs and usable NAT evidence are arriving and fresh.
For each source that is missing, stale or not possible, it says which question you cannot answer yet and which
exporter or interface setting to check, written for the firewall's vendor (OPNsense, pfSense, FortiGate, generic). A
WAN interface name that does not match the records is named, with the interfaces actually seen. The test only reads
received data: it changes no firewall setting and sends no traffic. - Collection quality on the same page: export loss, duplicate exports, clock skew, asymmetric return paths and
WAN-only capture, each with its effect on attribution. The health loop now records collector receive drops and
measures identical flow records per exporter (new health check "Duplicate flow exports"). - Export check:
python -m netmon.flowdiag ... --recordstores sequence gaps, duplicated export packets and
exporter clock drift for the matching firewall. flowdiag now counts repeated sequence numbers as duplicates instead
of exporter restarts. - Guided NAT validation (optional, analyst or admin): two hosts you operate, one destination you are authorised to
use. The page shows the upload commands to run on your hosts (the monitor sends nothing) and evaluates what was
attributed to which host. Simultaneous uploads must stay ambiguous and an unanswered attempt must stay
unanswered. The result per firewall is shown on Coverage. Actions are audited.
Fixed
- NAT attribution could overstate confidence. On a WAN-only (capture) exporter the reply to the shared NAT address
counted as "one host", and the NAT address itself was reported at high. A reply that returned through a different
firewall confirmed another firewall's egress. Unanswered SYN bytes were added to a host whose answered connection
shared the remote address, port and minute. Now only addresses inside the firewall's internal networks count as
return evidence, pairing stays on one exporter, and SYN-only bytes are always unanswered. WAN-only egress has its
own tier, wan_only. - Recorded export loss caps single-host results at low, and recorded duplicate exports cap them at medium, in the
affected window; the moved bytes and reasons are shown. When export-path loss has never been measured for a firewall,
every attribution says so instead of assuming none. - A user limited to some firewalls no longer sees another firewall's shared NAT addresses or last NAT validation.
Tests
- NAT attribution scenarios against a real (throwaway) ClickHouse: WAN-only capture, several hosts behind one shared
NAT, asymmetric routing, unanswered connections, clock skew / delayed export, duplicate exports, export loss and
per-firewall restrictions. None of these may raise a host's tier compared with the clean run of the same traffic.
CI runs them against a pinned ClickHouse image. - Onboarding page, vendor guidance, roles, CSRF, restricted users (404 for other firewalls), audit entries, ClickHouse
outage, and a guard that no firewall adapter is called.
Benchmarks
bench.py pipeline(end to end: UDP → Akvorado inlet → Kafka → outlet → ClickHouse) now records offered and
ingested flows/s per step, collector drops, decoder errors, queue lag, storage growth, retention (TTL per table),
query latency and image versions. A low-rate marker stream keeps the collector flushing and is excluded from counts.
bench/stack/is a ready isolated stack for it. First measured results: docs/performance-benchmarks.md.- docs/field-validation.md: which vendor paths are verified on hardware (OPNsense) and which are fixture-only
(pfSense, FortiGate and every other vendor), with a reproducible procedure. pfSense and FortiGate stay unverified.
Full changelog: v1.58...v1.59
v1.58
Changed
- Block requests: the page now says plainly that approval is a confirmation step: an admin can approve a request
they made themselves, so it is not two-person approval. - Users & 2FA: an optional 2FA policy (the default) is flagged with a recommendation to require it.
Docs
- Blocking: "one alias" applies to manual blocks only. Auto-response uses a second alias (
netmon_auto) and, with
"Create netmon's own firewall objects" on (default), creates its aliases and floating block rules itself. - Install: Caddy's
tls internalis a private CA, not a publicly trusted certificate, and how to trust or replace
it. New step: turn on Require 2FA after enrolling (2FA is optional by default).
Tests
- Block approval (pending until an admin approves, typed confirmation, CSRF, roles, rejected requests), the
Coverage page, and the finding page's evidence, baseline, alternatives and next steps.
Full changelog: v1.57...v1.58
v1.39
Added
- Device type "management controller (iDRAC / iLO / IPMI)" in a new Management category, for out-of-band
management cards. Auto-discovery sets it when the firewall's hostname for the address names one (idrac-…,
ilo-…,ipmi,bmc,imm,xcc,irmc). AI analysis treats internet connections to a management controller
as a serious exposure, while NTP, DNS, alert e-mail and vendor update traffic from it is normal.
Full changelog: v1.38...v1.39
v1.38 (includes v1.37)
v1.38 — 2026-11-03
Added
- Settings → Firewall permissions: every OPNsense API privilege netmon uses, what needs it (blocking, blocking
setup, name sync, discovery, dropping states) and the endpoints behind it, plus Check this account now, which
makes one read-only call per privilege and shows ✓ granted / ✗ missing for each firewall. pfSense REST API
privileges are listed for pfSense sites. Privilege names are taken from OPNsense's own ACL definitions. - New guide
docs/opnsense-permissions.mdwith the same list, how to create the API user and how to remove access.
Fixed
- The setup tips and the firewall connection form said a user with only "Diagnostics: PF Table IP addresses" was
enough; that covers blocking only. They now point to the full list. The firewall setup guide named privileges that
do not match OPNsense's; corrected.
v1.37 — 2026-11-03
Fixed
- Auto-discovery read no ARP table on current OPNsense. It called the legacy
getArpendpoint, which current
releases refuse even with the "Diagnostics: ARP Table" privilege. It now usessearch_arp(all rows) and falls
back togetArpon older firewalls.
Full changelog: v1.36...v1.38
v1.36 (includes v1.35)
v1.36 — 2026-11-03
Changed
- "Servers" is now "Internal devices" (navigation, page titles, Inventory, search results, filters, map tooltips,
wallboard KPI). Addresses (/netmon/servers/...) are unchanged, so bookmarks keep working.
Added
- Device categories. Every type belongs to a category: Servers, Virtual, Network, Endpoints, IoT & peripherals,
Other, and Unclassified (no type yet). The Internal devices page has category filter chips with counts; Settings →
Inventory groups the devices by category with jump links. AI analysis receives the category too. - MAC vendor lookup from the IEEE registry (MA-L, MA-M and MA-S prefixes, longest match wins; the
ieee-data
package, mounted read-only). Randomized (private) MACs are recognized. Vendors that make essentially one kind of
device (camera, printer, VoIP phone, NAS, network and firewall makers) set that type automatically; general
vendors (Dell, HP, Cisco, Apple, Intel, …) set none. Vendors are also filled in for MACs recorded earlier, and
passed to AI analysis.
v1.35 — 2026-11-03
Added
- Inventory fills itself. Every 15 minutes, auto-discovery adds internal addresses it has not seen before: any LAN
address that received internet traffic in the last 24 hours, and every entry in each OPNsense firewall's ARP table
(MAC address, vendor and hostname; needs the API privilege "Diagnostics: ARP Table"). A virtualization MAC prefix
(Proxmox, VMware, Hyper-V, Xen, KVM, VirtualBox, …) sets the type to virtual machine; any other vendor sets no type,
since one vendor makes servers, switches and management cards alike. New entries are tagged auto-discovered;
on existing entries only empty fields (type, MAC, vendor) are filled, so nothing a person set is overwritten.
Editing an entry makes it yours (MAC and vendor are kept). - Settings → Inventory has an Auto-discovery card: on/off, Run now, the last run, and per-firewall ARP status
(including the missing-privilege hint). MAC and vendor are shown under each name. - AI analysis is told when a device type came from discovery and when an entry was not reviewed by an admin.
Full changelog: v1.34...v1.36
v1.34
Added
- Device type for every internal address. Pick what an address is from a fixed list: physical server, virtual
machine, container, hypervisor host, workstation / laptop, mobile device, printer, IoT / camera, VoIP phone, network
device (switch / AP), firewall / router, storage / NAS, other. (Role stays what it does.) It is in the ✎ editor, the
server page, Settings → Inventory (new Type column), and the bulk card, now Assign type and role, which can
set only the type and keep each address's role. - AI analysis receives the device type and weighs it: a workstation, printer, phone or IoT device accepting
connections from the internet, or a printer/IoT device reaching many remotes, counts as stronger evidence than the
same pattern on a server.
Full changelog: v1.33...v1.34