Router integration policy: OpenWrt and SNMP in core, external agents for everything else

[gnacho]

NetPulse started as an OpenWrt-first tool: one static server binary, a small native agent on the router, and SNMP for switches and other generic devices. That boundary is now explicit.

What lives in NetPulse

• The native OpenWrt agent.
• SNMP polling, including vendor MIBs exposed through standard SNMP.
• The signed ingest contract at POST /api/ingest/agent, plus history, topology, alerts and the UI.
• Neutral payload fields that more than one source can use.

What runs beside NetPulse

• Vendor APIs and their credentials: MikroTik RouterOS, Bbox, ASUSWRT-specific collectors and similar.
• Per-vendor parsing, sessions, quirks and firmware differences.
• Hardware-specific validation.

Those belong in an external Go agent in the shared netpulse-integrations repository. Each vendor gets a directory and a maintainer chosen through CODEOWNERS. The agent can run on a PC, a NAS, a small server or a container, normally as a systemd service. Docker is not required.

The external agent sends signed payloads to NetPulse. It uses the same router identity, freshness rules, history and UI. NetPulse owns the contract and the generic data model. The maintainer of each directory owns the vendor client and its releases.

Why this boundary

A vendor adapter in the server needs hardware, firmware and time that the core project may not have. When nobody validates it, it stays in the tree as plausible-looking code that can break silently. Keeping vendor clients outside lets each integration ship and be tested on its own schedule, independently from other integrations and from NetPulse releases. It also means an OpenWrt router never carries code for RouterOS, Bbox or another platform.

A central repository makes integrations easy to find and gives them shared CI. Directory ownership keeps responsibility with the person who can test the hardware.

How to propose a new integration

1. Open an issue with the router model, firmware, API documentation, available data and a read-only first scope.
2. Propose a directory in netpulse-integrations and follow docs/external-integrations.md.
3. Include real fixtures and tests, plus evidence from real hardware.
4. Commit to maintaining that directory and its releases.
5. Agree any neutral contract change with the NetPulse maintainers before sending the PR.

Current transitions

• Bbox: an external bbox-pusher was proposed in Feat/bbox integration #1527. It would live in bbox/, with juflo as the directory maintainer if he accepts.
• MikroTik RouterOS: the current server adapter is legacy and will move to routeros/ in the integrations repository (routeros: move polling from the server to the external pusher #1574). Existing installations will migrate to the external agent. There will be no hidden RouterOS compatibility path in the server.
• ASUSWRT: its collectors need sysfs and wl inside the router, so a PC-side pusher is not enough. It would need a separate agent in the integrations repository, maintained by someone with that hardware.

The technical contract, payload semantics, authentication, testing and deployment details are in the guide: https://github.com/gnacho/netpulse/blob/main/docs/external-integrations.md

If you have hardware for one of these families and want to maintain its directory, comment with the model, firmware and the data you can collect read-only.

[gnacho]

This policy now has a matching implementation, so the boundary is not just a document any more.

NetPulse 2.49.x ships an Integrations section that deploys a vendor pusher from the app:

• It resolves the latest release of the vendor in netpulse-integrations, verifies the SHA-256 against the published checksums and only then extracts and runs it.
• It writes the pusher configuration with mode 0600 on the host that runs it. Router credentials never go to the database.
• It registers the router as an external source with its own token and interval, so history, freshness and the UI work exactly like any other router.
• There are two deployment modes: local, where the pusher runs as a managed process on the NetPulse host (RouterOS today), and ssh, where the binary is installed on the router itself (Asuswrt, arm64).
• Integrations can be edited in place since 2.49.2, including the interval and the credentials, without deleting and recreating them.

What is actually proven so far: RouterOS, end to end, against a real CHR on RouterOS 7.25. Read-only API user, 30 s interval, then edited to 45 s from the UI and the pushes moved to 45 s without touching anything else. Both bugs found during that test are fixed and released.

What is not proven: the ssh mode has never run against a real Asus router. Being honest about it, the install script currently assumes an OpenWrt-shaped target: binary in /usr/sbin, unit in /etc/init.d and a procd init with respawn. Asuswrt-Merlin has no procd and its persistent writable area is /jffs, so I expect the install path to need a Merlin-specific shape, probably a binary plus an env file under /jffs started from services-start. That is exactly the kind of detail that only someone with the hardware can settle.

@Bann-DEV, you have the dhd nodes (RT-AX92U) and you already know this platform better than I do from the wl tier and the ghost ports work. Would you be up for trying it on 2.49.2 or later and telling me what breaks?

Concretely, what I would like to learn from a real device:

1. Does the server reach dropbear with the key it uses for the OpenWrt agents, or does Asuswrt need a different user or a password path?
2. Where should the pusher binary and its configuration live so they survive a reboot on Merlin. /jffs plus a services-start hook seems right, but I would rather ask than guess.
3. Whether the pusher itself works on your nodes with iw, iwinfo and the wl fallback, or if the environment you get under services-start is too early for the radios to answer.
4. Anything the payload is missing for dhd mesh nodes, since that was the original gap in dhd Asuswrt mesh nodes (RT-AX92U): nl80211 exposes no stations or channel, so clients from #1513's hint show band '—' #1529.

@samex, the RouterOS path is the one I could validate end to end, so it is the closest to what you started in #1054. The difference is that RouterOS support now lives in the external pusher and the core only keeps the signed contract, which is why your fork does not need to be rebased against the server any more. If you still have a RouterOS box, I would be interested in what the pusher gets wrong on your model. The CHR I used is x86_64 with a read-only API user on port 8728, so real hardware may differ on the binary API handshake, on which resources answer, and on the units of the throughput counters. Families I have not touched: CCR, CRS and the Chateau line.

@pl-komuch, your MikroTik hardware is SwOS on the CSS switches, which is SNMP and already covered, so this particular path does not apply to you. If you also have a RouterOS device lying around, the same invitation stands. Otherwise ignore this part and keep the SNMP route working, it is doing the job.

No obligation and no rush. If you prefer, I can open a dedicated issue with a checklist and we work it there, keeping this thread for the policy. Either way, if the answer is that the ssh mode needs a different contract for non-OpenWrt targets, or that RouterOS needs a compatibility matrix before it can be called supported, those are useful answers too and they shape the next release.

[gnacho]

Tracked as #1681 with the layout part already designed: binary and configuration under /jffs/netpulse, a control script with a pidfile, and a managed block in /jffs/scripts/services-start instead of a procd service, with the block removed cleanly on uninstall.

Two things I would rather not guess and that change the design:

• whether dropbear accepts the server key at all on your routers, or whether the install has to fall back to password SSH,
• whether iw, iwinfo and wl answer when the pusher starts from services-start, or the first sample has to retry until the radios are up.

If you can answer those two, the implementation is a small change to the script generator plus its tests, and I can have it in a release right away.

[gnacho]

The announcement is out: NetPulse 2.50.0 ships the complete integrations system, and vendor compatibility is now a release headline rather than a plan.

What 2.50.0 announces as supported:

• MikroTik RouterOS: read-only pusher over the binary API, deployed from the app. This is the path validated end to end against a real CHR on RouterOS 7.25, including edits from the UI (30 s to 45 s without recreating anything).
• Asuswrt and Asuswrt-Merlin: the pusher installs on the router itself over SSH, the Merlin way (binary and config under /jffs/netpulse, boot hook in /jffs/scripts/services-start), and stock dropbear that only accepts passwords works too, with the password kept in a 0600 companion file outside the database. Thanks to @Bann-DEV, whose answers on dropbear and radio tools shaped the install and the boot expectations. Real-hardware validation on your RT-AX92U nodes is still the open item, the invitation from the previous comment stands.
• UniFi: a new read-only vendor for self-hosted UniFi Network controllers (UniFi OS consoles and classic standalone, one site): gateways, switches and APs with ports and uplinks, clients with band and signal, MAC-to-IP, gateway uptime. Explicitly not emitted: DHCP leases, per-interface counters, VLANs, roaming, CPU and temperature, because the read-only API has no verified match for them.

How it stays clean, for anyone joining the thread: every vendor lives in the separate netpulse-integrations repository and ships as a standalone binary against the signed ingest contract. The server only downloads and runs the code for a vendor you configure, so owning three routers does not mean carrying code for the other thirty platforms. Compatibility grows, the server binary carries no vendor code, and the shared surface everyone has to maintain stays small.

Bbox is the next one waiting: the integration is merged in netpulse-integrations and the core already lists it, it just needs its first bbox/v0.1.0 tag to appear as installable.

[MassiPi]

Hello,
i started working for an integration for my stupid netgear 308e managed switches.
I got a working GO to read data from them, so i thought i was well ahead in the work.
That was not the case, creating an integration is not that easy :)
However, as soon as i realised that my netgear does not show the mac addresses of devices connected to each port, i assumed it was just time lost to try to port them to netplulse. Can you confirm? i could get this sort of info, but nothing more, and without mac addresses they are useless.

================================================================================================================
                                      NETGEAR GS308E - PORT MONITOR
================================================================================================================
PORT  STATUS   LINK             RX BYTES       TX BYTES        CRC    PVID VLAN ID
------------------------------------------------------------------------------------------------------------------------
1     Up       1000M         11958367109     1063161487          0       1 1(U),999(T)
2     Down     No Speed                0              0          0       1 1(U)
3     Down     No Speed                0              0          0       1 1(U)
4     Down     No Speed                0              0          0       1 1(U)
5     Down     No Speed                0              0          0     999 999(U)
6     Down     No Speed         49440382     2623050992          0     999 999(U)
7     Up       100M            884834837     4370268209          0     999 999(U)
8     Down     No Speed         86871040      486001109          0     999 999(U)
------------------------------------------------------------------------------------------------------------------------
TOTALE                        12979513368     8542481797          0

However, i have a second part of the question: what about integrating just some fields in working data collection? let me explain: owrt based 5g modems.
You can easily understand that collecting connection info for 5G modems is crucial, so it's hardly a new integration, at the most it should be a "module" of the common owrt integration. What's the correct way of doing this? (thinking about basing the info on https://github.com/ddimension/wwand )
Thanks

[gnacho]

Two answers, and the first one has a shortcut you may not have tried.

The GS308E: check SNMP before writing a pusher. Without a MAC table per port the switch cannot feed the clients-per-port view, that is true. But NetPulse already reads managed switches over SNMP, and the core walks the standard bridge MIB (dot1dTpFdbTable, RFC 1493) with a dot1q fallback, plus ifTable for counters and link state. If your 308E exposes SNMP, some Plus firmwares do, you get ports, counters and the forwarding table without a line of new code: add it as a managed switch and point the SNMP community at it. If it does not, what you have is ports, link state, speed, PVID/VLAN and counters. That is useful for traffic per port, but it cannot place clients, and I would not build a vendor pusher for it: a lot of code for a view the SNMP path already covers when the firmware allows it. Send me the output of snmpwalk -v2c -c public <switch> 1.3.6.1.2.1.17.7.1.2.2 and we will know in a minute.

The 5G modem is not an integration, it is a section of the OpenWrt agent. The native agent already runs on the router and sends system, interfaces, wireless, DHCP and FDB. What wwand gives you is modem state (signal, operator, bands, SIM), which belongs in the same payload as a wwan section, collected only when the modem is present. That is core plus agent work, not a vendor pusher: it ships with a NetPulse release and reaches every OpenWrt unit with a modem, instead of one pusher per model. The contract is small: the fields you actually use from wwand, with a real capture from your modem, and the agent leaves the section out when there is no modem. Open an issue with the capture and I will write the contract.

One caveat for both: read-only, and real fixtures. That is the bar in this repository.

[MassiPi]

The GS308E: check SNMP before writing a pusher.

This little devil never thought to be a real namaged switch, no snmp, nothing, nada :)

If it does not, what you have is ports, link state, speed, PVID/VLAN and counters. That is useful for traffic per port, but it cannot place clients, and I would not build a vendor pusher for it: a lot of code for a view the SNMP path already covers when the firmware allows it.

I do agree. Totally.

That is core plus agent work, not a vendor pusher: it ships with a NetPulse release and reaches every OpenWrt unit with a modem, instead of one pusher per model.

That's my point. You talk about the agent. I ask why agent. It's something we already talked about :) a dedicated modem sits in the WAN zone (a cpe sits also out of the building..), why is it needed to open a connection toward the central NetPulse server?
It's secured, ok, it's just one port, ok, but what is the real advantage? It will probably have one fixed client (the real router). It easily will not even have a wifi (mine has a 2.4G for management that i keep disabled..), is it worth the agent to see the connection data timely?
And second point: WWAND is what i'm using. It is not the only way to use a 5g modem (think about modem manager). There is no standard, afaik, so is adopting a specific library the path you want to follow?
I mean, wwand is modern and works much better than modem manager, but this means bonding netpulse to a custom library.
Anyway, if this is the path, wwand exposes a lot of information via CLI, so not a problem at all to collect wathever we like..
https://github.com/ddimension/wwand/blob/main/docs/reference.md#wwandctl--command-line-control