Copyright LP Programming L.L.C. 2025
Inspired by bandwidthd, by David Hinkle. Special thanks to the bandwidthd team of developers. Thanks also to DerbyTech, now DerbyNet of Illinois, who funded the original bandwidthd.
Normally, everything I write I release under the ZLib license. However, in keeping with and gratitude for the original bandwidthd, this project is licensed under the GPL 2.0.
Bandwidthd-ng is a UNIX daemon for logging or graphing IPv4/IPv6 traffic generated by each machine on several configurable subnets. It can log to cdf files, or psql or sqlite databases, as well as generate graphs itself.
In live graphing mode, each IP which has transferred more than a small amount of data (configurable, 1mb default) is graphed, as is each monitored subnet as a whole. The graph format is mostly the same, down to the same colors, as the original, but the format is svg instead of png. This lets even a modest machine genrate graphs very quickly.
Bandwidthd-ng will compile and run on any platform with libpcap, and a recent version of clang and libcxx. This sounds like a bigger restriction than it is, as Gentoo provides libcxx for most platforms. A fully static build is about 1.7MB, upx gets it down to about 500kb. If you can use the target system’s libc, you can cut that about in half. It’s also possible to swap the stl containers for embedded stl implementations.
At runtime, it takes about 8mb of RAM on 64 bit platforms, and about half that on 32bit platforms. Again, swapping to an embedded stl, disabling exceptions, and disabling features at compile time can cut that about in half.
The primary build target of bandwidthd-ng is the classic target, which aims to support the same feature set and config compatibility with the original bandwidthd, all in one executable. There are some differnces, largely because I can’t run the original bandwidthd to verify its behavior. I also never used it to generate graphs. So I only have the old C source code and the docs to recreate it. Its psql backend entirely works as a drop-in replacemnt.
The original relied on php to genrate graphs from the database, which resulted in db-driven graphs looking different to the live graphs. bandwidthd-ng provides a graphing program that uses the same module to create the graphs. For fully dynamic graphs, this library is exposed through the Python wrapper, so you can query the db via python’s db2 api, and pass the data to the traffic grapher to generate the svgs.
bandwidthd-ng graphs all active IPs in targeted subnets, every 200 seconds. You can still use the skip interval to reduce that frequency. If you want more fine grained control, use a database. The end application code is also very simple, so you can easily make new frontends with different behavior. If you write one that works exactly as you wish, send it to me and I can include it.
bandwidthd-ng’s cdf logging is off by default. It uses the same cdf format as before. Currently, it can’t recover cdfs on startup, though that should be easy to add. It also doesn’t do its own log rotating anymore. On modern Linux systems, you usually want to put log rotation under the control of your service manager. It doesn’t keep the log file open between writes, so simply moving the file out from under it is sufficient to rotate the log.
Summary webpages are not generated. I may write a simple frontend using the python extension, but that is fairly low priority.
You can set the libpcap filter on the light utilities via the --filter
parameter. In the classic program, you can set it via the
filter “args”
format in the config file. The default is “ip or ipv6”, which usually works well.
Like with the original bandwidthd, bandwidthd-ng supports sending data to an external database. It also supports reading data back from postgres for graphs. It can send data to sqlite3, but you’ll need the python plugin to graph from that. Also keep in mind sqlite3 doesn’t like multiple concurrent access, so take a copy or otherwise prevent your reader from colliding with bandwidthd-ng’s writes. For psql, you can have multiple sensors running, sharing a database. The sensor id is included in the database records.
Using bandwidthd-ng with a database, especially if using the db-only sensor, reduces the resource demands considerably. An embedded-stl/noexcept build of the psql-sensor needs under 1mb of ram, for reasonable subnet sizes.
You must have a postgresql server running somewhere accessible to the user account running bandwidthd-ng or psql-sensor.
Setup:
- Create a database for bandwidthd.
- For the classic program, ensure your bandwidthd.conf has the following set
graph false
recover_cdf false
output_cdf false
# fqdns work well, in reverse order for sorting, but any ascii string under 255
# characters works well.
sensor_id "com.example.sensorN"
# same as php and classic uses. I think libpqxx will read environment variables
# for this, but this string must be non-empty to enable graphing.
pgsql_connect_string "user = un dbname = db host = example.com"- Start bandwidthd-ng. If you pass
-Dit won’t fork, which will make error identification easier. Wait 200 seconds, then check psql, you should see entries.
The original bandwidthd shipped with a script to compact old psql data, by virtually increasing the sample duration on data older than some threshold. It is worth grabbing that script and running it nightly if you are using this in a large scale environment. Since the database is the same format, the old script should work. If I get around to writing the web frontend, I’ll probably integrate the functionality into that.
You’ll note this project doesn’t use CMake, or a CMake frontend like qmake, premake, or similar. This is for 3 reasons.
CMake struggles with modules. It can build them, but clang’s implicit headers
aren’t easy to support, and it isn’t good at correctly detecting what actually
needs rebuilding. I recently found the documentation to get import std;
working, so this part could be partially mitigated.
CMake also can’t detect ABI incompatibilities in advance. When working with
Clang and libc++ on Linux, if you try to use a library built with libstdc++, you
can’t link. CMake waits until the link step and then gives opaque errors about
missing core componets like std::basic_string. No amount of additional work on
my part can resolve this with CMake.
The recommended solution is to not rely on prebuilt dependencies: Just refernce the cmake files of all your deps and build them all with your current toolchain. Also, C+ really needs an NPM-like package manager to make this easy.
This is a terible idea. It opens the door to supply chain attacks. It violates separation of concerns. It increases build times. It multiplies exponentially the number of dependency versions that might interact. In short, it creates dependency hell.
This project uses only the standard libraries, and libraies commonly available via most distribution package managers.
That dependency problem created by having CMake chase down and include other libraries extends to a special kind of Dependency Hell called License Hell. CMake does a lot for you. If you ask it to make static programs, it’ll just do it. It’ll even reach upstream to the cmake projects you included and make them static. The problem is it has no idea what licenses are involved. Even if you take the time to look them up and ensure the default build configuration creates legal-to-distribute programs, it has no way to encode that information to prevent future or downstream issues. I can’t entirely solve this, but I can ensure the build system prevents accidentally stepping in it.