Networking introduction
If you're collecting performance data for more than just one host, you probably want to have the data in one central place instead of having it spread out over potentially hundreds of servers. That's exactly what the Network plugin is there for. All the configuration options are documented in the collectd.conf(5) manual page.
collectd's Network plugin implements some very sophisticated features. We'll start with some simple examples and work our way up to the technically interesting setups. Since the statistics are collected on one host and stored on another host, the communication is unidirectional, i. e. one-way only. In the following we will call the process which is collecting the statistics (e. g. by reading a file in /proc) and sending them to another host the "client", while the process listening on a socket, receiving and storing the values is called the "server".
The most common and easiest setup is where you have one "server" and a bunch of "clients", which send their data directly to the server.
On the server you need to load the Network plugin, a plugin to store the data somewhere (the most common plugin to do that is the RRDtool plugin) and a plugin to handle log-messages, i.e. either the LogFile plugin or the SysLog plugin:
# Server
LoadPlugin "logfile"
LoadPlugin "network"
LoadPlugin "rrdtool"
The Network plugin now needs to be told that it should be receiving data, making this instance a "server". This is done by telling it which IP-address to listen on using the Listen configuration option. We use the IP-address 192.168.0.42 in this example. You need to substitute that with the IP-address of the server. IPv6-addresses can be used here, too.
# Server
<Plugin "network">
Listen "192.168.0.42"
</Plugin>
On the clients the Network plugin needs to be loaded, too. The output plugin (in our example that's the RRDtool plugin) does not need to be loaded. But you'll need some plugins that actually collect some data. In this example we'll use the CPU and Memory plugins as examples for plugins which collect data.
# Client
LoadPlugin "logfile"
LoadPlugin "network"
LoadPlugin "cpu"
LoadPlugin "memory"
Now you need to tell the Network plugin where it should send the data to. Again, in this example this will be the IPv4-address 192.168.0.42:
# Client
<Plugin "network">
Server "192.168.0.42"
</Plugin>
See Troubleshooting.
Multicast addresses (both, IPv4 and IPv6) are recognized automatically. On the client there is no difference to sending to a unicast address. The server, however, needs to "subscribe" to the multicast group. This is done automatically when a multicast group is configured. So the following example is all there's to it:
# Server
<Plugin "network">
Listen "ff18::efc0:4a42"
</Plugin>
By default UDP port 25826 is used. If that's a problem for some reason, you can configure a different port using the second argument to Listen and Server:
# Client
<Plugin "network">
Server "192.168.0.42" "25827"
</Plugin>
# Server
<Plugin "network">
Listen "192.168.0.42" "25827"
</Plugin>
If you have multiple servers and your clients are sending to a multicast group, all you need to do is install the next server exactly as the ones before. Multicast means the values are sent to all interested parties, so this is exactly what's happening. Sometimes multicast makes your job so easy, it's almost boring.. ;)
If you have multiple unicast servers (or, of course, a mixture of unicast servers and multicast groups) you need to configure all the servers on the clients. Simply use the Server option multiple times:
# Client
<Plugin "network">
Server "192.168.0.42"
Server "172.16.0.42"
Server "239.192.74.66"
Server "ff18::efc0:4a42"
</Plugin>
If you need to listen on more than just one interface, or you want to subscribe to more than one multicast group (or any variation of the two), just use the Listen option multiple times:
# Server
<Plugin "network">
Listen "192.168.0.42"
Listen "172.16.0.42"
Listen "239.192.74.66"
Listen "ff18::efc0:4a42"
</Plugin>
Starting with version-4.7 the Network plugin provides the possibility to cryptographically sign or encrypt the network traffic. This makes it (hopefully) impossible for an unauthorized person to send forged network packets and, in case of encrypted traffic, read the data being sent. The library used is GnuPG's libgcrypt which is available for many platforms.
On the server side, i.e. on the instance receiving values, you first need to create a authentication file (or password file). This file will be used to look up the password belonging the user the client claims to be. The syntax of the authentication file is very simple:
user0: foo
user1: bar
This file would specify two users, “user0” and “user1”, with the passwords “foo” and “bar”. The file is re-read whenever it changes.
You then need to decide on the security level required for incoming packets. The security level is one of three values:
- None: No cryptographic protection is required at all, i. e. all data will be accepted. For backwards compatibility, this is the default. Sign: Do not accept unauthenticated data. This means only signed or encrypted data is accepted. Encrypt: Only encrypted data is accepted.
The server configuration then looks somewhat like this:
<Plugin "network">
<Listen "192.168.0.42">
SecurityLevel "Sign"
AuthFile "/etc/collectd/auth_file"
</Listen>
</Plugin>
On the client the setup is a bit more straight forward: You have the same security levels available. Here, “None” means that the data is not cryptographically protected, “Sign” means that the data will be signed and “Encrypt” means that the data will be encrypted. For signed and encrypted communication, you also need to configure a user name and a password.
The client configuration then looks somewhat like this:
<Plugin "network">
<Server "192.168.0.42">
SecurityLevel "Encrypt"
Username "user0"
Password "foo"
</Server>
</Plugin>
This table gives an overview over what configurations between client and server are possible.
|
Client |
||||
|---|---|---|---|---|
|
None |
Sign |
Encrypt |
||
|
Server |
None (no AuthFile) |
accepted |
accepted |
not possible |
|
None (with AuthFile) |
accepted |
accepted |
accepted |
|
|
Sign |
not accepted |
accepted |
accepted |
|
|
Encrypt |
not accepted |
not accepted |
accepted |
|
If for some reason the server is not directly reachable by the client, you can use a third instance as proxy between the two. This setup requires the option Forward to be set to true on the proxy:
# Client
<Plugin "network">
Server "172.16.0.42"
</Plugin>
# Proxy
<Plugin "network">
Listen "172.16.0.42"
Server "192.168.0.42"
Forward true
</Plugin>
# Server
<Plugin "network">
Listen "192.168.0.42"
</Plugin>
Of course, you can use the above setting to translate between IPv4 and IPv6, or to convert between multicast and unicast communication. Or both at once. It's a proxy alright, I think you get the picture.
# Client, sending to IPv4 multicast group
<Plugin "network">
Server "239.192.74.66"
</Plugin>
# Proxy, translating from IPv4 multicast to IPv6 unicast
<Plugin "network">
Listen "239.192.74.66"
Server "ff18::efc0:4a42"
Forward true
</Plugin>
# Server
<Plugin "network">
Listen "ff18::efc0:4a42"
</Plugin>
Building on the idea of using a proxy, you can also write to a collectd server, and then from the server register a write plugin such as the Write Redis plugin or the Write MongoDB plugin to store the data. This gives you some access control as only the collectd servers will need database access. Furthermore, the dependencies of collectd client instances are reduced.