esmond: ESnet Monitoring Daemon
At this time esmond is only supported as part of the perfSONAR toolkit.
esmond is a system for collecting, storing, visualizing and analyzing large sets of timeseries data. It was driven by the needs of the ESnet engineering team but is likely useful to a much wider audience. esmond has a RESTful API which allows easy access to the data which is collected. The original focus was on collecting SNMP timeseries data which is still the system's forte, but there is support for generalized timeseries data. The perfSONAR project has begun using esmond to store timeseries of network measurements.
esmond uses a hybrid model for storing data. Timeseries data such as interface counters is stored using Cassandra. esmond will save the raw data, and create summarizations similar to RRD. However, the system never discards data through summarization, which distinguishes it from RRD (and whisper/ceres). Metadata (such as interface description and interface types from SNMP) are stored in an SQL database. Storing this data in an SQL database allows us to use the full expressiveness of SQL to query this data. Since this data changes relatively infrequently the demands placed on the SQL server are fairly modest. Our production server uses PostgreSQL, but it's likely that SQLite would work just fine. Data can be visualized using Graphite or through custom visualizations which can query the RESTful API.
The documentation has a lot more details.
This repository allows you to use Vagrant to create a VM on VirtualBox with the necessary components installed. The default VM is based on CentOS 7 and creates a shared folder in the VM that points at the top-level of your checked-out copy. This allows you to edit files on your base system and have the changes automatically appear in the VM.
Install VirtualBox according the the instructions on their site for your system.
Install Vagrant according the the instructions on their site for your system.
Install the vagrant-vbguest and vagrant-reload plugins with the following commands:
vagrant plugin install vagrant-vbguest vagrant plugin install vagrant-reload
Starting the VM
Clone the esmond github repo:
git clone https://github.com/esnet/esmond
Start the VM with
vagrant up. The first time you do this it will take awhile to create the initial VM.
Using the VM
The VM sets-up port forwarding by default so you can access esmond from the host system. You can test esmond runs on HTTP and HTTPS as follows:
curl "http://127.0.0.1:20080/esmond/perfsonar/archive/" curl -k "https://127.0.0.1:20443/esmond/perfsonar/archive/"
Any changes you make to the checked-out code on your host system get reflected in the host VM under the /vagrant directory
The following symlinks are setup to files in the git copy of the code:
- /etc/esmond -> /vagrant/vagrant-data/esmond-el7/etc/esmond
- /usr/lib/esmond -> /vagrant
- /usr/sbin/esmond_manage -> /vagrant/util/esmond_manage
- /etc/profile.d/esmond.csh -> /vagrant/rpm/config_files/esmond.csh
- /etc/profile.d/esmond.sh -> /vagrant/rpm/config_files/esmond.sh
You must run
systemctl restart httpdwhenever you make a code change
vagrant reloadto restart the VM
vagrant suspendto freeze the VM. Running
vagrant upagain will restore the state it was in when you suspended it.
vagrant haltto shutdown the VM. Running
vagrant upagain will run through the normal boot process.
vagrant destoryto completely delete the VM. Running again
vagrant upwill build a brand new VM.