Sucks up metrics from various external sources and puts the data into internal systems. This is a good thing to do because whilst all the external SAAS services we use are fantastic, it's also sometimes very useful to be able to be able to view and manipulate the data in one place. For example, we might want to make a single graph with some internally generated application metrics, with some perfomance metrics from New Relic, with some metrics from an AWS Elastic Load Balancer. Perhaps. Anyway, vacuumetrix is a collection of ruby scripts that talk to various APIs and allows the collection of this data in one place.
Currently supported Inputs
- New Relic
- AWS Cloudwatch (some services)
- Neustar Web Performance Management
- facebook "likes"
- twitter "followers"
Currently supported Outputs
git clone this repo
git clone https://github.com/99designs/vacuumetrix.git
apt-get install ruby build-essential libcurl3 libcurl3-gnutls libcurl4-openssl-dev
gem install json socket
If you're outputting to ganglia gmond you'll also need to
gem install gmetric
The config.rb file contains all the local configuration variables.
cd conf cp config.rb-sample config.rb
update config.rb with your organization's API key and account number
gem install curb xml-simple
update config.rb with your organization's AWSAccessKeyId and AWSSecretKey with permission to read the Cloudwatch API
gem install fog
update config.rb with your organization's API Key and Secret
gem install curb time
gem install curb
your Graphite server your Ganglia-gmond server
(Note: If you haven't already done so you will need to activate the New Relic REST API. See here: http://blog.newrelic.com/2011/06/20/new-data-api/
Running the scripts
The scripts are in the bin directory and are designed to be run from cron every minute.
* * * * * /opt/vacuumetrix/bin/NewrelicEnduser.rb 123 metricyouwant
Get New Relic End User (RUM) stats. Supply two args, app and metric.
Get the threshhold values for all your applications. This includes average RAM, CPU, DB etc.
Get EBS metrics. No arguments. Run every 5 minutes. No point running more frequently.
Get Elastic Load Balancer metrics. Supply the name of the ELB.
Get RDS metrics. Supply the name of the Relational Database Service instance. (Tested with MySQL). YMMV.
Count the number of EC2 instances of each flavor. No arguments.
Get Elasticache metrics. Interesting ones anyway. No arguments.
Get Neustar Web Performance Metrics. For each monitor get duration and status for each monitored location. No arguments. Run every 5 minutes. (Not very efficient at the moment. Needs some tuning).
Argument is the name of the page you want to check the like count of.
Argument is the name of the twitter user you want to check the number of followers of.
- Other AWS metrics
- Google Analytics
Pull requests are appreciated.