An alternative to StatsD/Graphite.
Ruby Shell
Switch branches/tags
Nothing to show
Latest commit 6803a0b May 19, 2013 @malditogeek malditogeek Merge pull request #2 from barisbalic/fix-leveldb-explosion
Check for LevelDB path at startup.

README.md

Astor, an alternative to StatsD+Graphite

Astor Piazzolla

This is: astor-collector

Metric collector and REST API. Based on EventMachine, Goliath and LevelDB.

You'll also need: astor-dashboard

Real-time metric visualization. Based on SocketStream (Node) and Backbone. Find it here.

Collector features

  • API compatible with StatsD, push data with any StatsD client
  • Second-by-second data resolution for active metrics
  • Minute-by-minute data resolution for archived metrics
  • Unlimited retention (well, as much as you can store)
  • Metric trend monitoring and alerting
  • Event streaming through ZeroMQ pub/sub
  • REST API

Pre-requisites

  • Ruby 1.9+
  • ZeroMQ dev (>= 2.1, < 3.0)

Getting started

If you already have a working Ruby environment, should be as easy as:

bundle install
./astor

Deployment

Deploying Astor to EC2 is pretty straight forward. On AWS Console:

  • Create a new Amazon EC2 instance using the Ubuntu 12.04 AMI.

  • We'll use capify-ec2 so you need to tag your instance properly

  • Customize your Security Groups and open the right ports

  • Once the instance is up and running, ssh into it and run:

    sudo mkdir /var/astor
    sudo chown -R ubuntu:ubuntu /var/astor
    sudo apt-get update
    sudo apt-get install git build-essential libxslt-dev libxml2-dev ruby1.9.3 libzmq-dev
    sudo gem install bundler
    

Then, on your local clone:

  • Copy config/astor.yml.sample to config/astor.yml and customize it if needed

  • Copy config/deploy.rb.sample to config/deploy.rb and customize it with your repository

  • Copy config/ec2.yml.sample to config/ec2.yml and complete it with your EC2 keys

    cap deploy:setup
    cap deploy
    

About Heroku: Is not possible to route UDP traffic at the moment but this may change in the future (https://groups.google.com/forum/?fromgroups=#!topic/heroku/z6haBrCQ9ZA)

REST API

id format is: type-metric-key, for example:

  • product.clicks counter => c-product-clicks)
  • api.request timer => ms-api-request
  • current.users gauge => g-current-users

Availabe methods:

Event Stream

Check example

TODO

  • Alerts:
    • Configurable thresholds (per metric maybe?)
    • Storage
    • Expose as REST resource (POST deploy notifications, etc)
  • Configurable data retention