Skip to content
master
Go to file
Code

README.md

Kafka CodaHale Metrics Reporter + Riemann metrics consumer

The goals of this repo is to have an end to end working environment to provide the ability for systems (applications and infrastructure) to send their metrics/sensor data to Kafka and then to report on that data for both alerts and charts.

More about this project in this blog post http://allthingshadoop.com/2014/04/18/metrics-kafka/.

Quick up and running

Use Vagrant to get up and running.

  1. Install Docker http://docs.docker.com/installation/
  2. Install Virtual Box https://www.virtualbox.org/

In the main metrics-kafka folder

  1. sudo ./bootstrap.sh
  2. ./gradlew test
  3. sudo ./shutdown.sh

once this is done

  • Zookeeper will be running localhost:2181
  • Broker 1 on localhost:9092
  • Riemann on localhost:5555
  • Riemann dash on localhost:4567
  • All the tests in metrics/src/test/java/* and riemann/src/test/scala/* should pass

You can access the brokers, zookeeper, riemann and riemann-dash by their IP from your local without having to go into vm.

e.g.

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic

bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic --from-beginning

http://192.168.86.5:4567/#Riemann ./gradlew :psutil:installDependencies ./gradlew :metrics-test:run

Kafka Yammer Metrics reporting

In order to assemble jar for metrics yammer do the following steps:

  1. ./gradlew :yammer:jar
  2. put the jar from metrics-yammer/build/libs to libs dir in root kafka folder
  3. add "kafka.metrics.reporters=ly.stealth.kafka.metrics.KafkaBrokerReporter" to config/server.properties in kafka root folder

About

Metrics produced to Kafka and consumers for monitoring them

Resources

License

Releases

No releases published

Packages

No packages published
You can’t perform that action at this time.