Skip to content

HTTPS clone URL

Subversion checkout URL

You can clone with
or
.
Download ZIP
Fetching contributors…

Cannot retrieve contributors at this time

14 lines (10 sloc) 0.729 kB

Running a Hadoop job

It's possible to setup Scribe to log into Hadoop. If you do this you can generate various reports from the data that is not easy to do on the fly in Zipkin itself.

We use a library called Scalding to write Hadoop jobs in Scala.

  1. To run a Hadoop job first make the fat jar. sbt 'project zipkin-hadoop' compile assembly
  2. Change scald.rb to point to the hostname you want to copy the jar to and run the job from.
  3. Update the version of the jarfile in scald.rb if needed.
  4. You can then run the job using our scald.rb script. ./scald.rb --hdfs com.twitter.zipkin.hadoop.[classname] --date yyyy-mm-ddThh:mm yyyy-mm-ddThh:mm --output [dir]
Jump to Line
Something went wrong with that request. Please try again.