A platform for visualization and real-time monitoring of data workflows
Switch branches/tags
Nothing to show
Pull request Compare This branch is 573 commits behind twitter:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Failed to load latest commit information.


Ambrose Build Status

Twitter Ambrose is a platform for visualization and real-time monitoring of MapReduce data workflows. It presents a global view of all the map-reduce jobs derived from your workflow after planning and optimization. As jobs are submitted for execution on your Hadoop cluster, Ambrose updates its visualization to reflect the latest job status, polled from your process.

Ambrose provides the following in a web UI:

  • A chord diagram to visualize job dependencies and current state
  • A table view of all the associated jobs, along with their current state
  • A highlight view of the currently running jobs
  • An overall script progress bar

Ambrose is built using the following front-end technologies:

  • d3.js - For chord diagram visualization
  • Bootstrap - For layout and CSS support

Ambrose is designed to support any Hadoop workflow runtime, but current support is limited to Apache Pig.

Follow @Ambrose on Twitter to stay in touch!

Supported runtimes


Below is a screenshot of the Ambrose UI. Each arc segment on the circle represents a map-reduce job. Dependencies between jobs are represented by chords which connect job arc segments. Grey jobs have not yet run, bright green jobs are running and light green jobs are completed. When the mouse hovers over a job, its arc and input dependencies are highlighted blue. Clicking on the job will select it, updating the contents of the table to the right of the diagram with information about the selected job.

Note that Each job arc is bisected; Chords on one half of the arc connect to predecessor jobs while chords on the other half connect to successor jobs. For example, in the diagram below jobs 1 and 3 have no predecessors while jobs 16, 20, 21, and 22 have no successors (their outputs represent the final result of this workflow).

Note that the chord diagram shown is our first pass at visualizing the workflow, and there's room for improvement. We'd like to support other visualizations as well, like a graph of the workflow DAG. If you develop an improved visualization, be sure to send us a pull request!

Ambrose UI screenshot


To get started with Ambrose, first clone the Ambrose Github repository:

git clone https://github.com/twitter/ambrose.git
cd ambrose

Next, you can try running the Ambrose demo on your local machine. The ambrose-demo script starts a local instance of the Ambrose app server with sample data. Start the demo Abrose server with the following command and then browse to http://localhost:8080/web/index.html?localdata=small:


Finally, you can run Ambrose with an actual Pig script. To do so, you'll need to build the Ambrose distribution and untar it:

tar zxvf ambrose-$VERSION.tar.gz

You can then run the following commands to execute path/to/my/script.pig with an Ambrose app server embedded in the Pig client:

cd ambrose-$VERSION
./bin/pig-ambrose -f path/to/my/script.pig

Now, browse to http://localhost:8080/web/ to see the progress of your script using the Ambrose UI. To override the default port, export AMBROSE_PORT before invoking pig-ambrose:

export AMBROSE_PORT=4567

Maven repository

An initial release will be pushed to Maven shortly.

How to contribute

Bug fixes, features, and documentation improvements are welcome! Please fork the project and send us a pull request on Github. You can submit issues on Github as well.

Here are some high-level goals we'd love to see contributions for:

  • Improve the front-end client
  • Add other visualization options, like a DAG view
  • Create a new back-end for a different runtime environment
  • Create a standalone Ambrose server that's not embedded in the workflow client


For transparency and insight into our release cycle, releases will be numbered with the follow format:


And constructed with the following guidelines:

  • Breaking backwards compatibility bumps the major
  • New additions without breaking backwards compatibility bumps the minor
  • Bug fixes and misc changes bump the patch

For more information on semantic versioning, please visit http://semver.org/.



Copyright 2012 Twitter, Inc.

Licensed under the Apache License, Version 2.0: http://www.apache.org/licenses/LICENSE-2.0