Skip to content
master
Go to file
Code

Latest commit

 

Git stats

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
 
 
 
 
src
 
 
 
 
 
 
 
 
 
 
 
 

README.md

Musketeer is a workflow manager which can dynamically map front-end workflow descriptions to a range of back-end execution engines. It is developed by CamSaS (http://camsas.org) at the University of Cambridge Computer Laboratory.

Musketeer is currently a prototype: it takes as input workflows defined in several domain specific languages and can generate code for several back-end execution engines (e.g, Spark, Naiad, Hadoop).

System requirements

Musketeer is currently known to work on Ubuntu LTS release 14.04 (trusty). Other configurations are untested.

Moreover, Musketeer currently generates code compatible with the following back-end execution engine versions:

  • GraphChi 0.2
  • Hadoop 2.0.0-mr1-cdh4.5.0
  • Metis e5b04e2
  • Naiad 0.4
  • PowerGraph 2.2
  • Spark 0.9

Other versions may work if the APIs have not changed too much. If you have added support for new versions (or just found them to work), please let us know or send us a pull request.

Getting started

After cloning the repository,

$ make dependencies

fetches dependencies, and may ask you to install required packages.

$ make all

If all goes well, you should end up with the Musketeer binary in the build directory.

Musketeer assumes that back-end execution engines are installed relative to MUSKETEER_ROOT, unless other locations are specified explicitly using flags. You are free to use any directory for MUSKETEER_ROOT. It does not have to be inside Musketeer's source directory.

In order to test your build you can generate Hadoop code for running PageRank using the following command:

./build/musketeer --logtostderr --stderrthreshold=0 --run_daemon=false --root_dir=MUSKETEER_ROOT --force_framework=hadoop --dry_run -beer_query=tests/pagerank/page_rank.rap

If you already installed Hadoop then if you pass --dry_run=false then Musketeer will run also run the computation.

About

The Musketeer workflow manager.

Resources

License

Releases

No releases published

Packages

No packages published
You can’t perform that action at this time.