Branch from PAR. Vanilla PAR uses PyRo as an RPC system, this one is using rfoo.
Python Shell
Switch branches/tags
Nothing to show
Clone or download
Pull request Compare This branch is 10 commits ahead, 10 commits behind UnixJunkie:master.
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
bin
cfg
src
tests
BIOINFORMATICS_EXAMPLE
CONTRIBUTORS
GPL_LICENSE.txt
Makefile
README
README.md
TODO
test_parallel.input
test_parallel.output.reference
test_parallel.sh

README.md

Citation

If you use this software, please cite the corresponding publication:

@article{Berenger2010,
author = {Berenger, Francois and Coti, Camille and Zhang, Kam Y. J.},
title = {{PAR: A PARallel And Distributed Job Crusher}},
doi = {10.1093/bioinformatics/btq542},
year = {2010},
journal = {Bioinformatics}
}

External dependencies

You need to install rfoo: http://code.google.com/p/rfoo/

Some usage examples:

Basic examples

Run in parallel commands from test_parallel.input

cat test_parallel.input | ./parallel.py -i /dev/stdin

Same than before but with a progress bar

cat test_parallel.input | ./parallel.py -i /dev/stdin -v

Run in parallel commands from test_parallel.input and store output in output.log

./parallel.py -i test_parallel.input -o output.log

Same than before but with a user defined output printer

./parallel.py -i test_parallel.input -o output.log -p post_proc_example

real world usage example

  1. server side
./parallel.py -v -i many_commands.sh -o par_many_commands.log -s
  1. client side, on each machine you want to join the computation replace SERVER_NAME by the machine name from where you launched parallel.py using -s
./parallel.py -c SERVER_NAME

For example, a set of clients can be spawned very quickly on a set of machines using TakTuk: http://taktuk.gforge.inria.fr/

taktuk -s -f machinefile broadcast exec {./parallel.py -c SERVER_NAME} 

where the list of machines is given in the machinefile file.

  1. be thrilled! ;)