Skip to content
This repository

SAGA-based Pilot-Job Implementation for Compute and Data

Octocat-spinner-32 api Added wait() function to bigjob class February 02, 2012
Octocat-spinner-32 bigjob added safety check January 21, 2014
Octocat-spinner-32 bigjob_dynamic Remove appwriting and pilotdata, these should not be front-facing pag… December 05, 2013
Octocat-spinner-32 bootstrap Catch file not found exception January 26, 2014
Octocat-spinner-32 cli further refinement of API documentation July 07, 2012
Octocat-spinner-32 coordination Merge branch 'master' into develop October 26, 2013
Octocat-spinner-32 doc Remove all references to the gw68 password in code and examples March 21, 2013
Octocat-spinner-32 docs Fix indices December 05, 2013
Octocat-spinner-32 examples fixed examples so that they do not contain compile errors January 21, 2014
Octocat-spinner-32 pilot fixed examples so that they do not contain compile errors January 21, 2014
Octocat-spinner-32 scripts work on path handling for irods December 14, 2012
Octocat-spinner-32 tests optimizing performance of ComputeDataService July 30, 2013
Octocat-spinner-32 util fix get_details() July 09, 2013
Octocat-spinner-32 .gitignore Further work on Google Storage integration for PD August 11, 2012
Octocat-spinner-32 CHANGES updated VERSION for release January 15, 2012
Octocat-spinner-32 LICENSE.md Added License (so people are forced now to attribution) July 29, 2012
Octocat-spinner-32 MANIFEST.in added VERSION to package January 22, 2014
Octocat-spinner-32 Makefile added 'clean' target June 28, 2013
Octocat-spinner-32 README-CONDOR.md separate README for Condor January 31, 2012
Octocat-spinner-32 README.md Merge branch 'develop-prod' July 27, 2013
Octocat-spinner-32 bigjob.conf updated bootstrapped version of BJ August 10, 2013
Octocat-spinner-32 bigjob.sublime-project alt. PyPi project to minimize potential damage June 28, 2013
Octocat-spinner-32 bigjob_agent.conf Make executor threadpool size configurable July 14, 2013
Octocat-spinner-32 ez_setup.py use setuptools 2.1 for packaging bj January 23, 2014
Octocat-spinner-32 setup.py force setuptools 2.x switch January 25, 2014
README.md

BigJob

The SAGA BigJob framework is a SAGA-based pilot job implementation. The Simple API for Grid Applications (SAGA) is a high-level, easy-to-use API for accessing distributed resources. SAGA BigJob supports a wide range of application types, and is usable over a broad range of infrastructures, i.e., it is general-purpose, extensible and interoperable. Unlike other common pilot job systems SAGA BigJob (i) natively supports MPI job and (ii) works on a variety of back-end systems, generally reflecting the advantage of using a SAGA-based approach.

Web Page & Mailing List

Web page: http://saga-project.github.io/BigJob/

Mailing list:

bigjob-user@googlegroups.com

bigjob-devel@googlegroups.com

Installation

$ curl https://raw.github.com/saga-project/BigJob/master/bootstrap/bigjob-bootstrap.py -o bigjob-bootstrap.py
$ python bigjob-bootstrap.py $HOME/.bigjob/python/
$ . $HOME/.bigjob/python/bin/activate

See https://github.com/saga-project/BigJob/wiki/ for further information. The Wiki includes among many things information on how to setup BigJob on various machines, e.g. on XSEDE (Ranger, Kraken) or FutureGrid.

Configuration of Redis Coordination Backend

BigJob requires Redis for communication between BigJob manager/agent. Use the following steps to install Redis.

Mac Homebrew:

brew install redis redis-tools 

Linux: Download Redis http://redis.io/download Currently only tested with Redis 2.2.12

$ tar -xzvf redis-2.2.12.tar.gz
$ cd redis-2.2.12
$ make
$ make install (if root)

Start Redis (redis-server executable is located in src/ dir):

cd src
./redis-server

Examples:

https://github.com/saga-project/BigJob/tree/master/examples/pilot-api

Please modify COORDINATION_URL on top of example to the correct Redis backend!

Development & Packaging

Requirements:

Building PyPi package

python setup.py build

Upload to PyPi

python setup.py sdist upload
Something went wrong with that request. Please try again.