Skip to content
This repository

SAGA-based Pilot-Job Implementation for Compute and Data

Fetching latest commit…

Cannot retrieve the latest commit at this time


The SAGA BigJob framework is a SAGA-based pilot job implementation. The Simple API for Grid Applications (SAGA) is a high-level, easy-to-use API for accessing distributed resources. SAGA BigJob supports a wide range of application types, and is usable over a broad range of infrastructures, i.e., it is general-purpose, extensible and interoperable. Unlike other common pilot job systems SAGA BigJob (i) natively supports MPI job and (ii) works on a variety of back-end systems, generally reflecting the advantage of using a SAGA-based approach.

Web Page


$ curl -o $ python $HOME/.bigjob/python/ $ . $HOME/.bigjob/python/bin/activate

see for further information

Configuration of Coordination Backend

BigJob supports different communication & coordination backends via a plugin mechanism. Currently supported backends:

A) SAGA Advert Service


C) ZMQ (ZeroMQ),

A) SAGA Advert Service

1) Utilise coordination url parameter of BigJob to utilize the Advert Service:

advert:// (Postgresql)

advert://localhost/ (SQLite)


1.) Install and run Redis (

Mac Homebrew: brew install redis redis-tools hiredis

Linux: Download Redis Currently only tested with Redis 2.2.12

$ tar -xzvf redis-2.2.12.tar.gz

$ cd redis-2.2.12

$ make

$ make install (if root)

Start Redis (redis-server executable is located in src/ dir):

cd src


2) Utilise coordination URL parameter of BigJob to connect to Redis instance:


Standard settings are:




1.) Install ZeroMQ and pyzmq

1.1 Install ZMQ:

$ wget

$ tar -xzvf zeromq-2.1.7.tar.gz

$ cd zeromq-2.1.7

$ ./configure --prefix=$HOME/sw/zeromq-2.1.7

$ make install

1.2 Activate BigJob virtualenv:

. $HOME/.bigjob/python/bin/activate

1.3 Install Python bindings:

$ export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$HOME/sw/zeromq-2.1.7/lib/

$ export ZMQ_DIR=$HOME/sw/zeromq-2.1.9/

$ pip install pyzmq

2) Utilise coordination url parameter of BigJob to utilize ZMQ:


will start a local tcp server.

Something went wrong with that request. Please try again.