A script to generate a lot of oar jobs, but to schedule and run them little at a time.
Switch branches/tags
Nothing to show
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Permalink
Failed to load latest commit information.
README.md
oarhold
oarwake

README.md

oarhold

This script allows you to generate a lot of jobs using the array-param-file option of oarsub but to execute few of them at a time.

Use this with a notify script that will wake up a new job a soon as one is completed.

./oarhold [NUMBER_OF_CONCURRENT_RUNNING_JOBS]

The default number of concurrent running jobs is set to 50.

oarwake

This script wakes up a held job whose id is in a file when another job completes.

To use with the oarhold script.

Configuration

Set the following variables in oarhold:

PROG="/path/to/myprogram"
PROGNAME="somename"
ARRAY_PARAM_FILE="./params.txt"
NOTIFY_SCRIPT="/path/to/oarwake"
JOBS_FILE="/path/to/jobs.txt"
JOBS_LOCK_FILE="/path/to/jobs.lck"

MAX_CONCURRENT_JOBS=50

Set the following variables in oarwake:

JOBS_FILE="/path/to/jobs.txt"
JOBS_LOCK_FILE="/path/to/jobs.lck"
JOBS_LOG_FILE="/path/to/jobs.log"

Ensure that the JOBS_FILE and JOBS_LOCK_FILE are consistent between both scripts.

Usage

After configuring both scripts, launch ./oarhold [N] where N is the desired number of concurrent running jobs.

It will create and hold as many instances of PROG required by the ARRAY_PARAM_FILE content. Those jobs will only be scheduled and run N by N.

Each time a job completes, the oarwake script will be triggered, which will schedule the next held job, until all held jobs have been executed.