Skip to content
This repository has been archived by the owner. It is now read-only.
Branch: master
Clone or download
Fetching latest commit…
Cannot retrieve the latest commit at this time.
Type Name Latest commit message Commit time
Failed to load latest commit information.



Instructions below are for testing the development code. Subject to change.


  • *nix
  • Python 3.4+
  • Redis 4.0+. To download and install Redis see here
  • virtualenv


It is recommended to install this package in a virtual environment, e.g:

Make a virtual environment:

virtualenv -p python3 testenv3

Activate the virtual environment:

source testenv3/bin/activate

Install the latest PRC code from github:

pip install git+

Install this repository's current development branch:

pip install git+


Start redis:

cd REDIS_DIR/src

From our virtual environment call the main app, e.g:

irodsqueue --help
irodsqueue ingest --help

Enqueue ingest jobs, for example to ingest a directory into iRODS:

irodsqueue ingest -f --timer /PATH/TO/LOCAL/DIR

Once we have jobs in the queue we can launch workers to process them. Each worker is its own process. We use a custom worker class that opens and maintains iRODS sessions. Open a separate terminal and activate our same virtual environment as above.

Test your iRODS connection:


Launch 16 worker processes (e.g. in bash):

for i in {1..16}; do sleep .1; rq worker -v --burst -w irodsqueue.irodsworker.IrodsWorker & done


You can specify a metadata extraction function to invoke as part of the ingest process, by using the --metadata option when enqueuing jobs. The value should be the path to a python file containing a function named extract_metadata. See examples

Enqueue jobs with metadata extraction:

irodsqueue ingest -f --extract-metadata ~/tests/metadata/ --timer /PATH/TO/LOCAL/DIR

Enqueue jobs with file checksums:

irodsqueue ingest -Kf --timer /PATH/TO/LOCAL/DIR
You can’t perform that action at this time.