This project is obsoleted by rabix/bunny
Reproducible Analyses for Bioinformatics
Rabix is an open source implementation of the specification being developed on the Common Workflow Language mailing list.
CWL is an informal task force consisting of people from various organizations that have an interest in portability of bioinformatics workflows. The goal is to specify a way to describe bioinformatics tools and workflows that is powerful, easy to use and allows for portability of tools/workflows and reproducibility of runs.
This repo includes a local python executor and some utilities for building docker images.
Using VirtualBox and Vagrant
We've pre-installed rabix and dependencies on a VirtualBox machine image. If you install VirtualBox and Vagrant, you can run the machine using the vagrantfile:
wget https://s3.amazonaws.com/rabix/Vagrantfile vagrant up && vagrant ssh # Example BWA run with local files sudo pip install -U rabix cd rabix/examples rabix https://s3.amazonaws.com/rabix/rabix-test/bwa-mem.json -- --reads test-data/example_human_Illumina.pe_1.fastq --reads test-data/example_human_Illumina.pe_2.fastq --reference test-data/chr20.fa
On an EC2 instance
If you launch an instance on AWS in the us-east region, you can use the public AMI
ami-60644508 which has rabix and dependencies installed. SSH to the instance using "rabix" as username and password.
To run the BWA example:
sudo pip install -U rabix cd examples rabix https://s3.amazonaws.com/rabix/rabix-test/bwa-mem.json -- --reads test-data/example_human_Illumina.pe_1.fastq --reads test-data/example_human_Illumina.pe_2.fastq --reference test-data/chr20.fa
Rabix requires Python 2.7 or 3.x to run.
There are several external dependencies for rabix.
The first one is Docker (v1.5 or later)
for running command line bioinformatics tools.
you can look for available options on
but the easiest way is probably to install something like PhantomJS or NodeJS
from your distro's repo.
Finally you should install
libyaml development package.
If you are running recent Ubuntu (14.04 or newer), the following should setup your system:
$ sudo apt-get install python-dev python-pip docker.io phantomjs libyaml-dev
although you'll probably want to install newer version of Docker from the above link.
Now we can install rabix via
$ pip install rabix
If you are using Anaconda there might be an issue with a version of
so you should create separate environment with requests 2.2.1,
prior to running
$ conda create -n rabix pip requests=2.2.1 $ source activate rabix
rabix command to see if everything went well.
There are two executables installed with rabix package:
rabix - the executor of apps and workflows, and
rabix-tools which is command line suit with various utilities.
Main argument for
rabix command is a URI (local file path or HTTP URL) to a JSON document that describes an app.
spec, so you can reference a description within larger JSON document:
When you run
rabix with tool description document, you'll be presented with arguments you need to fill in in order to run it so for example:
will produce the following output:
Usage: rabix <tool> [-v...] [-hcI] [-d <dir>] [-i <inp>] [--resources.mem=<int> --resources.cpu=<int>] [-- --reads=<file>... --reference=<file> [--minimum_seed_length=<integer>] [--min_std_max_min=<array_number_separator(None)>...]...]
We see a lot of options here, but ideally, we'll be dealing with workflows that have lot of options pre-populated, so that we only need to supply few additional arguments such as yours input files or similar.
Let's try running this tool with some example files:
$ rabix -v https://s3.amazonaws.com/rabix/rabix-test/bwa-mem.json -- \ --reference https://s3.amazonaws.com/rabix/rabix-test/chr20.fa \ --reads https://s3.amazonaws.com/rabix/rabix-test/example_human_Illumina.pe_1.fastq \ --reads https://s3.amazonaws.com/rabix/rabix-test/example_human_Illumina.pe_2.fastq
Once again, we can supply either paths to local files or HTTP URLs.
Of course of we wanted to type down command line arguments for command line programs, we wouldn't need a middleman. Here is a more meaningful example:
rabix -v -i https://s3.amazonaws.com/rabix/rabix-test/inputs-workflow-remote.json \ https://s3.amazonaws.com/rabix/rabix-test/bwa-mem-workflow.json
Now we have entire workflow defined in one file and all the options we want to apply to that workflow in another, specified after
These are simple JSON files that you can write yourself, or more conveniently, use UI on rabix.org.
When running an app, rabix will look for a docker images specified in the document, pull them and run according to description. If for some reason you want to pre-fetch required images, you can use "--install" switch:
$ rabix --install https://s3.amazonaws.com/rabix/rabix-test/bwa-mem.json
How to contribute
As mentioned earlier, rabix is an implementation of Common Workflow Language, plus a playground for future ideas. First question is whether your intended contribution related to CWL in general or rabix in particular. The way you should decide this is whether it's related to overall functionality of tools, workflows and document formats (CWL) or some aspect of implementation, auxiliary utilities, etc. (rabix). If former, you should probably present your idea on the mailing list or submit an issue to the Common Workflow Language project.
If you are interested in contributing to rabix, feel free to submit issues and pull requests. If you are contributing code, pay attention to "devel" branch as it's the place where feature branches get merged. We will merge thing to master branch periodically in sync with releases on PyPI.