oswitch - virtual environments for flexible and reproducible bioinformatics analyses
More and more bioinformatics software are being "containerised" with docker, making them installable on personal computers and compute clusters alike with a single command in a reproducible manner. However, using containerised software can still be a challenge as the containers are effectively a different operating system altogether. oswitch enhances usability of docker containers by:
- Making available the local file-system inside the container.
- Ensuring you are the same user inside the container as you are on the host.
- Running your login shell, including sourcing your config files when the container is booted.
- Switching to the directory that you were in on the host when invoking oswitch.
The net effect is similar to entering "virtual environment" on the host system containing specific versions of software of interest.
mymacbook:~/2015-02-01-myproject> abyss-pe k=25 reads.fastq.gz zsh: command not found: abyss-pe # List available images. mymacbook:~/2015-02-01-myproject> oswitch -l yeban/biolinux:8 ubuntu:14.04 ontouchstart/texlive-full ipython/ipython hlapp/rpopgen bioconductor/release_sequencing # Enter the continaer and run commands interactively. mymacbook:~/2015-02-01-myproject> oswitch biolinux ###### You are now running: biolinux in container biolinux-7187. ###### biolinux-7187:~/2015-02-01-myproject> abyss-pe k=25 reads.fastq.gz [... just works on your files where they are...] biolinux-7187:~/2015-02-01-myproject> exit mymacbook:~/2015-02-01-myproject> [... output is where you expect it to be ...] # Use a container non-interactively. pixel:~/test/ $ oswitch yeban/biolinux blastp -remote -query mygene.fa -db nr > mygene_blastp_nr.txt
oswitch first requires a working docker installation.
Download and run oswitch
Install oswitch using RubyGems
If you have Ruby 2.0 or higher (available by default on Mac and through package managers on Linux), you can install oswitch systemwide by running the command below:
$ sudo gem install oswitch
If you are on Mac, you can also install oswitch using homebrew:
brew tap homebrew/science brew install oswitch
This will install oswitch systemwide or only for your user depending on how homebrew is setup.
Currently this installs version 0.2.6 of oswitch.
deb package of
oswitch is available in BioLinux repository for Trusty,
Vivid and Jessie.
$ sudo add-apt-repository ppa:nebc/bio-linux $ sudo apt-get update $ sudo apt-get install oswitch
This will install oswitch systemwide. Please note that the package may not be up to date as the first two options.
Currently this installs version 0.2.6 of oswitch.
- Volume mounting on Mac OS hosts is imperfect.
- SELinux must be disabled on CentOS hosts for mounting volumes to work (check the SELinux documentation to see the implications of doing this).
- We have tested oswitch on Debian, Ubuntu, CentOS based docker images on the
- Mac OS X Yosemite, El Captain
- Ubuntu 14.04.1
- CentOS 7
Q. Directories mounted within container on Mac host are empty.
The problem is, on Mac
docker-machine is the real host, not OS X.
oswitch can mount only what's available to it from
/Applications directory as an example. Run
docker-machine ssh default ls /Applications and you will find it empty as well. The workaround is to correctly mount the directories you want inside
docker-machine stop default VBoxManage sharedfolder remove default --name Applications VBoxManage sharedfolder add default --name Applications --hostpath /Applications --automount docker-machine start default docker-machine ssh default "sudo mkdir -p /Applications && sudo mount -t vboxsf -o uid=1000,gid=50 Applications /Applications"
Q. cwd is empty in the container
This means the said directory was not mounted by oswitch, or was incorrectly
mounted. On Linux host, directories that can conflict with paths within
container are not mounted. On Mac,
docker-machine can get in the way.
- the directory in question
- the operating system you are running
Q. oswitch does not work with my docker image
Q. How does all this work?
We create a new image on the fly that inherits from the given image. While creating the new image we execute a shell script that installs packages required for oswitch to work and creates a user in the image (almost) identical to that on the host.
Q. How can I connect to an existing container?
In another shell, use
docker ps to see which containers are already running. Copy the identifier from the
CONTAINER ID (column this looks something like
37e4e6ada6a4), and use it to run
docker attach 37e4e6ada6a4 (replace with your container's id). This will create a new ssh connection to your existing container.
$ git clone https://github.com/yeban/oswitch $ cd oswitch $ gem install bundler && bundle $ bundle exec bin/oswitch biolinux
Contributors & Funding
- Anurag Priyam - firstname.lastname@example.org | @yeban
- Bruno Vieira (@bmpvieira)
- Saurabh Kumar
- Richard Nichols - http://www.sbcs.qmul.ac.uk/staff/richardnichols.html | @qmwugbt112
- Yannick Wurm - http://wurmlab.github.io | @yannick__