Skip to content

biobakery_advanced

Jump to bottom
ljmciver edited this page May 23, 2022 · 9 revisions

bioBakery: Advanced configuration

In addition to the basic functions, bioBakery allows users to perform advanced functions including building bioBakery for different platforms and also installing individual bioBakery tools in an existing environment.

Table of Contents

1. Install bioBakery (Manually)

If you would like to install bioBakery in a location other than the default install folder, follow these steps to install bioBakery manually. Before you begin installing bioBakery, please install VirtualBox and Vagrant.

  1. Open a terminal
  2. Move to the directory you would like to install bioBakery a. $ cd $BIOBAKERY_FOLDER (replace $BIOBAKERY_FOLDER with the location to install bioBakery)
  3. Create a bioBakery Vagrantfile a. $ vagrant init biobakery/biobakery
  4. Install and start bioBakery a. $ vagrant up --provider virtualbox b. It is important to note when starting bioBakery, for the first time and future starts, this command must be run from the same folder as the init command was run. Vagrant uses the Vagrantfile in the current working directory to start the guest OS.

Please note the first time you start bioBakery it will download and extract the box. This might take about 20 minutes (depending on your download rate) as the box is around 6 GB. When you start bioBakery in the future, it should take much less time (~30 seconds).

2. Build bioBakery (Local Install)

Before you begin building bioBakery locally, please install VirtualBox and Vagrant.

2.1. Build bioBakery with a GUI

  • Clone the biobakery repository

    $ git clone https://github.com/biobakery/biobakery

  • Change directory to the biobakery-gui directory

    $ cd biobakery/vagrant/biobakery-gui

  • Start the biobakery VM (username and password are both "vagrant")

    $ vagrant up --provider virtualbox

This process may take some time (~1.5 hours) depending on your machine and network connectivity.

Once the process is complete, a window will automatically launch displaying bioBakery as shown below:

image

  • Stop the biobakery VM

    $ vagrant halt

2.2. Build bioBakery without a GUI

To install bioBakery without a GUI (Graphic User Interface), please follow the instructions below.

  • Clone the biobakery repository

    $ git clone https://github.com/biobakery/biobakery

  • Change directory to the biobakery-nogui directory

    $ cd biobakery/vagrant/biobakery-nogui

  • Start the biobakery VM (username and password are both "vagrant")

    $ vagrant up --provider virtualbox

2.3. Running and Stopping bioBakery

Once bioBakery is installed, you may choose to run it on your computer at your convenience.

  • To start bioBakery, run the following command from the terminal (the current working directory needs to be the folder where the bioBakery vagrantfile exists).

    $ vagrant up --provider virtualbox

  • Once launched, you may shutdown bioBakery by running the following command from the terminal.

    $ vagrant halt

3. Shared folders

By default, the bioBakery /vagrant folder becomes the shared folder between the bioBakery VM (guest OS) and your local machine (host OS).

For more information about this feature and others on Virtual Box, please refer to the documentation

To see the list of shared folders with bioBakery follow the instructions below:

  • Open VirtualBox. You will see the bioBakery virtual image listed on the left pane. The name may vary. (The format is that it starts from the folder name that it is installed in. So in our case, the name will be vagrant_).

vbox home.png

  • Click on the name of the bioBakery virtual image: vagrant_*, and click on Settings on the top pane.

image

  • Click on the Shared folders icon

shared folder local.png

  • The list will show the folder shared from your local computer (red), and the folder that it is on the bioBakery (blue).

folder listed now.png

  • Press OK.

3.1. Adding shared folders

You may choose to share more folders with bioBakery for easy access of the existing files (alternatively, you may just choose to copy the files over to the existing shared folder). To add more shared folders to bioBakery please follow the instructions below:

* Open VirtualBox

You will see the bioBakery virtual image listed on the left pane. The name may vary.(The format is that it starts from the folder name that it is installed in. So in our case, the name will be vagrant_)*.

image

* Click on the name of the bioBakery virtual image: vagrant_*, and click on Settings on the top pane.

image

  • Click on the Shared folders icon

image

  • Click on the Add folder icon on the right pane (folder icon with the + sign).

add folder highlighted.png

  • Select the folder you would like to share with bioBakery, by clicking on the icon at the end of the field and selecting the folder as shown below:

image

  • Select the appropriate folder. Make sure the Auto-mount checkbox is checked. Now press OK.

image

  • Once added, the folder will show up in the list of shared folders for bioBakery:

folder listed now (1).png

Now you should have access to all the files in the folder you added every time you launch bioBakery. The added folder can be found under Folders>>Computer>>vagrant or Folders>>Computer>>media.

3.2. Deleting shared folders

* Open VirtualBox

You will see the bioBakery virtual image listed on the left pane. The name may vary. (The format is that it starts from the folder name that it is installed in. So in our case, the name will be vagrant_...).

image

  • Click on the name of the bioBakery virtual image: vagrant_…, and click on Settings on the top pane.

image

  • Click on the Shared folders icon

image

  • Select the folder you would like to delete from being shared with bioBakery, and click on the Remove folder icon on the right pane.

remove folder.png

  • Press OK.

4. bioBakery Memory Allocation

bioBakery is initially allocated 8 GB of RAM. If you would like to increase or decrease this amount, follow these instructions. Please note, it is not recommended to run bioBakery with less than 4 GB of RAM as this will not be enough memory to run some of the demos.

The memory allocation can be changed with two different methods. Please select one of the following sets of instructions.

To change memory through VirtualBox:

  1. Start VirtualBox
  2. Select the bioBakery VM
  3. Click on "System"
  4. Change the Base Memory
  5. Click "OK"

To change memory by editing the Vagrantfile:

  1. Find the bioBakery Vagrantfile (it is located at $HOME/biobakery/Vagrantfile if you ran the auto-install)

  2. Open this file with a text editor

  3. Remove the comment characters (ie "#" characters at the beginning of the lines) from the section in the file that looks as follows, replace the memory value with the value you selected, and save the file :

    # config.vm.provider "virtualbox" do |vb|
#   # Display the VirtualBox GUI when booting the machine
#   vb.gui = true
#
#   # Customize the amount of memory on the VM:
#   vb.memory = "1024"
# end

Now the next time you start up bioBakery, it will have the new memory allocation.

5. Build bioBakery (Google Cloud)

  1. Create a Google Cloud Account (or login to an existing account)

    a. As of June 2016, the time to setup and install bioBakery (about 2 minutes hands on and about 65 minutes for the automated build) is estimated to cost about $0.20. Running this instance for 8 hours is estimated to cost about $0.60. For current costs, please refer to the Google Cloud Cost Calculator .

  2. Create a new instance

    a. From the left-hand menu select Compute Engine -> Images b. From the Images page, select Create Image c. On the Create an Image page, select the following and then click Create:

    i.  Machine type: n1-standard-2
    A.  This machine has 2 CPUs and 7.5 GB of RAM. The machine
        selected is the minimal configuration we suggest.
ii. Boot disk: Ubuntu 16.04 (the Linux distribution and version
    the build script was tested against)
iii. Disk size: 32 GB

  3. When your instance is ready, ssh to the instance by clicking on the "SSH" selection

    a. A terminal window will appear containing a command prompt b. If there are any issues connecting to your instance with ssh, please refer to the Google Cloud documentation about ssh access .

  4. Build bioBakery (all commands should be executed in the cloud instance terminal)

    a. Clone the bioBakery repository

    i.  `$ sudo apt-get update && sudo apt-get install git`
ii. `$ git clone https://github.com/biobakery/biobakery`

    b. Run the script to build bioBakery

    i.  `$ cd biobakery/google_cloud/`
ii. `$ bash -xe build_biobakery.sh`

  5. Follow the instructions to install bioBakery dependencies that require licences

  6. Optionally, follow the instructions to test the bioBakery install

Please stop your bioBakery instance when it is not in use. As of June 2016, if an instance is not in use you will only be charged for the disk space and not the instance.

Refer to the following Google Cloud documentation about how to access files from your cloud instance with your file browser .

6. Installing individual bioBakery tools

Tools can be installed with Conda or Docker. If you have issues installing with Conda and you have root permissions or a container engine is available, we recommend using Docker.

6.1 Install with Conda

  1. Install Conda.

  2. Set up your channels so conda can find the bioBakery packages and their dependencies. bioBakery conda packages rely on bioConda for many of their dependencies. :

    $ conda config --add channels defaults
$ conda config --add channels bioconda
$ conda config --add channels conda-forge
$ conda config --add channels biobakery

  3. Create a new conda environment and install a bioBakery tool. :

    $ conda create -n biobakery_env
$ conda activate biobakery_env
$ conda install -c biobakery humann2

  4. Test the tool by running a demo (downloading the demo file to a local folder named demo). :

    $ mkdir demo
$ wget --directory-prefix demo https://github.com/biobakery/biobakery/raw/master/test_suite/biobakery_tests/data/humann/input/demo.fastq
$ humann --input demo/demo.fastq --output demo/output

  5. Exit the conda environment. :

    $ conda deactivate

For the complete list of bioBakery conda packages, visit the bioBakery repo in Anaconda Cloud.

6.2 Install with Docker

  1. Install Docker or another container engine.
  2. Create and run a container which contains the tool (or tools) of your choice.

For example, lets run the HUMAnN demo in a docker container.

  1. Download the demo file to a local folder and name that folder "demo". :

    $ mkdir demo
$ wget --directory-prefix demo https://github.com/biobakery/biobakery/raw/master/test_suite/biobakery_tests/data/humann/input/demo.fastq

  2. Download the image with HUMAnN installed and create a container from the image. This command will also mount the local folder "~/demo" to the location "/home/demo" in the container. Please note, depending on your docker install you might not need to include "sudo" in the docker commands. :

    $ sudo docker run -dt -v ~/demo:/home/demo --name humann biobakery/humann bash

  3. Access a bash terminal in the container and run humann on a demo file. Alternatively you could run the command directly in the container without first accessing a prompt by running $ sudo docker exec humann humann --input /home/demo/demo.fastq --output /home/demo/output . :

    $ sudo docker exec -it humann bash
$ humann --input /home/demo/demo.fastq --output /home/demo/output

  4. Exit the container. Stop and delete the container if you would like.:

    $ exit
$ sudo docker stop humann
$ sudo docker rm humann

For a complete list of Docker images, see the bioBakery Docker Hub repositories.

7. Test bioBakery

Now that bioBakery and its dependencies are installed, you can optionally test the full bioBakery tool suite to verify it is functioning as expected on your platform. The tests will ensure all dependencies are installed and all features are operational.

$ biobakery_tests --tool all --mode test

Running tests on the full tool suite should take about 45 minutes.

Alternatively you can test individual tools by replacing all with the tool name.

Tutorials Home

Microbial community profiling
Downstream analysis and statistics
Infrastructure and utilities
Clone this wiki locally