We describe here how the analysis pipeline can be installed. The pipeline must be available from a git repository and follows the expected organization (see :ref:`overview-source-tree`). An example of installation is provided with the |geniacdemo|_ pipeline.
Installation requires |cmake|_ (version 3.0 or above) and consists of the following sequence.
# Create the geniac conda environment (use conda >= 4.12.0) export GENIAC_CONDA="https://raw.githubusercontent.com/bioinfo-pf-curie/geniac/release/environment.yml" wget ${GENIAC_CONDA} conda env create -f environment.yml conda activate geniac # Prepare the working directory for the use case export WORK_DIR="${HOME}/tmp/myPipeline" export SRC_DIR="${WORK_DIR}/src" export INSTALL_DIR="${WORK_DIR}/install" export BUILD_DIR="${WORK_DIR}/build" export GIT_URL="https://github.com/bioinfo-pf-curie/geniac-demo.git" mkdir -p ${INSTALL_DIR} ${BUILD_DIR} # clone the repository # the option --recursive is needed if you use geniac as a submodule # the option --remote-submodules will pull the last geniac version # using the release branch from https://github.com/bioinfo-pf-curie/geniac git clone --remote-submodules --recursive ${GIT_URL} ${SRC_DIR} cd ${BUILD_DIR} # configure the pipeline cmake ${SRC_DIR}/geniac -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR} # build the files needed by the pipeline make # install the pipeline make install
Note
If you use a Linux distribution which is binary-compatible with the Red Hat Enterprise Linux (RHEL) operating system source code (such as |almalinux|_, |centos|_ or |rockylinux|_), |cmake|_ version 3 is available as cmake3
executable. You can alias cmake3
as cmake
in your .bashrc
if needed.
Different options can be passed to |cmake|_ for the configuration step. They are described in the following section.
The configure options for the analysis pipeline start with the prefix ap and are in lower case. Options in upper case are |cmake|_ language variables.
This is the |cmake|_ variable to set the install directory.
annotations
with the given target will be created in the install directory. This is useful if the annotations are already available.py311_23.5.1-0
. See also options :ref:`install-ap_linux_distro` and :ref:`install-ap_docker_registry`.environment.yml
file from the source code if set to ON. The environment.yml
generated by geniac will be discarded. Indeed, it can be impossible to automatically generate the conda recipe. We may have no choice but to write the environment.yml
manually.almalinux:8.8
. See also options :ref:`install-ap_conda_release` and :ref:`install-ap_docker_registry`.containers/singularity
with the given target will be created in the install directory. This is useful if the singularity containers are already available.Warning
Options ap_install_singularity_images
and ap_singularity_image_path
are exclusive.
Warning
Options ap_install_singularity_images
and ap_singularity_image_path
are exclusive.
All the options can be set on the command line interface. If your want to install the pipeline in $HOME/myPipeline
directory and build and install the singularity images, run:
cd ${BUILD_DIR} cmake ${SRC_DIR}/geniac -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR} -Dap_install_singularity_images=ON
Tip
To have all the available options and help, run cmake -LAH ${SRC_DIR}/geniac
in the build
directory. The different options are displayed in the Cache values section.
The file geniac/install/cmake-init-default.cmake
provides a script to set all the available variables during the configuration step. We recommend that you copy this file into geniac/install/cmake-init.cmake
, edit it and set the different variables to match your configuration. Then you can configure the project as follows:
cp ${SRC_DIR}/geniac/install/cmake-init-default.cmake ${SRC_DIR}/geniac/install/cmake-init.cmake # edit the file ${SRC_DIR}/geniac/install/cmake-init.cmake to set your options cd ${BUILD_DIR} cmake -C ${SRC_DIR}/geniac/install/cmake-init.cmake ${SRC_DIR}/geniac
Note
If you use a Linux distribution which is binary-compatible with the Red Hat Enterprise Linux (RHEL) operating system source code (such as |almalinux|_, |centos|_ or |rockylinux|_), |cmake|_ version 3 is available as cmake3
executable, the syntax is cmake3 ${SRC_DIR}/geniac -C ${SRC_DIR}/geniac/install/cmake-init.cmake
Warning
In order to build singularity images, root credentials are required:
- either type make if you have fakeroot singularity credentials
- or sudo make if you have sudo privileges
- then make install
In order to build the containers, you can either pass the required options during the configure stage (see :ref:`install-configure`) or use custom targets (see :ref:`install-target-containers`).
The custom targets will be available once you have performed the configuration step with cmake
.
Assume you are in the build
directory. The following custom target allows you to build only the configuration files:
make build_config_files
The configuration files will be generated in build/workDir/results/conf
.
Assume you are in the build
directory. The following custom targets allows you to build recipes and containers even if you did not ask for them during the configure stage:
make build_singularity_recipes
make build_singularity_images
make build_docker_recipes
make build_docker_images
make build_podman_recipes
make build_podman_images
For singularity:
- Recipes will be generated in
build/workDir/results/singularity/deffiles
. - Images will be generated in
build/workDir/results/singularity/images
.
For docker:
- Recipes will be generated in
build/workDir/results/docker/Dockerfiles
. - Images will be created in the registry.
In order to make the deployment and testing of the pipeline easier, several custom targets are provided such that you only need to type one of the following command to install the pipeline:
make test_conda
make test_docker
make test_multiconda
make test_multipath
make test_path
make test_singularity
make test_standard
Assuming that you configured the build directory such that CMAKE_INSTALL_PREFIX=${INSTALL_DIR}
, typing make test_conda
is similar to:
make make install cd ${SRC_DIR}/pipeline nextflow -c conf/test.config run main.nf -profile conda
If you want to add the :ref:`run-profile-cluster` profile, just type the following:
make test_conda_cluster
make test_docker_cluster
make test_multiconda_cluster
make test_multipath_cluster
make test_path_cluster
make test_singularity_cluster
make test_standard_cluster
Note
For these custom targets to be available, test data and conf/test.config
file have to be provided in the repository.
├── annotations ├── containers │ └── singularity ├── multipath │ ├── alpine │ │ └── bin │ ├── fastqc │ │ └── bin │ ├── helloWorld │ │ └── bin │ ├── rmarkdown │ │ └── bin │ └── trickySoftware │ └── bin ├── path │ ├── bin └── pipeline ├── assets ├── bin │ └── fromSource ├── conf ├── docs ├── env ├── modules │ └── helloWorld ├── recipes │ ├── conda │ ├── dependencies │ ├── docker │ └── singularity └── test └── data
Important
You must have |conda|_ installed locally, if not, proceed as follows:
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh bash Miniconda3-latest-Linux-x86_64.sh
Then, edit your file .bashrc
and add $HOME/miniconda3/bin
(or the install directory you set) in your PATH.
export WORK_DIR="${HOME}/tmp/myPipeline" export SRC_DIR="${WORK_DIR}/src" export INSTALL_DIR="${WORK_DIR}/install" export BUILD_DIR="${WORK_DIR}/build" export GIT_URL="https://github.com/bioinfo-pf-curie/geniac-demo.git" mkdir -p ${INSTALL_DIR} ${BUILD_DIR} # clone the repository # the option --recursive is needed if you use geniac as a submodule # the option --remote-submodules will pull the last geniac version # using the release branch from https://github.com/bioinfo-pf-curie/geniac git clone --remote-submodules --recursive ${GIT_URL} ${SRC_DIR} cd ${BUILD_DIR} cmake ${SRC_DIR}/geniac -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR} make make install cd ${INSTALL_DIR}/pipeline nextflow -c conf/test.config run main.nf -profile multiconda
Note
If you use both the :ref:`run-profile-conda` and :ref:`run-profile-cluster` profile, check that your master job that launches nextflow has been submitted with enough memory, otherwise the creation of the conda environment may fail.
export WORK_DIR="${HOME}/tmp/myPipeline" export SRC_DIR="${WORK_DIR}/src" export INSTALL_DIR="${WORK_DIR}/install" export BUILD_DIR="${WORK_DIR}/build" export GIT_URL="https://github.com/bioinfo-pf-curie/geniac-demo.git" mkdir -p ${INSTALL_DIR} ${BUILD_DIR} # clone the repository # the option --recursive is needed if you use geniac as a submodule # the option --remote-submodules will pull the last geniac version # using the release branch from https://github.com/bioinfo-pf-curie/geniac git clone --remote-submodules --recursive ${GIT_URL} ${SRC_DIR} cd ${BUILD_DIR} cmake ${SRC_DIR}/geniac -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR} -Dap_install_singularity_images=ON ### /!\ with sudo, the singularity and nextflow commands must be ### /!\ in the secure_path option declared in the file /etc/sudoers ### build the files needed by the pipeline sudo make ### change file owner/group to the current user sudo chown -R $(id -gn):$(id -gn) ${BUILD_DIR} make install cd ${INSTALL_DIR}/pipeline nextflow -c conf/test.config run main.nf -profile singularity
Note
Whenever you explicitely set an option on the command line such as -Dap_install_singularity_images=ON
, and then you want to reconfigure your build directory by specifying only another option on the command line such as -DCMAKE_INSTALL_PREFIX=${HOME}/myPipelineNewDir
, the ap_install_singularity_images
will remain ON
unless you specify -Dap_install_singularity_images=OFF
.
As geniac automatically generates the recipes of the containers, they are not available in the git repository. However, they can be easily retrieved in several ways. We provide here an example.
First, let's assume you want the Singularity definition files. Then, pass the option -Dap_install_singularity_recipes=ON
to cmake during the configuration stage:
export WORK_DIR="${HOME}/tmp/myPipeline" export SRC_DIR="${WORK_DIR}/src" export INSTALL_DIR="${WORK_DIR}/install" export BUILD_DIR="${WORK_DIR}/build" export GIT_URL="https://github.com/bioinfo-pf-curie/geniac-demo.git" mkdir -p ${INSTALL_DIR} ${BUILD_DIR} # clone the repository # the option --recursive is needed if you use geniac as a submodule # the option --remote-submodules will pull the last geniac version # using the release branch from https://github.com/bioinfo-pf-curie/geniac git clone --remote-submodules --recursive ${GIT_URL} ${SRC_DIR} cd ${BUILD_DIR} cmake ${SRC_DIR}/geniac -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR} -Dap_install_singularity_recipes=ON make install ### the Singularity definition files are available here: ls ${INSTALL_DIR}/pipeline/recipes/singularity
Second, let's assume you want the Dockerfiles. Then, use a custom target (see :ref:`install-target-containers`) as follows:
make build_docker_recipes make install ### the Dockerfiles are available here: ls ${INSTALL_DIR}/pipeline/recipes/docker
Obviously, if you want both the Singularity definition files and Dockerfiles you can simply do the following during the configuration stage:
cmake ${SRC_DIR}/geniac -DCMAKE_INSTALL_PREFIX=${INSTALL_DIR} -Dap_install_singularity_recipes=ON -Dap_install_docker_recipes=ON