Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Nextflow config update (autoMounts, envWhitelist) #18

Merged
merged 9 commits into from
Jun 28, 2021
7 changes: 6 additions & 1 deletion doc/source/configuration.rst
Original file line number Diff line number Diff line change
Expand Up @@ -125,9 +125,12 @@ The Nextflow configuration file is used to configure various settings for the *k
singularity {
enabled = true
cacheDir = ''
autoMounts = true
envWhitelist = ''
Cecilia-Sensalari marked this conversation as resolved.
Show resolved Hide resolved
}
docker {
enabled = false
envWhitelist = ''
}

executor.name = ''
Expand All @@ -149,7 +152,9 @@ The Nextflow configuration file is used to configure various settings for the *k
* The **singularity** and **docker** scopes configure container type usage and execution:

* **enable** enables or disables the use of the respective container
* **cacheDir** the directory where remote the Singularity image from Docker Hub is stored. When using a computing cluster it must be a shared folder accessible to all computing nodes.
* **cacheDir** (only for Singularity) the directory where remote the Singularity image from Docker Hub is stored. When using a computing cluster it must be a shared folder accessible to all computing nodes.
* **autoMounts** (only for Singularity) automatically mounts host paths in the executed container and allows the user to run the pipeline from any directory in a cluster [Default: false]. It requires the `*user bind control* <https://sylabs.io/guides/3.7/admin-guide/configfiles.html?highlight=user%20bind%20control#bind-mount-management`>__ feature in Singularity installation, which is active by default.
* **envWhitelist** accepts a comma separated list of environment variable names to be included in the container environment. If working on a SGE cluster, please include the ``NSLOTS`` variable (defining the number of cores/processes/slots), since it is used in the Nextflow pipeline.

* The **executor.name** setting defines the system type or HPC scheduler to be used (e.g. ``sge``, ``local``)
* The **process** scope defines the configuration for the processes of the *ksrates* pipeline:
Expand Down
3 changes: 3 additions & 0 deletions doc/source/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,9 @@ The machine where *ksrates* will be executed (either local computer or remote co
For Linux installation we suggest to follow the *Install from Source* `section <https://sylabs.io/guides/3.7/admin-guide/installation.html#before-you-begin>`__ (*Install Dependencies*, *Install Go*, *Download Singularity from a release* and *Compile Singularity*).
For up-to-date and version-specific instructions, please refer to this `page <https://sylabs.io/docs/>`__.

.. note::
To allow users to run the pipeline from any directory in a cluster (i.e. not necessarily from their home directory), the `*user bind control* <https://sylabs.io/guides/3.7/admin-guide/configfiles.html?highlight=user%20bind%20control#bind-mount-management>__` feature needs to be left active during Singularity installation [Default: "YES"].

When using the *ksrates* Nextflow pipeline, the only other dependency that must be installed is Nextflow (for more information see its official installation `page <https://www.nextflow.io/docs/latest/getstarted.html#requirements>`__).

* First install Java 8 or later. ``default-djk`` works as well::
Expand Down
6 changes: 4 additions & 2 deletions doc/source/template_docker_nextflow.config
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,10 @@


// CONTAINER SETTINGS:
docker.enabled = true

docker {
enabled = true
// envWhitelist = 'NSLOTS' // to define the number of cores when running the container on a SGE cluster
}

// CLUSTER SETTINGS:
executor.name = '' // set cluster type, e.g. sge
Expand Down
3 changes: 2 additions & 1 deletion doc/source/template_singularity_nextflow.config
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,8 @@
singularity {
enabled = true
cacheDir = '' // set a directory where to download the container file
// envWhitelist = 'NSLOTS' // to be used when running the Singularity container on a SGE cluster
autoMounts = true // to automatically mount host paths in the executed container
// envWhitelist = 'NSLOTS' // to define the number of cores when running the container on a SGE cluster
}

// CLUSTER SETTINGS:
Expand Down
9 changes: 6 additions & 3 deletions example/custom_nextflow.config
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,14 @@
// CONTAINER SETTINGS:
singularity {
enabled = true
// envWhitelist = 'NSLOTS' // to be used when running the Singularity container on a SGE cluster
// cacheDir = '' // to set a directory where to download the container file
autoMounts = true // to automatically mount host paths in the executed container
// envWhitelist = 'NSLOTS' // to define the number of cores when running the container on a SGE cluster
}
// docker.enabled = true

//docker {
// enabled = true
// // envWhitelist = 'NSLOTS' // to define the number of cores when running the container on a SGE cluster
//}

// CLUSTER SETTINGS:
executor.name = '' // e.g. sge
Expand Down
9 changes: 6 additions & 3 deletions test/custom_nextflow.config
Original file line number Diff line number Diff line change
Expand Up @@ -5,11 +5,14 @@
// CONTAINER SETTINGS:
singularity {
enabled = true
// envWhitelist = 'NSLOTS' // to be used when running the Singularity container on a SGE cluster
// cacheDir = '' // to set a directory where to download the container file
autoMounts = true // to automatically mount host paths in the executed container
// envWhitelist = 'NSLOTS' // to define the number of cores when running the container on a SGE cluster
}
// docker.enabled = true

//docker {
// enabled = true
// // envWhitelist = 'NSLOTS' // to define the number of cores when running the container on a SGE cluster
//}

// CLUSTER SETTINGS:
executor.name = '' // e.g. sge
Expand Down