In this section we will install the Cromwell Workflow Management system and configure it, so it can use the local scratch device on the compute nodes.
(these installations are done in a centos 8
enviornment)
dnf config-manager --add-repo https://www.scala-sbt.org/sbt-rpm.repo
dnf -y install sbt
mkdir cromwell
b. Now, we set the owner of that directory to the cromwell user and group and change the permissions in order to allow access to all users.
chmod 775 -R cromwell
chown -R cromwell:cromwell cromwell
su - cromwell
cd cromwell
git clone https://github.com/broadinstitute/cromwell.git
e. This guide was tested and validated with version 52
of cromwell, so we will checkout that version.
cd cromwell
git checkout 52
//import java.util.UUID
val tempPath: String = {
val uuid = UUID.randomUUID().toString
val hash = uuid.substring(0, uuid.indexOf('-'))
callRoot.resolve(s"tmp.$hash").pathAsString
}
def tempPath: String = "/genomics_local"
f. Open the file backend/src/main/scala/cromwell/backend/standard/StandardAsyncExecutionActor.scala
for editing
|export _JAVA_OPTIONS=-Djava.io.tmpdir="$$tmpDir"
|export TMPDIR="$$tmpDir"
|mkdir -p $$tmpDir/tmp.$$$$
|export _JAVA_OPTIONS=-Djava.io.tmpdir="$$tmpDir/tmp.$$$$"
|export TMPDIR="$$tmpDir/tmp.$$$$
sed -i "s/^\(import\ java.util.UUID\)/\/\/\1/" \
backend/src/main/scala/cromwell/backend/RuntimeEnvironment.scala
sed -i '23,27d' \
backend/src/main/scala/cromwell/backend/RuntimeEnvironment.scala
sed -i '23i \ \ \ \ \ \ def tempPath: String = \"/genomics_local\"' \
backend/src/main/scala/cromwell/backend/RuntimeEnvironment.scala
sed 's/\(\s*|export _JAVA_OPTIONS.*\)\"/\ \ \ \ \ \ \ \ |mkdir -p \$\$tmpDir\/tmp\.\$\$\$\$\n\1\"/' \
backend/src/main/scala/cromwell/backend/standard/StandardAsyncExecutionActor.scala
sed 's/\(\s*|export _JAVA_OPTIONS.*tmpDir\)\"/\1\/tmp\.\$\$\$\$\"/' \
backend/src/main/scala/cromwell/backend/standard/StandardAsyncExecutionActor.scala
sed 's/\(\s*|export TMPDIR=.*tmpDir\)\"/\1\/tmp\.\$\$\$\$\"/' \
backend/src/main/scala/cromwell/backend/standard/StandardAsyncExecutionActor.scala
sbt clean
sbt assembly
cp server/target/scala-2.12/cromwell-52-*-SNAP.jar \
cromwell/cromwell-52-fix.jar
In this section we will configure the Cromwell execution environment.We will use the default configuration file as a starting point and change it to reflect our needs.
*• Configure MariaDB as the database Cromwell should use
*• Add the SLURM backends
, so Cromwell can schedule job using SLURM
wget https://raw.githubusercontent.com/broadinstitute/cromwell/52_hotfix/core/\
src/main/resources/reference.conf cromwell/
default = "Local"
providers {
Local {
actor-factory = "cromwell.backend.impl.sfs.config.ConfigBackendLifecycleActorFactory"
config {
include required(classpath("reference_local_provider_config.inc.conf"))
e. Now add the following text after line 479, i.e. after the line reading default ="SLURM"
. Ensure that the lines that show line-breaks in this document are, in fact, single lines in reference.conf
providers {
SLURM {
#Modifying temp directory to write to local disk
temporary-directory = "$(/genomics_local/)"
actor-factory = "cromwell.backend.impl.sfs.config.ConfigBackendLifecycleActorFactory"
config {
root = "cromwell-slurm-exec"
runtime-attributes = """
Int runtime_minutes = 600
Int cpu = 2
Int memory_mb = 1024
String queue = "all"
String? docker
"""
submit = """
sbatch -J ${job_name} -D ${cwd} -o ${out} -e ${err} -t ${runtime_minutes} -p ${queue} ${"-c " +
cpu} --mem ${memory_mb} --wrap "/bin/bash ${script}"
"""
kill = "scancel ${job_id}"
check-alive = "squeue -j ${job_id}"
job-id-regex = "Submitted batch job (\\d+).*"
}
}
SLURM-BWA {
temporary-directory = "$(/genomics_local/)"
actor-factory = "cromwell.backend.impl.sfs.config.ConfigBackendLifecycleActorFactory"
config {
root = "cromwell-slurm-exec"
runtime-attributes = """
Int runtime_minutes = 600
Int cpu = 2
Int memory_mb = 1024
String queue = "bwa"
String? docker
"""
submit = """