Two patterns describing how to run a process when another process fails

README.md

@@ -42,3 +42,4 @@ A curated collections of Nextflow implementation patterns
 * [Optional input](docs/optional-input.adoc)
 * [Optional output](docs/optional-output.adoc)
 * [Process when empty](docs/process-when-empty.adoc)
+* [Process when another process fails](docs/process-when-another-process-fails.adoc)

docs/index.adoc

@@ -60,5 +60,6 @@ include::feedback-loop.adoc[]
 include::optional-input.adoc[]
 include::optional-output.adoc[]
 include::process-when-empty.adoc[]
+include::process-when-another-process-fails.adoc[]
 :leveloffset: -2
 

docs/process-when-another-process-fails.adoc

@@ -0,0 +1,104 @@
+= Execute when another process fails 
+
+== Problem 
+
+You need to execute a process if another process fails. 
+
+== Solution 
+
+Use the https://www.nextflow.io/docs/latest/process.html#output-env[env] qualifier
+to capture the exit status of the process. `set +e` prevents the script from exiting
+immediately if the command exits with a non-zero status.
+
+== Example 
+
+[source,nextflow,linenums,options="nowrap"]
+----
+process foo {
+
+  output:
+  env PROCESS_FOO_FAILED into process_foo_failed
+
+  script:
+  """
+  set +e
+  PROCESS_FOO_FAILED=0
+  trap 'PROCESS_FOO_FAILED=1' ERR
+  cat thisfiledoesnotexist.txt
+  """
+}  
+
+process bar {
+  input: 
+  val x from process_foo_failed
+
+  when:
+  x=="1"
+
+  script:
+  """
+  echo $x
+  """
+}
+----
+
+== Run it 
+
+Use the the following command to execute the example:
+
+```
+nextflow run patterns/process-when-another-process-fails.nf
+```
+
+== Dynamic computing resources
+
+Use `set +e` only for the last task attempt if the process `foo` implements
+dynamic directives to control the computing resources requested in case of 
+a process failure.
+
+== Code 
+
+[source,nextflow,linenums,options="nowrap"]
+----
+process foo {
+
+  memory { 1.GB * task.attempt }
+  maxRetries 2
+  errorStrategy 'retry'
+
+  output:
+  env PROCESS_FOO_FAILED into process_foo_failed
+
+  script:
+  """
+  if [ ${task.attempt} = ${task.maxRetries + 1 } ]; then
+    set +e
+  fi
+
+  PROCESS_FOO_FAILED=0
+  trap 'PROCESS_FOO_FAILED=1' ERR
+  cat thisfiledoesnotexist.txt
+  """
+}  
+
+process bar {
+  input: 
+  val x from process_foo_failed
+
+  when:
+  x=="1"
+
+  script:
+  """
+  echo $x
+  """
+}
+----
+
+== Run it 
+
+Use the the following command to execute the example:
+
+```
+nextflow run patterns/process-when-another-process-fails2.nf
+```

process-when-another-process-fails.nf

@@ -0,0 +1,54 @@
+#!/usr/bin/env nextflow
+
+/*
+ * Copyright (c) 2020, Centre for Genomic Regulation (CRG).
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ * 
+ * The above copyright notice and this permission notice shall be included in all
+ * copies or substantial portions of the Software.
+ * 
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+ /*
+  * author Davide Albanese <davide.albanese@gmail.com> 
+  */
+
+process foo {
+
+  output:
+  env PROCESS_FOO_FAILED into process_foo_failed
+
+  script:
+  """
+  set +e
+  PROCESS_FOO_FAILED=0
+  trap 'PROCESS_FOO_FAILED=1' ERR
+  cat thisfiledoesnotexist.txt
+  """
+}  
+
+process bar {
+  input: 
+  val x from process_foo_failed
+
+  when:
+  x=="1"
+
+  script:
+  """
+  echo $x
+  """
+}

process-when-another-process-fails2.nf

@@ -0,0 +1,61 @@
+#!/usr/bin/env nextflow
+
+/*
+ * Copyright (c) 2020, Centre for Genomic Regulation (CRG).
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a copy
+ * of this software and associated documentation files (the "Software"), to deal
+ * in the Software without restriction, including without limitation the rights
+ * to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+ * copies of the Software, and to permit persons to whom the Software is
+ * furnished to do so, subject to the following conditions:
+ * 
+ * The above copyright notice and this permission notice shall be included in all
+ * copies or substantial portions of the Software.
+ * 
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+ * AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+ * OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+ /*
+  * author Davide Albanese <davide.albanese@gmail.com> 
+  */
+
+process foo {
+
+  memory { 1.GB * task.attempt }
+  maxRetries 2
+  errorStrategy 'retry'
+
+  output:
+  env PROCESS_FOO_FAILED into process_foo_failed
+
+  script:
+  """
+  if [ ${task.attempt} = ${task.maxRetries + 1 } ]; then
+    set +e
+  fi
+
+  PROCESS_FOO_FAILED=0
+  trap 'PROCESS_FOO_FAILED=1' ERR
+  cat thisfiledoesnotexist.txt
+  """
+}  
+
+process bar {
+  input: 
+  val x from process_foo_failed
+
+  when:
+  x=="1"
+
+  script:
+  """
+  echo $x
+  """
+}