Add access to workflow metadata and runtime information

[mfoll]

It would be interesting for reproducibility to have access to more environment variables to be able to write them in the output created during the pipeline. I can think for example of:

• repository (github, gitlab...)
• pipeline version (commit number and tag/version)
• the full command line used like written in .nextflow.log
• the docker image when provided using nextflow run -with-docker

Currently only nextflow version number is accessible with $NXF_VER (see here).

[pditommaso]

I'm proposing to add these properties in the main script scope

workflow.repository     // clone url of the project repository
workflow.commitId       // commit of the project repository
workflow.revision       // tag.branch of the project repository
workflow.startTime      // start execution timestamp 
workflow.endTime        // end execution timestamp
workflow.duration       // how long it take to run
workflow.container      // Docker image when available
workflow.commandLine    // execution command line
workflow.nextflow.version    // nextflow version number

workflow.onComplete { <do something> }

The last one is ah event handler that would allow to take an action when the pipeline execution complete e.g. send a notification, write the workflow metadata, etc.

[mfoll]

It seems perfect to me. For the Docker image you will only put it there when it's a single global image (-with-dockeroption), and we can refer to the trace file when they are process-specific?

[pditommaso]

An alternative, when more than a container is used, could be to have the container property to return a map of pairs process name: container name.

What do you think?

[mfoll]

Yes that would be even better if you can do that!

[pditommaso]

OK, I've just uploaded a new snapshot implementing that. You can download it running:

NXF_VER=0.16.0-SNAPSHOT CAPSULE_RESET=1 nextflow info 

Bonus: the onComplete event handler can be defined also in the config file, but in this case it needs to defined using the assignment operator. For example:

workflow.onComplete = {
  println "DONE!"
  println "Info: $workflow"
}

[pditommaso]

See the related documentation.