Documentation: workflow.stats

[ewels]

I don't think that workflow.stats is documented anywhere. It was added way back in 2017 (see #534). It's used in a bunch of places in Nextflow itself, eg:

nextflow/modules/nextflow/src/main/resources/nextflow/trace/ReportTemplate.html

Line 120 in 0bce2be

<% if ( workflow.success && workflow.stats.ignoredCount ){ %>

It'd be nice to document this under Workflow introspection.

[mribeirodantas]

Quite interesting all these variables here: ./modules/nextflow/src/main/groovy/nextflow/trace/WorkflowStats.groovy. Now I know where Tower learns about this 😅

[pditommaso]

This is an internal data structure, don't think it should be documented

[ewels]

Ok - use case on Slack was to detect if any errors had been ignored at the end of a run. Reason being to go past errorStrategy: finish - really go as far as possible in running the pipeline, but then exit 1 at the very end if any errors were ignored.

If not documenting workflow.stats, any possibility to make this summary info more accessible (documented) somehow?

[adamrtalbot]

really go as far as possible in running the pipeline, but then exit 1 at the very end if any errors were ignored.

This is a pretty useful feature. Imagine the scenario where you have 96 independent samples, if 1 fails and it brings the whole pipeline crashing down that's annoying, but if you use errorStrategy = 'ignore' then you have to write your own solution. We could have process.errorStrategy = 'catch'. This might be a separate ticket though.

Having some form of introspection on tasks in the workflow.onComplete block would also help for similar reasons and be more generally useful.

[ewels]

Closing, as no plans to document this for now. Might be something that turns up in the future if we ever spin up some (semi-) automated API docs for Nextflow. But it's a deliberate decision to omit from the prose docs for now.