v0.8.0: Simplified API, Graph plotting, Audit to HTML, TeX and Bash, and more

This release contains a very large number of improvements, too large to list individually here, but a few selected one are covered further below. This release brings in another contributor, @jonalv, who did fantastic work on the TeX template for the audit report conversion feature.

Notable new features

A simplified API

Each task are now executed in its own isolated temporary folder, so that extra files generated by commands are properly captured and handled in an atomic way (to avoid mixing up non-finished and finished files).

Among the improved areas is that setting paths is now not even required. If you still want to set the output file extension for outputs, you can do that with the following syntax in an out-port placeholder in commands: {o:portname|.csv}, for the .csv extension.

Furthermore, the many different Process.SetPath... methods are now unified to only two: Process.SetOut(portName string, pattern string) and Process.SetOutFunc(portName string, pathFunc func(Task) string).

SetOut() takes placeholder similar to those used to define the command pattern, such as {i:portname} for input files and {p:param1} for parameters. It also allows certain modifiers after the port name, separated by | characters, such as for trimming the end of a string, which is done like so: {i:bamfile|%.bam}, given that we have an in-port named "bamfile", for which we want to re-use its filename, but without the .bam file extension.

As always, for more information about this, see the documentation.

Graph plotting

SciPipe can now plot the graph of a workflow to a .dot file, which can be converted to PDF with the GraphViz dot command (See the documentation for this feature).

This can be done by adding this line in the workflow Go file:

myWorkflow.PlotGraph("myworkflow.dot")

One can also let SciPipe execute the dot command as well to convert to PDF in one go (requires having GraphViz installed):

myWorkflow.PlotGraphPDF("myworkflow.dot")

An example plot can be seen here:

Convert Audit report to TeX / PDF

This is an experimental feature. (See the documentation for this feature).

Usage:

scipipe audit2tex somefile.audit.json
pdflatex somefile.audit.tex
open somefile.audit.pdf

How it looks currently:

Convert Audit report to HTML

This is an experimental feature. (See the documentation for this feature).

Usage:

scipipe audit2html somefile.audit.json

How it looks currently:

Convert Audit report to Bash

This is an experimental feature. (See the documentation for this feature).

Usage:

scipipe audit2bash somefile.audit.json

How it looks currently: