Optional Outputs

[tmdefreitas]

Broad's GDAC often deals with sparsely and inconsistently populated data, and many analyses support a range of acceptable inputs. For example, our methylation preprocessor will correlate methylation data to expression data if it is available for a sample set. However, this means that in addition to optional inputs (already supported by WDL), output files are also optional, or at least conditional upon inputs. Currently cromwell raises an error when an expected output is not present, and I don't believe there is syntax to support it in WDL.

It would be nice to be able to specify that an output is optional. One solution could be to mirror to the optional input syntax, and I could specify optional outputs like this:

task my_task {
    ... inputs, commands, etc...

    output {
        required_output="file1"
        optional_output?="file2"
    }
}

Having this would eliminate the need to write additional logic to create/handle blank files.

[scottfrazer]

Seems legit to me

[cjllanwarne]

I think you can get almost the same thing already using globs - I believe optional_output = glob("file2") will give you either a 0 or 1 element array of files depending on the existence or otherwise of the file.

[tmdefreitas]

Thanks for pointing out that workaround, that would at least allow this:

output {
  Array[File] optional_output = glob("file2")
}

It would still be nice at some point to not worry about 0/1-element arrays, since that syntax doesn't make it clear that the array will only be size 0 or 1.