WES 'rebuild' route, or other per-section pipeline running

[stevekm]

The WES pipeline produces a lot of intermediary files (fastq's, intermediary .bam files, etc) which are not required after the pipeline is finished. They take up a large amount of space, so I think they can probably be deleted (?).

However, if you end up needing those files again, is there an easy way to 'rebuild' those parts of the pipeline output without running the entire pipeline? For example, just run everything ending before the variant calling?

Not sure how easy this would be to implement, or if its already possible without breaking anything. Alternatively, maybe it would be worth having a way to custom-define the segments of a pipeline to be run?

[igordot]

It is arguable what is considered intermediary and final. It is up to the user to keep or delete any files.

If you re-run SNS, it will process any missing segments. One caveat is that segments that generate large files also generate samples.[segment].csv files. That prevents those files from being generated again. This can be useful if you already deleted some files you don't need, but want to add more samples to the same project without having to regenerate all those files again. You can force them to be regenerated if you delete or edit the corresponding samples.[segment].csv file.