Process "trimming" doesnt provide required input for process "qiime_import"

[d4straub]

Process "qiime_import" expects as "--input-path $trimmed" a path/to/folder containing all files with trimmed reads. The files with trimmed reads need to follow the naming scheme "*_L001_R{1,2}_001.fastq.gz".

Problems - process "trimming":

• outputs files with the naming scheme "*_L001_R{1,2}_001.fastq.gz.trimmed" instead of "*_L001_R{1,2}_001.fastq.gz"
• outputs files are not collected in a folder that is accessible with the process ""qiime_import" parameter "--input-path $trimmed"
• publishing all trimmed files would create significant data overhead

Problems - process "qiime_import":

• "--input-path $trimmed" is a channel containing all files from process "trimming" instead of a folder containing all these files

[d4straub]

Solved by publishing properly named symlinks for process "qiime_import" by process "trimming"

[apeltzer]

Problem: This can fail if its really using the publishDir directive and we should use dynamic output file names instead. I'll have a look and update this accordingly. Directly using files in paths such as $PWD is not working reliably...

[d4straub]

Part of the problem remains after commit 3d96744
Qiime import is now using trimmed.* filenames which leads to sample names with prefix "trimmed.". Therefore the sample names are not matching the ID's in the metadatafile and several processes fail.

[apeltzer]

I guess I'll have to find a way to rename files accordingly to metadata... or stick strictly to using the names of the metadata....

Workaround would also be to create a directory, move the trimmed files there with the same name and use these for the next step.

[apeltzer]

Can confirm this works now :-) Its just not possible to store the same file names in the same work directory, so I create a temporary directory in our work directory for each trimming process in fec5d44 and push the trimmed files (with the same name!) into the next step :-)