Processing unclassified metagenomics contigs

[yesimon]

Assemble via trinity

Run the following analysis on assembled contigs

• prodigal orf -> rpsblast on CDD database.
• infernal on RNA database.
• blastn
• blastx

[yesimon]

Let's address with #795 with this.

[dpark01]

Instead of the self-referencing #795, did you mean to say #784?

[dpark01]

Two comments:

1. Does this need to be a PR right now? PRs should be entirely unnecessary for dev purposes until you're sure it's ready to go (including tests passing).
2. Instead of pulling from NCBI FTP for nr, switch to a standard WDL Input file. We already know that production at-scale dependencies on NCBI can cause all sorts of non-deterministic failures (which is why we comment out most of the NCBI-based unit tests from Travis). Additionally, it means we lose control of database versioning if we pull straight from the source. Also, using the normal Input mechanism allows for underlying platform optimizations (such as DNAnexus's piped input file) to reduce staging time. Use a .tar.lz4 for input.

[yesimon]

Yeah 784. Maybe the best would be a config switch between downloading from NCBI versus downloading a controlled version, with the controlled versioned db as the default.

[dpark01]

Is there any reason to prefer an uncontrolled version? On the snakemake side the general pattern is to have a separate rule to download, on the WDL side, maybe there could be a dedicated Task just for a one-off download-and-tar-up job... I certainly think that hitting their FTP server in parallel isn't going to bode well...

[tomkinsc]

@yesimon: Just wanted to check in on this PR. Do you have work in mind before it's ready to merge? (Apart from correcting the DNAnexus file- ID mentioned in the most recent Travis failure)

[dpark01]

does this replace #737 (should we close that one?)