Efficiency improvements

[ktmeaton]

Currently, recombination detection is slow at 5 seconds / sequences. Multiprocessing helps (--threads) but certainly there is code efficiency improvements needed.

[ktmeaton]

I think most of the slowdown is purely in passing large objects as parameters to genome_mp.

• Passing simple parameters (boolean, logger, int), runs at 200-350 sequences / second.
• Passing complex parameters (tree, dataframe), runs at 1-3 sequences/second.
• Passing+analyzing complex parameters, runs at 0.2 sequences/second (~5 seconds / sequence).

Because of this, I don't think my implementation of the analysis is the biggest problem.

[ktmeaton]

This might be a problem in which rust could solve.

[wtchoga]

[ktmeaton]

The code base rewrite to Rust (PR #5) was extremely helpful for this issue. On a single core, I've seen processing speeds ranging from 1-100 sequences/second. And that's not even taking --threads into account 😀

But I am leaving this unresolved until I benchmark with a larger dataset (ex. VirusSeq).