Skip to content

v1.3.2

Latest

Choose a tag to compare

@donovan-h-parks donovan-h-parks released this 19 Nov 21:58
· 4 commits to master since this release
a140d26

Reduces the memory needed to call genes and fixes occasional high memory usage by analyze. Results are unchanged: all outputs of lineage_wf and taxonomy_wf are identical to CheckM 1.3.1 on 128 test genomes, and quality_report.tsv is identical on 1,000 genomes.

  • Gene calling (the first step of tree, lineage_wf and taxonomy_wf) no longer starts the largest genomes all at once. Genomes are still processed from largest to smallest, but a genome is only started if the estimated memory of Prodigal fits within a budget set from the sizes of the genomes, and smaller genomes are processed in the meantime. With 64 threads and 1,000 genomes, gene calling needs 5.5 GiB instead of 8.3 GiB and takes 4% longer (about 10 s).
  • analyze (including the analyze step of lineage_wf and taxonomy_wf) could occasionally need much more memory: its worker processes sometimes made private copies of the memory of the main process when they ran Python's garbage collector, as fixed for other steps in 1.3.1 (e.g., 10.3 GiB instead of ~3 GiB in one run on 1,000 genomes with 64 threads). Worker processes now keep sharing this memory.
  • Coding density is calculated from the positions of genes instead of an array with 8 bytes per base of each contig, which reduces the memory used by each thread when calling genes and calculating genome statistics (e.g., by ~80 MB for a 10 Mbp genome).

See the change log for earlier releases.