In CPU mode, smaller chunk_size does not reduce the memory use.

[dawnmy]

The memory use seems to be not related to the chunk_size setting but the total number of input sequence bases. Need to find a way to reduce the memory use for large input files.

[HongxiangXu]

I seems met similar problem, as I used 4 threads and 256 chunk size, which should use a little memory right? as 20 thread with 1024 chunk size will only use about 20G memory as it described. However, my PC was stuck and my task manager showed nearly all memory(16G) were consumed, please figured out this problems. Thanks very much!

[dawnmy]

I seems met similar problem, as I used 4 threads and 256 chunk size, which should use a little memory right? as 20 thread with 1024 chunk size will only use about 20G memory as it described. However, my PC was stuck and my task manager showed nearly all memory(16G) were consumed, please figured out this problems. Thanks very much!

Thank you for your interest in RiboDetector. I am working on the new version. Next release will solve this issue. BTW, how large is your input fastq file (number of nucleotides)?

[dawnmy]

@xhxlilium This issue has been solved in the latest version v0.2.6. It can be updated with pip. When running on your large input files, you can use smaller threads and chunk_size (e.g. --threads 10 chunk_size 400, which will use about 10GB RAM but may be up to 20GB). I suggest to run RiboDetector on a PC with at least 32GB RAM.