-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Is process hanging or will it just take a while? #31
Comments
Hello @nmb85, Thank you very much for your interest in MetaCoAG! I haven't tested MetaCoAG on datasets having more than a couple of hundred thousand contigs. I don't know how long it will take to complete (maybe a couple of days?). If it is possible, can you share with me the data you are testing on? I would like to give it a try and see. 35 million contigs sound very interesting! Thank you! |
Thank you, @Vini2! I will reach out to you via your contact form on your professional website in order to share the data. The data is proprietary and entails tens of GBs, so I cannot post it on a public link. In the meantime, this was my attempt at parallelizing the I imported
Then in
Changes ran fine to completion without error messages on a toy dataset, but couldn't observe multiprocessing via htop |
Brief update: I allowed |
Hi @Vini2, you're probably swamped - should we move ahead with trying to parallelize this on our end? If so, any hints or suggestions based on our attempt above? |
Hi @nmb85, I'm so sorry I couldn't get back to you regarding this issue. Is everything sorted? Were you able to parallelise the step? I tested your suggested method and it works fine. |
Closing issue due to inactivity. Please re-open if needed. |
I love the concept for MetaCoAG; what a great idea! I'm trying to use your awesome tool to bin contigs from a 30 Gbp MEGAHIT metagenomic assembly with 35 million contigs and it has been paused (or working?) at the step after initially assigning contigs with marker genes to bins for a little more than 48 hours. There is no sign that memory or CPU usage has changed in that time and there haven't been any messages printed to the log file or stdout/stderr. The only files in the output directory are the tetranucleotide frequency pickle file and the log file. The log file is attached below.
Is my process hanging, does it require more memory (current usage is steady at 65% of max: 175 GB/250 GB), or is it working? If it is in fact working, what would you expect the time to complete this step to be and is there a flag that I missed for parallelizing this step?
Thanks for any help! Would love to see how MetaCoAG performs!
metacoag.log
The text was updated successfully, but these errors were encountered: