-
Notifications
You must be signed in to change notification settings - Fork 8
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Speeding up dnmtools states #26
Comments
Possibly. I'll keep this open and probably rename the issue, add detail so we might have a roadmap on how to do that. However, there's no simple switch that we can use to make this happen right away. |
Hi @andrewdavidsmith Seems like "states" could be made capable of using more memory and cores. The input sam could be split the temporary samlets and those samlets can be read in parallel. Thanks. |
That's not the expected behavior. If the reads are not sorted in the expected order, there's a chance it could turn from a linear time computation into a quadratic time one. If you could find a way to share the data with me I can check. Although I know you might not want to share all the data, the problem might not be easily revealed on just a small part. Let me know and feel free to email me. |
The right thing for us to do is have the code verify the sorting of reads, but sometimes the code attempts to just proceed and compensate if it gets unexpected input. I also notice from your screen capture that the program is using 17.2g of vmem, but only 3.2g of pmem, which suggests something else is going on and the program is likely thrashing. Are you sure you have sufficient available physical memory for that process? |
@andrewdavidsmith You were right, the algorithm was compensating for unsorted inputs. I have now been successfully ran the conversion to epiread pretty quickly. |
@hchetia I'm closing this because I think the issue has been solved. It should not continue if the input is unsorted in a way that will cause slowdown. Specifically, all reads from the same chromosome need to be consecutive. |
Hi,
Is there a way to use higher memory/cores with DNMtools states?
Best
H
The text was updated successfully, but these errors were encountered: