Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

chroder MemoryError #49

Closed
jinshangkun opened this issue Nov 4, 2020 · 5 comments
Closed

chroder MemoryError #49

jinshangkun opened this issue Nov 4, 2020 · 5 comments
Assignees
@mnshgl0110
Labels
enhancement

Comments

@jinshangkun
Copy link

Hi! I got a MemoryError error when I was running chroder, How can I solve it? Thank you！
The following is the error log：
Traceback (most recent call last):
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 732, in
scaf(args)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 340, in scaf
refdata = getdata(reflength, refid, refdir)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 144, in getdata
path = getpath(nodes)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 34, in getpath
tmppath = getpath(nodes, added, j, last)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 62, in getpath
tmppath = getpath(nodes, added, j, last)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 62, in getpath
tmppath = getpath(nodes, added, j, last)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 62, in getpath
tmppath = getpath(nodes, added, j, last)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 62, in getpath
tmppath = getpath(nodes, added, j, last)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 62, in getpath
tmppath = getpath(nodes, added, j, last)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 50, in getpath
path = path + getpath(nodes, added, nodes[i].children[0], last)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 62, in getpath
tmppath = getpath(nodes, added, j, last)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 62, in getpath
tmppath = getpath(nodes, added, j, last)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 62, in getpath
tmppath = getpath(nodes, added, j, last)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 62, in getpath
tmppath = getpath(nodes, added, j, last)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 50, in getpath
path = path + getpath(nodes, added, nodes[i].children[0], last)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 62, in getpath
tmppath = getpath(nodes, added, j, last)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 50, in getpath
path = path + getpath(nodes, added, nodes[i].children[0], last)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 50, in getpath
path = path + getpath(nodes, added, nodes[i].children[0], last)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 62, in getpath
tmppath = getpath(nodes, added, j, last)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 50, in getpath
path = path + getpath(nodes, added, nodes[i].children[0], last)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 50, in getpath
path = path + getpath(nodes, added, nodes[i].children[0], last)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 64, in getpath
tmppath = getpath(nodes, added + [nodes[i].name], j, nodes[i].name)
File "/public/home/zhangtz/user/jsk/tools/syri-master/syri/bin/chroder", line 54, in getpath
path = path + getpath(nodes, added + [nodes[i].name], nodes[i].children[0], nodes[i].name)
MemoryError
@mnshgl0110 mnshgl0110 self-assigned this Nov 4, 2020
@mnshgl0110
Copy link
Member

Hi. The script ran out of memory, adding more memory should fix this. Alternatively, remove small contigs and contigs consisting of repetitive sequence.

@jinshangkun
Copy link
Author

Thank you for your reply！Could you tell me how to add more memory? I don't know where to add parameters。

@mnshgl0110
Copy link
Member

To add more memory, you would just need to run it on a machine with more memory. This is a problem of the current heuristics where in some cases the amount of required memory increases significantly, which makes the script rather useless.

So, I would guess that the more pragmatic solution would be to try using it with a smaller number of only large contigs.

@jinshangkun
Copy link
Author

OK！I get it，Thanks again！

@mnshgl0110
Copy link
Member

The chroder script now uses the string clustering algorithm developed in Gunnar Klau's lab and is now much more memory efficient.

This should resolve this issue with most of the practical cases.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mnshgl0110 mnshgl0110

Labels
enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@mnshgl0110 @jinshangkun