Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

TypeError Can't broadcast #305

Closed
jeffinrockey opened this issue Aug 9, 2018 · 16 comments
Closed

TypeError Can't broadcast #305

jeffinrockey opened this issue Aug 9, 2018 · 16 comments

Comments

@jeffinrockey
Copy link

Dear Deren,
Please see the error below.
version: ipyrad 0.7.28
Could you please let me know what would be causing this.

[####################] 100% chunking clusters | 0:00:33 | s5 |
[####################] 100% consens calling | 0:28:39 | s5 |
[####################] 100% concat/shuffle input | 0:01:32 | s6 |
[####################] 100% clustering across | 3:19:17 | s6 |
[####################] 100% building clusters | 0:01:20 | s6 |
[####################] 100% aligning clusters | 0:08:43 | s6 |
[################### ] 99% database indels | 0:04:49 | s6 |
Encountered an unexpected error (see ./ipyrad_log.txt)
Error message is below -------------------------------
TypeError(Can't broadcast (100, 1130, 133) -> (100, 187, 133))
@isaacovercast
Copy link
Collaborator

isaacovercast commented Aug 9, 2018 via email

@isaacovercast
Copy link
Collaborator

isaacovercast commented Aug 9, 2018 via email

@jeffinrockey
Copy link
Author

Dear Isaac,
Sorry about being late in replying, was traveling during the weekend.
No branching was done. Given below is the complete script, except some paths that I am required to remove. Please let me know how can I set the -d mentioned, in the below script. A snippet with the -d usage would be very helpful.
Jeffin


## import ipyrad
import ipyrad as ip

## create an Assembly and modify some parameter settings
data1 = ip.Assembly("Fish")
data1.set_params("project_dir", ".../ipyrad/PROJECT-DIR")
data1.set_params("sorted_fastq_path", ".../ipyrad/RAWFILES/*fastq.gz")

## run steps 3-7 for the two Assemblies
data1.run("1234567")

@isaacovercast
Copy link
Collaborator

Now it's my turn to apologize for the delay, I was at Evolution in France.

Looks like you are using API mode (which is cool!), the easiest way to use the debug flag is through the CLI, so you could do something like this in a new notebook cell:

data1.write_params("params-tmp.txt")
!ipyrad -p params-tmp.txt -s 6 -f -d

This will write the params to a file, then it will run only step 6 with the debug turned on.

@jeffinrockey
Copy link
Author

Hi Isaac,
I executed the two lines as advised. First line in python and second line in CLI subsequently. Still error occurring but this time error log is more informative . Please see below and let me know what can be done to rectify.
Jeffin

Step 6: Clustering at 0.85 similarity across 100 samples
[####################] 100% concat/shuffle input | 0:01:33
[####################] 100% clustering across | 3:18:39
[####################] 100% building clusters | 0:01:19
[####################] 100% aligning clusters | 0:09:33
[################### ] 99% database indels | 0:04:27
Encountered an unexpected error (see ./ipyrad_log.txt)
Error message is below -------------------------------
TypeError(Can't broadcast (100, 1130, 133) -> (100, 187, 133))
Traceback (most recent call last):
File "/home/jeffin.r/anaconda2/bin/ipyrad", line 11, in
load_entry_point('ipyrad==0.7.28', 'console_scripts', 'ipyrad')()
File "/home/jeffin.r/anaconda2/lib/python2.7/site-packages/pkg_resources/init.py", line 572, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/home/jeffin.r/anaconda2/lib/python2.7/site-packages/pkg_resources/init.py", line 2755, in load_entry_point
return ep.load()
File "/home/jeffin.r/anaconda2/lib/python2.7/site-packages/pkg_resources/init.py", line 2408, in load
return self.resolve()
File "/home/jeffin.r/anaconda2/lib/python2.7/site-packages/pkg_resources/init.py", line 2414, in resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "/home/jeffin.r/anaconda2/lib/python2.7/site-packages/ipyrad/init.py", line 20, in
from . import load as _load
File "/home/jeffin.r/anaconda2/lib/python2.7/site-packages/ipyrad/load/init.py", line 14, in
from .load import test_assembly
File "/home/jeffin.r/anaconda2/lib/python2.7/site-packages/ipyrad/load/load.py", line 14, in
from ipyrad.assemble.util import IPyradWarningExit, ObjDict
File "/home/jeffin.r/anaconda2/lib/python2.7/site-packages/ipyrad/assemble/init.py", line 5, in
from . import demultiplex
File "/home/jeffin.r/anaconda2/lib/python2.7/site-packages/ipyrad/assemble/demultiplex.py", line 22, in
from ipyrad.core.sample import Sample
File "/home/jeffin.r/anaconda2/lib/python2.7/site-packages/ipyrad/core/init.py", line 5, in
from . import parallel
File "/home/jeffin.r/anaconda2/lib/python2.7/site-packages/ipyrad/core/parallel.py", line 8, in
import ipyparallel as ipp
File "/home/jeffin.r/anaconda2/lib/python2.7/site-packages/ipyparallel/init.py", line 11, in
import zmq
File "/home/jeffin.r/anaconda2/lib/python2.7/site-packages/zmq/init.py", line 34, in
from zmq import backend
File "/home/jeffin.r/anaconda2/lib/python2.7/site-packages/zmq/backend/init.py", line 40, in
reraise(*exc_info)
File "/home/jeffin.r/anaconda2/lib/python2.7/site-packages/zmq/backend/init.py", line 27, in
_ns = select_backend(first)
File "/home/jeffin.r/anaconda2/lib/python2.7/site-packages/zmq/backend/select.py", line 26, in select_backend
mod = import(name, fromlist=public_api)
File "/home/jeffin.r/anaconda2/lib/python2.7/site-packages/zmq/backend/cython/init.py", line 6, in
from . import (constants, error, message, context,
ImportError: /home/jeffin.r/anaconda2/lib/python2.7/site-packages/matplotlib/../../../libstdc++.so.6: version `GLIBCXX_3.4.20' not found (required by /home/jeffin.r/anaconda2/lib/python2.7/site-packages/zmq/backend/cython/../../../../../libzmq.so.5.1.2)

@isaacovercast
Copy link
Collaborator

Are you running on a cluster? The GLIBC error is normally a problem on older HPC systems with old, out of date GLIBC installs.

@jeffinrockey
Copy link
Author

Dear Isaac ,
I checked with our admin on the GLIBC issue. It was due to an anaconda mix up. Deleted and re-installed conda and ipyrad. The GLIBC went away. However at step 6 the actual broad cast error resurfaced even with the -d -f option from CLI.

I ran the program in Redhat as well as Ubuntu servers. The issue persists.

Is there anything else that I may try to get over this issue.

Jeffin

@isaacovercast
Copy link
Collaborator

The -d option just turns on debug mode, it won't fix anything, and the -f option forces overwriting previous results. Can you paste here the last 20-30 lines of the ipyrad_log.txt file?

@jeffinrockey
Copy link
Author

jeffinrockey commented Sep 11, 2018 via email

@isaacovercast
Copy link
Collaborator

The file got blanked. Can you run ipyrad step 6 with the -d flag again, there should be hundreds or thousands of lines of output in the ipyrad_log.txt file.

@jeffinrockey
Copy link
Author

jeffinrockey commented Sep 11, 2018 via email

@jeffinrockey
Copy link
Author

Hi Isaac,
The last few lines of ipyrad_log.txt are as below. Please see and advise.

2018-09-12 07:38:11,893         pid=17622       [cluster_across.py]     INFO    inarr shape (100, 2360, 133)
2018-09-12 07:38:11,893         pid=17622       [cluster_across.py]     INFO    iset shape (100, 513443, 133)
2018-09-12 07:38:11,985         pid=17622       [cluster_across.py]     INFO    inarr shape (100, 2360, 133)
2018-09-12 07:38:11,986         pid=17622       [cluster_across.py]     INFO    iset shape (100, 513443, 133)
2018-09-12 07:38:12,145         pid=17622       [cluster_across.py]     INFO    inarr shape (100, 2360, 133)
2018-09-12 07:38:12,145         pid=17622       [cluster_across.py]     INFO    iset shape (100, 513443, 133)
2018-09-12 07:38:12,299         pid=17622       [cluster_across.py]     INFO    inarr shape (100, 2360, 133)
2018-09-12 07:38:12,300         pid=17622       [cluster_across.py]     INFO    iset shape (100, 513443, 133)
2018-09-12 07:38:12,527         pid=17622       [cluster_across.py]     INFO    inarr shape (100, 2360, 133)
2018-09-12 07:38:12,528         pid=17622       [cluster_across.py]     INFO    iset shape (100, 513443, 133)
2018-09-12 07:38:12,726         pid=17622       [cluster_across.py]     INFO    inarr shape (100, 2360, 133)
2018-09-12 07:38:12,726         pid=17622       [cluster_across.py]     INFO    iset shape (100, 513443, 133)
2018-09-12 07:38:12,912         pid=17622       [cluster_across.py]     INFO    inarr shape (100, 2360, 133)
2018-09-12 07:38:12,912         pid=17622       [cluster_across.py]     INFO    iset shape (100, 513443, 133)
2018-09-12 07:38:13,171         pid=17622       [cluster_across.py]     INFO    inarr shape (100, 2360, 133)
2018-09-12 07:38:13,171         pid=17622       [cluster_across.py]     INFO    iset shape (100, 513443, 133)
2018-09-12 07:38:13,415         pid=17622       [cluster_across.py]     INFO    inarr shape (100, 2360, 133)
2018-09-12 07:38:13,415         pid=17622       [cluster_across.py]     INFO    iset shape (100, 513443, 133)
2018-09-12 07:38:13,633         pid=17622       [cluster_across.py]     INFO    inarr shape (100, 2360, 133)
2018-09-12 07:38:13,633         pid=17622       [cluster_across.py]     INFO    iset shape (100, 513443, 133)
2018-09-12 07:38:13,884         pid=17622       [cluster_across.py]     INFO    inarr shape (100, 2360, 133)
2018-09-12 07:38:13,884         pid=17622       [cluster_across.py]     INFO    iset shape (100, 513443, 133)
2018-09-12 07:38:14,059         pid=17622       [cluster_across.py]     INFO    inarr shape (100, 2248, 133)
2018-09-12 07:38:14,059         pid=17622       [cluster_across.py]     INFO    iset shape (100, 513443, 133)
2018-09-12 07:38:22,096         pid=17570       [assembly.py]   ERROR   TypeError(Can't broadcast (100, 2248, 133) -> (100, 1323, 133))
2018-09-12 07:38:24,799         pid=17570       [assembly.py]   INFO      shutting down engines
2018-09-12 07:38:26,002         pid=17570       [assembly.py]   INFO      finished shutdown
2018-09-12 07:38:26,071         pid=17570       [__init__.py]   INFO    debugging turned off

@isaacovercast
Copy link
Collaborator

hm. Can you zip up the *_across directory and dropbox it to me? I can't figure it out without looking at the data. Or you can send the raw sample data..

@jeffinrockey
Copy link
Author

jeffinrockey commented Sep 13, 2018 •
edited

Dear Isaac,
I cannot really share the raw data or any sample specific data. It is not the data size or so that is the problem, other restrictions instead. Very sorry about this limitation.
Would you please share me a public/test data set link, which I shall download and then run ipyrad for it in the server. This way we can gather the whole data required to resolve.
Let me know your thoughts on this alternative.

@isaacovercast
Copy link
Collaborator

You can download a test dataset following the instructions on the pedicularis tutorial:

https://ipyrad.readthedocs.io/pedicularis_.html

There are also several simulated datasets in the github repository in tests/ipsimdata.tar.gz

@jeffinrockey
Copy link
Author

Dear Isaac,
At last got it working. It was my mistake indeed. The restriction site overhang given was incorrect. It lead to 0 filtered for almost all samples ,thereby causing broadcast error downstream.

Very much appreciate your support in resolving. I am closing this issue.Thanks to you again, Isaac.

Jeffin

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@isaacovercast @jeffinrockey