ClassBadValue Error with Cosmology #534
Comments
|
The test case is clear. I tried but could not reproduce this with classylss 0.2.8, on Python 3.6. Which version of classylss are you using?
|
|
This is what I get with
|
|
I got a 2.7 environment up and did Then run Could it be possible you had local you see if helps? Did you set any special CLASS environment variables? |
|
Doing But now I get another error when runing
This looks to be a different problem entirely, should I open another issue? P.S.: I didn't do |
|
I found another thread with this problem, #508. The solution was to install gsl=2.4. I did that, and it was fixed. But now, when running I'm absolutely clueless as to how to proceed. |
|
There are two paths to proceed, and we shall follow both.
I strongly suspect it may be because your CPU lacks some instruction sets (e.g. avx) that the binary CorrFunc (highly optimized) you downloaded could have targeted. We can find out this by comparing your cpuinfo with the one where the conda package is compiled. If this is the case then a possible temporary fix is to compile CorrFunc locally by (or a similar command)
|
|
Ok. This is what cat /proc/cpuinfo yields. I'm working on a cluster so I chose only one processor to copy here.
This chunk of code should be sufficient: `import numpy as np ra = 360.*np.random.random(1000) ra = 360.*np.random.random(1000) cosmo = cosmology.Cosmology(h=0.676).match(Omega0_m=0.31) SurveyData2PCF('1d',data,ran,edges,cosmo=cosmo,show_progress=True)` |
|
I cannot reproduce this locally, but your CPU (a 2009 model) indeed does not have AVX support. Could you try the |
|
I just did. This is what I get
Will doing what the output suggests undo the previous pip installation? |
|
Looks like the installation of corrfunc failed. Could you post the output of the installation? |
|
Doesn't look like it...
By the way, this is the IPython I'm using:
|
|
See if this helps: |
|
I get the "Illegal instruction" message and automatic exit again :( |
|
Could you produce a stack trace by running the script with
|
|
I believe the corrfunc there should have been compiled with no AVX support. So it is likely something else. The stacktrace will help us to figure this out. |
|
Here's the stack trace: GNU gdb (GDB) Red Hat Enterprise Linux (7.2-92.el6) Program received signal SIGILL, Illegal instruction. Quit anyway? (y or n) y |
|
Figure I will start commenting here. In the |
|
@FedeDK , could you add to your corresponding file to https://github.com/bccp/nbodykit/blob/master/nbodykit/algorithms/pair_counters/corrfunc/base.py#L201 The file should be at
The line number may be a bit off from 201. I suppose this should pass isa='fallback'? |
|
I got an error that says "fallback" is not defined, so I asume you meant that I write it as a string. Doing so doesn't seem to have changed anything, I still get the "Illegal instruction" message. |
|
If the Corrfunc being used is the one built with pip, the illegal instruction message definitely shouldn't appear! So maybe there are multiple versions of Corrfunc installed now. Might need to repeatedly Just to double check: you're pip-installing corrfunc on the same machine that you're running the tests on, right? Some clusters have login nodes that don't support AVX but compute nodes that do, for example. |
Yeah, I'm trying everything in the cluster, where I have my virtual environment.
I did Doesn't seem to be a path problem. |
|
What happens if you now do `import Corrfunc.tests; Corrfunc.tests.tests()`
? This should tell us whether Corrfunc is working in this state. nbodykit
is probably complaining because it can't find its version of Corrfunc.
…On Thu, Nov 15, 2018 at 9:28 AM FedeDK ***@***.***> wrote:
Just to double check: you're pip-installing corrfunc on the same machine
that you're running the tests on, right? Some clusters have login nodes
that don't support AVX but compute nodes that do, for example.
Yeah, I'm trying everything in the cluster, where I have my virtual
environment.
If the Corrfunc being used is the one built with pip, the illegal
instruction message definitely shouldn't appear! So maybe there are
multiple versions of Corrfunc installed now. Might need to repeatedly pip
uninstall corrfunc and conda remove corrfunc until you get to a clean
slate.
I did pip uninstall corrfunc and conda remove corrfunc, then pip install
-U -I --no-deps CorrFunc. Now I get: ImportError: No module named
nbodykit.lab
Doesn't seem to be a path problem.
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#534 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AFYPy3TavczHZiS_fIeDj8sA1Kp5EhiHks5uvXn3gaJpZM4YcVEt>
.
|
|
When you did conda remove corrfunc nbodykit got removed too. conda should have asked you to confirm before you pressed yes. I agree we shall first confirm corrfunc is working properly. |
|
Ok, I got nbodykit back to working shape, but CorrFunc isn't installed. Trying Any ideas on how to use the package from the pip installation? Is this a path problem? |
|
What does `which pip` give you? Hopefully it will be pip from your conda
installation.
…On Thu, Nov 15, 2018 at 12:32 PM FedeDK ***@***.***> wrote:
Ok, I got nbodykit back to working shape, but CorrFunc isn't installed.
Trying pip install -U -I --no-deps CorrFunc didn't seem to work, it acts
as if it's still not installed. I think this explains why I still had the
"Illegal instruction" message before, the pip install wasn't affecting
anything I was working with.
Any ideas on how to use the package from the pip installation? Is this a
path problem?
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#534 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AFYPy1SXJduHrzUCLSDVV92r5gUGmB2fks5uvaUxgaJpZM4YcVEt>
.
|
|
It is, yes. |
|
Something's funny with your Python environment then! pip should work fine inside conda. Can you send the output of the You may wish to |
|
Ok, I got it to work. I bumped into another problem, but I'll get to that in the end. This is what i did: I reinstalled gsl=2.4.
So Corrfunc was indeed installed, I'm not sure what the problem was. Maybe the version posed an issue. Just in case this isn't actually finally solved, I'll post the outputs of what you asked for:
So no issue there. Then
Finally, the test
This is all correct, yes? The problem I'm facing now is that doing
So checking my variable fc.R1R2 I find that indeed there is no attribute "pairs", and I'm unsure as to whether there should be or not. And whether this is related or warrants another thread. Thanks for all the help so far!! |
|
Hmm, the "Segmentation fault" at the end of the Corrfunc tests indicates an
error. But github.com/nickhand/Corrfunc is a fork of an older version of
Corrfunc; it's possible this error is fixed in the latest version from
github.com/manodeep/Corrfunc. (Or the latest version straight from pip).
…On Thu, Nov 15, 2018 at 2:10 PM FedeDK ***@***.***> wrote:
Ok, I got it to work. I bumped into another problem, but I'll get to that
in the end. This is what i did:
I reinstalled gsl=2.4.
I pip installed Corrfunc with pip install git+git://
github.com/nickhand/Corrfunc, this is the output
Collecting git+git://github.com/nickhand/Corrfunc
Cloning git://github.com/nickhand/Corrfunc to /tmp/pip-21sDpM-build
Requirement already satisfied: numpy>=1.7 in
./.conda/envs/fdavilakurban/lib/python2.7/site-packages (from
Corrfunc==2.0.1)
Requirement already satisfied: future in
./.conda/envs/fdavilakurban/lib/python2.7/site-packages (from
Corrfunc==2.0.1)
Installing collected packages: Corrfunc
Found existing installation: Corrfunc 2.2.0
Uninstalling Corrfunc-2.2.0:
Successfully uninstalled Corrfunc-2.2.0
Running setup.py install for Corrfunc ... done
Successfully installed Corrfunc-2.0.1
You are using pip version 9.0.1, however version 18.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
So Corrfunc was indeed installed, I'm not sure what the problem was. Maybe
the version posed an issue.
I think this was all I did (I was trying a bunch of things in parallel),
if I see anything else I'll edit the comment.
Just in case this isn't actually finally solved, I'll post the outputs of
what you asked for:
which python, which pip, which conda yield:
~/.conda/envs/fdavilakurban/bin/python
~/.conda/envs/fdavilakurban/bin/pip
~/.conda/envs/fdavilakurban/bin/conda
So no issue there.
Then python -c 'import Corrfunc; print(Corrfunc.__file__)'
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/Corrfunc/
*init*.pyc
Finally, the test import Corrfunc.tests; Corrfunc.tests.tests():
Done reading the data - time taken = 1.1 seconds
Beginning Theory Correlation functions calculations
Running 3-D correlation function DD(r)
Running with points in [xmin,xmax] = 0.000400,419.999817 with periodic
wrapping = 420.000000
Running with points in [ymin,ymax] = 0.000100,419.999512 with periodic
wrapping = 420.000000
Running with points in [zmin,zmax] = 0.000300,419.999207 with periodic
wrapping = 420.000000
In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y,
nmesh_z] = 35,35,17. Time taken = 0.109 sec
Using SSE kernel
0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100%
done. Time taken = 5.346 secs
DD(r): first 5 bins
rmin rmax rpavg npairs weightavg
0.1675 0.2388 0.0000 325832 1.0000
0.2388 0.3403 0.0000 498880 1.0000
0.3403 0.4849 0.0000 770944 1.0000
0.4849 0.6910 0.0000 1173536 1.0000
0.6910 0.9848 0.0000 1709944 1.0000
------------------------------
Running 2-D correlation function DD(rp,pi)
Running with points in [xmin,xmax] = 0.000400,419.999817 with periodic
wrapping = 420.000000
Running with points in [ymin,ymax] = 0.000100,419.999512 with periodic
wrapping = 420.000000
Running with points in [zmin,zmax] = 0.000300,419.999207 with periodic
wrapping = 420.000000
In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y,
nmesh_z] = 35,35,10. Time taken = 0.108 sec
Using SSE kernel
0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100%
done. Time taken = 12.109 secs
****** DD(rp,pi): first 5 bins *******
rmin rmax rpavg pi_upper npairs weightavg
0.1675 0.2388 0.0000 1.0 488624 1.0000
0.1675 0.2388 0.0000 2.0 41796 1.0000
0.1675 0.2388 0.0000 3.0 16322 1.0000
0.1675 0.2388 0.0000 4.0 10674 1.0000
0.1675 0.2388 0.0000 5.0 8496 1.0000
------------------------------
Running 2-D correlation function DD(s,mu)
Running with points in [xmin,xmax] = 0.000400,419.999817 with periodic
wrapping = 420.000000
Running with points in [ymin,ymax] = 0.000100,419.999512 with periodic
wrapping = 420.000000
Running with points in [zmin,zmax] = 0.000300,419.999207 with periodic
wrapping = 420.000000
In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y,
nmesh_z] = 35,35,35. Time taken = 0.137 sec
Using SSE kernel
0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100%
done. Time taken = 4.963 secs
****** DD(s,mu): first 5 bins *******
smin smax savg mu_max npairs weightavg
0.1675 0.2388 0.2034 0.1 16696 1.0000
0.1675 0.2388 0.2033 0.1 16488 1.0000
0.1675 0.2388 0.2037 0.2 16020 1.0000
0.1675 0.2388 0.2036 0.2 16060 1.0000
0.1675 0.2388 0.2035 0.2 15934 1.0000
------------------------------
Running 2-D projected correlation function wp(rp)
In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y,
nmesh_z] = 35,35,10. Time taken = 0.108 sec
Using SSE kernel
0%.........10%.........20%.........30%.........40%.........50%.........60%.........70%.........80%.........90%.........100%
done. Time taken = 10.966 secs
****** wp: first 5 bins *******
rmin rmax rpavg wp npairs weightavg
0.1675 0.2388 0.0000 303.6 718966 1.0000
0.2388 0.3403 0.0000 223.8 1156266 1.0000
0.3403 0.4849 0.0000 164.5 1890416 1.0000
0.4849 0.6910 0.0000 118.5 3116582 1.0000
0.6910 0.9848 0.0000 83.5 5212852 1.0000
------------------------------
Running 3-D auto-correlation function xi(r)
In gridlink_index_particles_double> Running with [nmesh_x, nmesh_y,
nmesh_z] = 35,35,17. Time taken = 0.094 sec
Using SSE kernel
0%Segmentation fault
This is all correct, yes?
The problem I'm facing now is that doing SurveyData2PCF.run() (I defined
fc=SurveyData2PCF) yields an attribute error:
------------------------------
AttributeError Traceback (most recent call last)
in ()
----> 1 fc.run()
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/algorithms/paircount_tpcf/tpcf.pyc
in run(self)
462 """
463 # this does most of the work
--> 464 BasePairCount2PCF.run(self)
465
466 # compute wp(rp) if we computed xi(rp, pi)
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/algorithms/paircount_tpcf/tpcf.pyc
in run(self)
125 result = LandySzalayEstimator(pair_counter, self.data1, self.data2,
126 self.randoms1, self.randoms2, R1R2=self.R1R2,
--> 127 logger=self.logger, **attrs)
128 self.D1D2, self.D1R2, self.D2R1, self.R1R2, self.corr = result
129
/home/fdavilakurban/.conda/envs/fdavilakurban/lib/python2.7/site-packages/nbodykit/algorithms/paircount_tpcf/estimators.pyc
in LandySzalayEstimator(pair_counter, data1, data2, randoms1, randoms2,
R1R2, logger, **kwargs)
204 fRD = R1R2.attrs['total_wnpairs']/D2R1.attrs['total_wnpairs']
205
--> 206 nonzero = R1R2.pairs['npairs'] > 0
207
208 # init
AttributeError: 'BinnedStatistic' object has no attribute 'pairs'
So checking my variable fc.R1R2 I find that indeed there is no attribute
"pairs", and I'm unsure as to whether there should be or not. And whether
this is related or warrants another thread.
Thanks for all the help so far!!
—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<#534 (comment)>, or mute
the thread
<https://github.com/notifications/unsubscribe-auth/AFYPy0LQJYtm_OtLHilTw0wOjlZfHHy0ks5uvbwSgaJpZM4YcVEt>
.
|
|
I installed the version you mentioned:
But still get a similar output
|
|
I do not see the RR.pairs error running your script with nbodykit 0.3.7 Python 2.7 and CorrFunc 2.2.0. |
|
I'm not sure I understand. Did you modify the code so that it would work? Could you post it? I tried it on my pc, with a newly created environment and freshly installed nbodykit, and I got the same error of the missing "pairs" attribute. Is it something in my code? |
|
This is the code I used. It is the same as yours, but I set a seed and swapped out the Table. I measured the time and the walk clock time is the same using dict or using Table. Either case the code runs. What's the version of your nbodykit, on your own linux box? |
|
From where I am right now I can only access the cluster, not my workplace pc (which is the other pc I tried the code out). Just for clarification, that code runs fine, but the Attributes Error pops up when running P.S.: I edited the code in my post way above because I messed up some numbers in the "edges" definition. |
|
I see. I don't think you are supposed to call |
|
@FedeDK In case there are multiple versions of Corrfunc installed, will you please print out |
|
@rainwoodman Ok, my bad. @manodeep It prints: u'2.2.0'. Which is expected, that's the version I pip installed from git. Just to clarify, I never explicitly import Corrfunc in the code I'm using. |
|
@FedeDK Thanks. From the segfault, it certainly looks like there is some sort of a bug in the SSE kernels, but I am not sure whether that's a hardware or a hardware+compiler, or an actual bug in the code. Running the tests with I am at a loss as to how to reproduce the bug :( |
|
@FedeDK We released Corrfunc v2.3. Not really if that fixes anything but might be worth a shot... |
So, I'm having trouble with a very simple task. Trying to do:
cosmo = cosmology.Cosmology(h=0.676).match(Omega0_m=0.31)Yields this error:
I also get the same error importing a cosmology from Astropy and then trying to calculate the 2PCF with SurveyData2PCF. Any ideas as to why this is happening?
The text was updated successfully, but these errors were encountered: