New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error on PRE=1 (cython / numpy) #1579

Closed
skoudoro opened this Issue Jun 28, 2018 · 5 comments

Comments

Projects
None yet
3 participants
@skoudoro
Copy link
Member

skoudoro commented Jun 28, 2018

Description

Equality operator generate a ValueError with the pre cython version. here a small example:

simple_mp.pyx file:

import numpy as np
import cython

def masking_test():
	simple_wm = np.array([[0, 0, 0, 0, 0, 0],
                          [0, 0, 1, 0, 0, 0],
                          [0, 1, 1, 1, 0, 0],
                          [0, 1, 1, 1, 0, 0],
                          [0, 0, 0, 0, 0, 0]])
	mask = np.zeros(simple_wm.shape)

	simple_wm[mask==0] = 1

Result in a python terminal:

>>> import simple_mp as m
>>> m.masking_test()
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "simple_mp.pyx", line 13, in simple_mp.masking_test
    simple_wm[mask==0] = 1
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

Python 3.6
cython 0.29a0
numpy 1.16.0.dev0+2804c03

PRE Error:

======================================================================
ERROR: dipy.align.tests.test_parzenhist.test_parzen_joint_histogram
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/koudoro/anaconda/envs/dipy-pre-env-3/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/koudoro/Software/dipy/dipy/align/tests/test_parzenhist.py", line 136, in test_parzen_joint_histogram
    P.setup(static, moving, static_mask, moving_mask)
  File "dipy/align/parzenhist.pyx", line 105, in dipy.align.parzenhist.ParzenJointHistogram.setup
    self.smin = np.min(static[smask != 0])
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

======================================================================
ERROR: dipy.align.tests.test_parzenhist.test_parzen_densities
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/koudoro/anaconda/envs/dipy-pre-env-3/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/koudoro/Software/dipy/dipy/align/tests/test_parzenhist.py", line 193, in test_parzen_densities
    parzen_hist.setup(static, moving)
  File "dipy/align/parzenhist.pyx", line 105, in dipy.align.parzenhist.ParzenJointHistogram.setup
    self.smin = np.min(static[smask != 0])
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

======================================================================
ERROR: dipy.align.tests.test_parzenhist.test_joint_pdf_gradients_dense
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/koudoro/anaconda/envs/dipy-pre-env-3/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/koudoro/Software/dipy/dipy/align/tests/test_parzenhist.py", line 353, in test_joint_pdf_gradients_dense
    parzen_hist.setup(static, moving, smask, mmask)
  File "dipy/align/parzenhist.pyx", line 105, in dipy.align.parzenhist.ParzenJointHistogram.setup
    self.smin = np.min(static[smask != 0])
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

======================================================================
ERROR: dipy.align.tests.test_parzenhist.test_joint_pdf_gradients_sparse
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/koudoro/anaconda/envs/dipy-pre-env-3/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/koudoro/Software/dipy/dipy/align/tests/test_parzenhist.py", line 437, in test_joint_pdf_gradients_sparse
    parzen_hist.setup(static, moving, smask, mmask)
  File "dipy/align/parzenhist.pyx", line 105, in dipy.align.parzenhist.ParzenJointHistogram.setup
    self.smin = np.min(static[smask != 0])
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

======================================================================
ERROR: dipy.align.tests.test_parzenhist.test_exceptions
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/koudoro/anaconda/envs/dipy-pre-env-3/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/koudoro/Software/dipy/dipy/align/tests/test_parzenhist.py", line 619, in test_exceptions
    theta, transform, s, m, p, g)
  File "/Users/koudoro/anaconda/envs/dipy-pre-env-3/lib/python3.6/site-packages/numpy/testing/_private/utils.py", line 1235, in assert_raises
    return _d.assertRaises(*args,**kwargs)
  File "/Users/koudoro/anaconda/envs/dipy-pre-env-3/lib/python3.6/unittest/case.py", line 733, in assertRaises
    return context.handle('assertRaises', args, kwargs)
  File "/Users/koudoro/anaconda/envs/dipy-pre-env-3/lib/python3.6/unittest/case.py", line 178, in handle
    callable_obj(*args, **kwargs)
  File "dipy/align/parzenhist.pyx", line 388, in dipy.align.parzenhist.ParzenJointHistogram.update_gradient_sparse
    if (self.joint_grad is None) or (self.joint_grad.shape[2] != n):
AttributeError: 'ParzenJointHistogram' object has no attribute 'joint_grad'

======================================================================
ERROR: This test the number of inital direction."
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/koudoro/anaconda/envs/dipy-pre-env-3/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/koudoro/Software/dipy/dipy/direction/tests/test_bootstrap_direction_getter.py", line 34, in test_bdg_initial_direction
    boot_dg = BootDirectionGetter.from_data(voxel, dti_model, 30, sh_order=6)
  File "dipy/direction/bootstrap_direction_getter.pyx", line 52, in dipy.direction.bootstrap_direction_getter.BootDirectionGetter.from_data
    boot_gen = BootPmfGen(np.asarray(data, dtype=float), model, sphere,
  File "dipy/direction/pmf.pyx", line 113, in dipy.direction.pmf.BootPmfGen.__init__
    self.dwi_mask = model.gtab.b0s_mask == 0
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

======================================================================
ERROR: This tests the bootstrapping residual.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/koudoro/anaconda/envs/dipy-pre-env-3/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/koudoro/Software/dipy/dipy/direction/tests/test_bootstrap_direction_getter.py", line 119, in test_bdg_residual
    sh_order=6)
  File "dipy/direction/pmf.pyx", line 113, in dipy.direction.pmf.BootPmfGen.__init__
    self.dwi_mask = model.gtab.b0s_mask == 0
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

======================================================================
ERROR: This tests the local model used for the bootstrapping.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/koudoro/anaconda/envs/dipy-pre-env-3/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/koudoro/Software/dipy/dipy/direction/tests/test_pmf.py", line 60, in test_boot_pmf
    boot_pmf_gen = BootPmfGen(data, model=tensor_model, sphere=hsph_updated)
  File "dipy/direction/pmf.pyx", line 113, in dipy.direction.pmf.BootPmfGen.__init__
    self.dwi_mask = model.gtab.b0s_mask == 0
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
======================================================================
ERROR: This tests that the cmc tissue classifier returns expected
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/koudoro/anaconda/envs/dipy-pre-env-3/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/koudoro/Software/dipy/dipy/tracking/local/tests/test_tissue_classifier.py", line 174, in test_cmc_tissue_classifier
    average_voxel_size=1)
  File "dipy/tracking/local/tissue_classifier.pyx", line 137, in dipy.tracking.local.tissue_classifier.ConstrainedTissueClassifier.from_pve
    include_map[(wm_map + gm_map + csf_map) == 0] = 1
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

======================================================================
ERROR: This tests that the ParticleFilteringTracking produces
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/koudoro/anaconda/envs/dipy-pre-env-3/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/koudoro/Software/dipy/dipy/tracking/local/tests/test_tracking.py", line 274, in test_particle_filtering_tractography
    tc = ActTissueClassifier.from_pve(simple_wm, simple_gm, simple_csf)
  File "dipy/tracking/local/tissue_classifier.pyx", line 137, in dipy.tracking.local.tissue_classifier.ConstrainedTissueClassifier.from_pve
    include_map[(wm_map + gm_map + csf_map) == 0] = 1
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()

======================================================================
ERROR: This tests that the Bootstrat Peak Direction Getter plays nice
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/koudoro/anaconda/envs/dipy-pre-env-3/lib/python3.6/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/Users/koudoro/Software/dipy/dipy/tracking/local/tests/test_tracking.py", line 492, in test_bootstap_peak_tracker
    boot_dg = BootDirectionGetter.from_data(data, csd_model, 60)
  File "dipy/direction/bootstrap_direction_getter.pyx", line 52, in dipy.direction.bootstrap_direction_getter.BootDirectionGetter.from_data
    boot_gen = BootPmfGen(np.asarray(data, dtype=float), model, sphere,
  File "dipy/direction/pmf.pyx", line 113, in dipy.direction.pmf.BootPmfGen.__init__
    self.dwi_mask = model.gtab.b0s_mask == 0
ValueError: The truth value of an array with more than one element is ambiguous. Use a.any() or a.all()
```
@arokem

This comment has been minimized.

Copy link
Member

arokem commented Jun 28, 2018

@skoudoro

This comment has been minimized.

Copy link
Member

skoudoro commented Jun 28, 2018

Yes, it works @arokem. You can look at this issue on cython, there is more information.

@arokem

This comment has been minimized.

Copy link
Member

arokem commented Jun 28, 2018

Oh good - so looks like maybe they will fix it on their end before their next release?

In general -- we've been treating failures on the PRE=1 machine as blockers for merge, and it's holding up a bunch of things. @matthew-brett : are we doing this right? Should we use the PRE=1 machine just to report these things upstream, or should we actually be blocking merges of new things based on this?

@matthew-brett

This comment has been minimized.

Copy link
Member

matthew-brett commented Jun 29, 2018

I'm not sure what the right way is - probably not as blockers for merge, because then the poor author has to deal with odd Numpy changes that are not related to their code. Nightlies seems like a good idea - we only need these as reasonable warning that we're compatible with upcoming releases.

@skoudoro

This comment has been minimized.

Copy link
Member

skoudoro commented Jul 9, 2018

This error disappeared on PRE. Fixed via cython/cython@c04ca70

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment