Tests failing under NumPy 1.13

[jni]

Description

Oops! Turns out -= with Boolean arrays is deprecated, and we use it at least thrice:

https://travis-ci.org/scikit-image/scikit-image/jobs/241801969

_______________ [doctest] skimage.morphology.watershed.watershed _______________
220     >>> mask_circle2 = (x - x2)**2 + (y - y2)**2 < r2**2
221     >>> image = np.logical_or(mask_circle1, mask_circle2)
222 
223     Next, we want to separate the two circles. We generate markers at the
224     maxima of the distance to the background:
225 
226     >>> from scipy import ndimage as ndi
227     >>> distance = ndi.distance_transform_edt(image)
228     >>> from skimage.feature import peak_local_max
229     >>> local_maxi = peak_local_max(distance, labels=image,
UNEXPECTED EXCEPTION: TypeError('numpy boolean subtract, the `-` operator, is deprecated, use the bitwise_xor, the `^` operator, or the logical_xor function instead.',)
Traceback (most recent call last):
  File "/opt/python/3.6.1/lib/python3.6/doctest.py", line 1330, in __run
    compileflags, 1), test.globs)
  File "<doctest skimage.morphology.watershed.watershed[9]>", line 3, in <module>
  File "/home/travis/build/scikit-image/scikit-image/skimage/feature/peak.py", line 127, in peak_local_max
    if np.any(np.diff(label_values) != 1):
  File "/home/travis/venv/lib/python3.6/site-packages/numpy/lib/function_base.py", line 1926, in diff
    return a[slice1]-a[slice2]
TypeError: numpy boolean subtract, the `-` operator, is deprecated, use the bitwise_xor, the `^` operator, or the logical_xor function instead.
/home/travis/build/scikit-image/scikit-image/skimage/morphology/watershed.py:229: UnexpectedException
________________________ test_3d_fallback_white_tophat _________________________
    def test_3d_fallback_white_tophat():
        image = np.zeros((7, 7, 7), dtype=bool)
        image[2, 2:4, 2:4] = 1
        image[3, 2:5, 2:5] = 1
        image[4, 3:5, 3:5] = 1
    
        with expected_warnings(['operator.*deprecated|\A\Z']):
>           new_image = grey.white_tophat(image)
skimage/morphology/tests/test_grey.py:161: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
skimage/morphology/misc.py:37: in func_out
    return func(image, selem=selem, *args, **kwargs)
skimage/morphology/grey.py:404: in white_tophat
    out = ndi.white_tophat(image, footprint=selem, output=out)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
input = array([[[False, False, False, False, False, False, False],
        [False, False, False, False, False, False, False],
...se, False, False, False, False, False, False],
        [False, False, False, False, False, False, False]]], dtype=bool)
size = None
footprint = array([[[False, False, False],
        [False,  True, False],
        [False, False, False]],
       [[False,  True, ...e, False]],
       [[False, False, False],
        [False,  True, False],
        [False, False, False]]], dtype=bool)
structure = None
output = array([[[False, False, False, False, False, False, False],
        [False, False, False, False, False, False, False],
...se, False, False, False, False, False, False],
        [False, False, False, False, False, False, False]]], dtype=bool)
mode = 'reflect', cval = 0.0, origin = 0
    def white_tophat(input, size=None, footprint=None, structure=None,
                     output=None, mode="reflect", cval=0.0, origin=0):
        """
        Multi-dimensional white tophat filter.
    
        Parameters
        ----------
        input : array_like
            Input.
        size : tuple of ints
            Shape of a flat and full structuring element used for the filter.
            Optional if `footprint` or `structure` is provided.
        footprint : array of ints, optional
            Positions of elements of a flat structuring element
            used for the white tophat filter.
        structure : array of ints, optional
            Structuring element used for the filter. `structure`
            may be a non-flat structuring element.
        output : array, optional
            An array used for storing the output of the filter may be provided.
        mode : {'reflect', 'constant', 'nearest', 'mirror', 'wrap'}, optional
            The `mode` parameter determines how the array borders are
            handled, where `cval` is the value when mode is equal to
            'constant'. Default is 'reflect'
        cval : scalar, optional
            Value to fill past edges of input if `mode` is 'constant'.
            Default is 0.0.
        origin : scalar, optional
            The `origin` parameter controls the placement of the filter.
            Default is 0.
    
        Returns
        -------
        output : ndarray
            Result of the filter of `input` with `structure`.
    
        See also
        --------
        black_tophat
    
        """
        tmp = grey_erosion(input, size, footprint, structure, None, mode,
                           cval, origin)
        if isinstance(output, numpy.ndarray):
            grey_dilation(tmp, size, footprint, structure, output, mode, cval,
                          origin)
>           return numpy.subtract(input, output, output)
E           TypeError: numpy boolean subtract, the `-` operator, is deprecated, use the bitwise_xor, the `^` operator, or the logical_xor function instead.
../../../venv/lib/python3.6/site-packages/scipy/ndimage/morphology.py:1693: TypeError
________________________ test_3d_fallback_black_tophat _________________________
    def test_3d_fallback_black_tophat():
        image = np.ones((7, 7, 7), dtype=bool)
        image[2, 2:4, 2:4] = 0
        image[3, 2:5, 2:5] = 0
        image[4, 3:5, 3:5] = 0
    
        with expected_warnings(['operator.*deprecated|\A\Z']):
>           new_image = grey.black_tophat(image)
skimage/morphology/tests/test_grey.py:175: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
skimage/morphology/misc.py:37: in func_out
    return func(image, selem=selem, *args, **kwargs)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
image = array([[[ True,  True,  True,  True,  True,  True,  True],
        [ True,  True,  True,  True,  True,  True,  True],
...ue,  True,  True,  True,  True,  True,  True],
        [ True,  True,  True,  True,  True,  True,  True]]], dtype=bool)
selem = array([[[False, False, False],
        [False,  True, False],
        [False, False, False]],
       [[False,  True, ...e, False]],
       [[False, False, False],
        [False,  True, False],
        [False, False, False]]], dtype=bool)
out = array([[[ True,  True,  True,  True,  True,  True,  True],
        [ True,  True,  True,  True,  True,  True,  True],
...ue,  True,  True,  True,  True,  True,  True],
        [ True,  True,  True,  True,  True,  True,  True]]], dtype=bool)
    @default_selem
    def black_tophat(image, selem=None, out=None):
        """Return black top hat of an image.
    
        The black top hat of an image is defined as its morphological closing minus
        the original image. This operation returns the dark spots of the image that
        are smaller than the structuring element. Note that dark spots in the
        original image are bright spots after the black top hat.
    
        Parameters
        ----------
        image : ndarray
            Image array.
        selem : ndarray, optional
            The neighborhood expressed as a 2-D array of 1's and 0's.
            If None, use cross-shaped structuring element (connectivity=1).
        out : ndarray, optional
            The array to store the result of the morphology. If None
            is passed, a new array will be allocated.
    
        Returns
        -------
        out : array, same shape and type as `image`
            The result of the morphological black top hat.
    
        Examples
        --------
        >>> # Change dark peak to bright peak and subtract background
        >>> import numpy as np
        >>> from skimage.morphology import square
        >>> dark_on_grey = np.array([[7, 6, 6, 6, 7],
        ...                          [6, 5, 4, 5, 6],
        ...                          [6, 4, 0, 4, 6],
        ...                          [6, 5, 4, 5, 6],
        ...                          [7, 6, 6, 6, 7]], dtype=np.uint8)
        >>> black_tophat(dark_on_grey, square(3))
        array([[0, 0, 0, 0, 0],
               [0, 0, 1, 0, 0],
               [0, 1, 5, 1, 0],
               [0, 0, 1, 0, 0],
               [0, 0, 0, 0, 0]], dtype=uint8)
    
        """
        if out is image:
            original = image.copy()
        else:
            original = image
        out = closing(image, selem, out=out)
>       out -= original
E       TypeError: numpy boolean subtract, the `-` operator, is deprecated, use the bitwise_xor, the `^` operator, or the logical_xor function instead.
skimage/morphology/grey.py:456: TypeError

[stefanv]

This is going to be problematic for our SciPy 2017 tutorial. @jni Do you think we can patch the current release in the next week or so?

[matthew-brett]

I think the plan is to release numpy 1.13.1 soon, where the error is reverted to deprecation again.

[jni]

@matthew-brett Can "soon" be taken to be synonymous with "before SciPy 2017"? =)

[matthew-brett]

:) - @charris - what are the odds of numpy 1.13.1 being out before Scipy 2017?

[charris]

@matthew-brett Pretty good. I would have done it this week, but there are still a few issues to deal with. Then there is the wheel-build Python 3.6.1 problem. Have we determined a solution to that yet?

[jakirkham]

Also fun fact that I discovered recently. numpy.diff raises this error on boolean arrays as well. So if one calls peak_local_max, which can take a boolean mask, this will fail.

Edit: Hmm...missed that was already noted in the traceback and raised to NumPy as issue ( numpy/numpy#9251 ). Sorry for the noise.

[stefanv]

I guess the question now becomes: will numpy 1.13.2 be out in time for SciPy 2017?

[charris]

@stefanv What's the problem.

[sciunto]

I guess we can close this issue if numpy solve the problem in newer versions.

[stefanv]

That seems correct, thanks @sciunto