Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

some tests fail on aarch64 dawin #7104

Open
natsukium opened this issue Aug 21, 2023 · 4 comments
Open

some tests fail on aarch64 dawin #7104

natsukium opened this issue Aug 21, 2023 · 4 comments
Labels
🐛 Bug 💻 Specific arch failure Failure that occurs for a specific architecture

Comments

@natsukium
Copy link

Description:

It may be due to the initial values in the array, but when I run the test on aarch64-darwin, the following two tests fail.

measure/tests/test_fit.py
_____________________________________________________________________________ test_ellipse_parameter_stability _____________________________________________________________________________

    def test_ellipse_parameter_stability():
        """The fit should be modified so that a > b
        """
    
        for angle in np.arange(0, 180 + 1, 1):
            # generate rotation matrix
            theta = np.deg2rad(angle)
            c = np.cos(theta)
            s = np.sin(theta)
            R = np.array([
                [c, -s],
                [s, c]]
            )
    
            # generate points on ellipse
            t = np.linspace(0, 2 * np.pi, 20)
            a = 100
            b = 50
            points = np.array([a * np.cos(t), b * np.sin(t)])
            points = R @ points
    
            # fit model to points
            ellipse_model = EllipseModel()
            ellipse_model.estimate(points.T)
            _, _, a_prime, b_prime, theta_prime = ellipse_model.params
    
>           assert_almost_equal(theta_prime, theta)

.venv/lib/python3.10/site-packages/skimage/measure/tests/test_fit.py:247: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

args = (2.356194490192345, 0.7853981633974483), kwds = {}

    @wraps(func)
    def inner(*args, **kwds):
        with self._recreate_cm():
>           return func(*args, **kwds)
E           AssertionError: 
E           Arrays are not almost equal to 7 decimals
E            ACTUAL: 2.356194490192345
E            DESIRED: 0.7853981633974483

/nix/store/6xmv46hawz0iqgvzwllmagzqc9pws00d-python3-3.10.12/lib/python3.10/contextlib.py:79: AssertionError
measure/tests/test_moments.py
_____________________________________________________________________ test_analytical_moments_calculation[3-1-float32] _____________________________________________________________________

dtype = <class 'numpy.float32'>, order = 1, ndim = 3

    @pytest.mark.parametrize('dtype', [np.uint8, np.int32, np.float32, np.float64])
    @pytest.mark.parametrize('order', [1, 2, 3, 4])
    @pytest.mark.parametrize('ndim', [2, 3, 4])
    def test_analytical_moments_calculation(dtype, order, ndim):
        if ndim == 2:
            shape = (256, 256)
        elif ndim == 3:
            shape = (64, 64, 64)
        else:
            shape = (16, ) * ndim
        rng = np.random.default_rng(1234)
        if np.dtype(dtype).kind in 'iu':
            x = rng.integers(0, 256, shape, dtype=dtype)
        else:
            x = rng.standard_normal(shape, dtype=dtype)
        # setting center=None will use the analytical expressions
        m1 = moments_central(x, center=None, order=order)
        # providing explicit centroid will bypass the analytical code path
        m2 = moments_central(x, center=centroid(x), order=order)
        # ensure numeric and analytical central moments are close
        thresh = 1e-4 if x.dtype == np.float32 else 1e-9
>       compare_moments(m1, m2, thresh=thresh)

.venv/lib/python3.10/site-packages/skimage/measure/tests/test_moments.py:237: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

m1 = array([[[140.60745,   0.     ],
        [  0.     ,   0.     ]],

       [[  0.     ,   0.     ],
        [  0.     ,   0.     ]]], dtype=float32)
m2 = array([[[ 1.4060745e+02, -9.5214844e-03],
        [ 1.5533447e-02, -1.1226189e+06]],

       [[ 5.2490234e-03, -1.1670678e+06],
        [-8.2869125e+05,  1.7116478e+08]]], dtype=float32)
thresh = 0.0001

    def compare_moments(m1, m2, thresh=1e-8):
        """Compare two moments arrays.
    
        Compares only values in the upper-left triangle of m1, m2 since
        values below the diagonal exceed the specified order and are not computed
        when the analytical computation is used.
    
        Also, there the first-order central moments will be exactly zero with the
        analytical calculation, but will not be zero due to limited floating point
        precision when using a numerical computation. Here we just specify the
        tolerance as a fraction of the maximum absolute value in the moments array.
        """
        m1 = m1.copy()
        m2 = m2.copy()
    
        # make sure location of any NaN values match and then ignore the NaN values
        # in the subsequent comparisons
        nan_idx1 = np.where(np.isnan(m1.ravel()))[0]
        nan_idx2 = np.where(np.isnan(m2.ravel()))[0]
        assert len(nan_idx1) == len(nan_idx2)
        assert np.all(nan_idx1 == nan_idx2)
        m1[np.isnan(m1)] = 0
        m2[np.isnan(m2)] = 0
    
        max_val = np.abs(m1[m1 != 0]).max()
        for orders in itertools.product(*((range(m1.shape[0]),) * m1.ndim)):
            if sum(orders) > m1.shape[0] - 1:
                m1[orders] = 0
                m2[orders] = 0
                continue
            abs_diff = abs(m1[orders] - m2[orders])
            rel_diff = abs_diff / max_val
>           assert rel_diff < thresh
E           assert 0.000110473855 < 0.0001

.venv/lib/python3.10/site-packages/skimage/measure/tests/test_moments.py:50: AssertionError

Way to reproduce:

on Aarch64 darwin

pytest skimage/measure/tests/test_moments.py skimage/measure/tests/test_fit.py

Version information:

>>> import sys; print(sys.version)
3.10.12 (main, Jul 29 2023, 02:25:35) [Clang 11.1.0 ]
>>> import platform; print(platform.platform())
macOS-13.3.1-arm64-arm-64bit
>>> import skimage; print(f'scikit-image version: {skimage.__version__}')
scikit-image version: 0.21.0
>>> import numpy; print(f'numpy version: {numpy.__version__}')
@hmaarrfk
Copy link
Member

I think it is worth mentioning that for test_fit

np.pi - 0.7853981633974483 = 2.356194490192345

For test_moments it seems like the precision might have to be relaxed.

@lagru
Copy link
Member

lagru commented Aug 26, 2023

So is it an orientation issue? Seems #2646 related but might also conflate a few different problems.

@risicle
Copy link

risicle commented Jan 10, 2024

On darwin x86_64, I see this failure too, though as of 0.22.0 we're only seeing it for test_ellipse_parameter_stability.

@lagru lagru added the 💻 Specific arch failure Failure that occurs for a specific architecture label Mar 16, 2024
@lagru
Copy link
Member

lagru commented Mar 16, 2024

This test seems to be problematic / unstable to me judging from other similar issues. I hope that we can maybe address the test or code in question and make it more stable. See #7348.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 Bug 💻 Specific arch failure Failure that occurs for a specific architecture
Projects
None yet
Development

No branches or pull requests

5 participants