Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Rewrite assert np.* tests to use numpy.testing #14377

Merged
merged 1 commit into from Jun 1, 2019

Conversation

timhoffm
Copy link
Member

PR Summary

Use np.testing.assert_allclose() instead of assert np.allclose() and similar because the numpy assertions will give context on what fails whereas pytest cannot introspect np.allclose() and will just notify an overall failure.

@timhoffm timhoffm added this to the v3.2.0 milestone May 30, 2019
@anntzer
Copy link
Contributor

anntzer commented May 30, 2019

Perhaps use assert foo == pytest.approx(bar) (https://docs.pytest.org/en/latest/reference.html#pytest-approx) instead of assert_allclose? This way you get the nice pytest-like syntax, context info on failure, and it's also clearer (to me?) what value is the "reference" (result = approx(reference)).

(Also there's a bunch of "allclose-asserters" (https://docs.scipy.org/doc/numpy/reference/routines.testing.html#asserts: assert_almost_equal, assert_approx_equal, assert_array_almost_equal, assert_allclose, assert_array_almost_equal_nulp, assert_array_max_ulp), I'm sure there may be cases where one clearly wants one of them, but in matplotlib I'd guess most often that was not really thought out, so pytest.approx also just provides a nice, standard way to spell that.)

@timhoffm
Copy link
Member Author

pytest.approx gives reasonable error messages for scalars:

>       assert a == pytest.approx(6)
E       assert 5 == 6 ± 6.0e-06
E        +  where 6 ± 6.0e-06 = <function approx at 0x7f4b00930d08>(6)
E        +    where <function approx at 0x7f4b00930d08> = pytest.approx

but for numpy arrays, they are not quite readable (in particular it does not give the position of the error):

        a = np.array([1, 2, 3])
        b = np.array([1, 2, 4])
>       assert a == pytest.approx(b)
E       assert array([1, 2, 3]) == approx([1 ± 1.0e-06, 2 ± 2.0e-06, 4 ± 4.0e-06])
E        +  where approx([1 ± 1.0e-06, 2 ± 2.0e-06, 4 ± 4.0e-06]) = <function approx at 0x7f12827fcd08>(array([1, 2, 4]))
E        +    where <function approx at 0x7f12827fcd08> = pytest.approx

Therfore, I would just stick with numpy.testing for these cases for now. The changes here are an incremental improvement. We have more numpy.testing calls. They could all be changed together later when there's better introspection into numpy arrays from within pytest.

Semi-related: I'm investigating if there are better ways for exact comparison: pytest-dev/pytest#5347

@NelleV
Copy link
Member

NelleV commented Jun 1, 2019

Thanks @timhoffm !

@NelleV NelleV merged commit 14228f0 into matplotlib:master Jun 1, 2019
@timhoffm timhoffm deleted the numpy-testing branch June 2, 2019 18:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants