Miscellaneous improvements to approx() #3741

kalekundert · 2018-07-31T07:43:58Z

This PR fixes a couple small issues with approx():

Change the output of repr(approx(np.array(...))) to reflect the dimension of the array being compared. For example, repr(approx([[1, 2], [3, 4]])) would look something like approx([[1±..., 2±...], [3±..., 4±...]]) rather than approx([1±..., 2±..., 3±..., 4±...]). This was supposed to fix Broken output when a multidimensional NumPy array fails to be approximate something #3712, but it turns out that Bug fix #3593 - approx repr in a 0-d numpy array #3615 already fixed it. Still, I think it's nice to have a more accurate repr.
Check for type errors as soon as the approx object is instantiated, rather than waiting until a comparison is made. Fixes 'approx' should error immediately if an invalid parameter is passed #3473. This includes very clear error messages if the user tries to use nested dictionaries or lists, since people seem to assume that this will work. (And maybe it should, but that's a different discussion.)
Improve some comments.

nicoddemus · 2018-07-31T18:33:51Z

Hi @kalekundert, thanks a lot for the PR.

I did not take a good look at it yet, but a cursory glance reminds me that we should add changelog notes to the PR. Feel free to add more than one if you feel like it. 👍

coveralls · 2018-07-31T19:07:32Z

Coverage increased (+0.1%) to 92.605% when pulling a5c0fb7 on kalekundert:approx_misc_tweaks into 2534193 on pytest-dev:master.

nicoddemus · 2018-07-31T23:59:09Z

CHANGELOG.rst

@@ -54,6 +54,9 @@ Bug Fixes
 - `#3695 <https://github.com/pytest-dev/pytest/issues/3695>`_: Fix ``ApproxNumpy`` initialisation argument mixup, ``abs`` and ``rel`` tolerances were flipped causing strange comparsion results.
  Add tests to check ``abs`` and ``rel`` tolerances for ``np.array`` and test for expecting ``nan`` with ``np.array()``

+- `#3712  <https://github.com/pytest-dev/pytest/issues/3712>`_: Correctly represent the dimensions of an numpy array when calling ``repr()`` on ``approx()``.


Sorry, I should have been more clear: you should add changelog files to the changelog/ folder. See the README on https://github.com/pytest-dev/pytest/tree/master/changelog for more instructions. 👍

Nevermind, I did some a small tweak in your branch and fixed the changelog notes myself. 👍 😁

* Hide the internal traceback * Use !r representation instead of !s (the default for {} formatting)

nicoddemus

Awesome work, thank you a lot!

RonnyPfannschmidt · 2018-08-01T05:19:48Z

src/_pytest/python_api.py

+        for x in self.expected.values():
+            if isinstance(x, type(self.expected)):
+                raise TypeError(
+                    "pytest.approx() does not support nested dictionaries, e.g. {!r}".format(


for extra support this should also include the key first and it might help to use "pretty formatting" to split larger dicts to something readable

Good idea, done:

____________________________________ test1 ____________________________________ def test1(): > pytest.approx({'a': 1.2, 'b': {1:2}}) E TypeError: pytest.approx() does not support nested dictionaries: key='b' value={1: 2} E full mapping={'a': 1.2, 'b': {1: 2}} .tmp\foo.py:5: TypeError ____________________________________ test2 ____________________________________ def test2(): > pytest.approx([1,2,3, [1,2,3]]) E TypeError: pytest.approx() does not support nested data structures: [1, 2, 3] at index 3 E full sequence: [1, 2, 3, [1, 2, 3]] .tmp\foo.py:8: TypeError ____________________________________ test3 ____________________________________ def test3(): > pytest.approx('foo') E TypeError: cannot make approximate comparisons to non-numeric values: 'foo' .tmp\foo.py:11: TypeError ____________________________________ test4 ____________________________________ def test4(): > pytest.approx([1,2,3, ['foo']]) E TypeError: pytest.approx() does not support nested data structures: ['foo'] at index 3 E full sequence: [1, 2, 3, ['foo']] .tmp\foo.py:14: TypeError ____________________________________ test5 ____________________________________ def test5(): > pytest.approx({'a': 1.2, 'b': {1:'foo'}}) E TypeError: pytest.approx() does not support nested dictionaries: key='b' value={1: 'foo'} E full mapping={'a': 1.2, 'b': {1: 'foo'}} .tmp\foo.py:17: TypeError ========================== 5 failed in 0.12 seconds ===========================

RonnyPfannschmidt · 2018-08-01T05:21:24Z

testing/python/approx.py

-        np_array = np.array([5.])
-        assert approx(np_array) == 5.0
-        assert repr(approx(np_array)) == string_expected
+        examples = [


can we put the parameter to np.array into a parametrize - the importorskip and wrapping can stay in the test

RonnyPfannschmidt

as extra note i realized that is_numpy_array can be massively simplified by sys.modules.get('numpy') followed by a getattr(...., 'ndarray', None)

also a potential follow-up is support for pypys micronumpy`

RonnyPfannschmidt · 2018-08-01T10:15:43Z

src/_pytest/python_api.py

-        # It might be nice to rewrite this function to account for the
-        # shape of the array...
-        import numpy as np
+        def recursive_map(f, x):


this helper is independent, lets move it out

RonnyPfannschmidt · 2018-08-01T10:16:08Z

src/_pytest/python_api.py

+        for key, value in self.expected.items():
+            if isinstance(value, type(self.expected)):
+                msg = "pytest.approx() does not support nested dictionaries: key={!r} value={!r}\n  full mapping={}"
+                raise TypeError(msg.format(key, value, pprint.pformat(self.expected)))


lovely 👍

RonnyPfannschmidt · 2018-08-01T10:18:16Z

testing/python/approx.py

@@ -439,6 +443,13 @@ def test_foo():
            ["*At index 0 diff: 3 != 4 * {}".format(expected), "=* 1 failed in *="]
        )

+    @pytest.mark.parametrize(
+        "x", [None, "string", ["string"], [[1]], {"key": "string"}, {"key": {"key": 1}}]


i believe test ids would help here

nicoddemus · 2018-08-01T10:34:44Z

as extra note i realized that is_numpy_array can be massively simplified by sys.modules.get('numpy') followed by a getattr(...., 'ndarray', None)

Done as well, let me know if that's what you had in mind. 👍

RonnyPfannschmidt

fabulous 👍

kalekundert · 2018-08-01T16:12:10Z

src/_pytest/python_api.py

+        pass
+
+
+def recursive_map(f, x):


I'm a little hestiant about this being a module-level function, because it only works for nested lists (e.g. not tuples or any other kind of iterable). That's fine for ApproxNumpy.__repr__(), because numpy.tolist() is guaranteed to return nested lists, but this isn't really a generally useful function.

You got a point, but I think it doesn't hurt leaving it there; it is not part of the public API after all. @RonnyPfannschmidt any thoughts?

name it recursive_list_map

Done, renamed it _recursive_list_map as well to make it clear it is an internal API.

kalekundert · 2018-08-01T16:24:05Z

Thanks for all the revisions!

kalekundert added 5 commits July 30, 2018 23:23

Reflect dimension in approx repr for numpy arrays.

7d8688d

Improve docstrings for Approx classes.

ad305e7

approx(): Detect type errors earlier.

cd2085e

Let black reformat the code...

032db15

Fix the unused import.

d024919

Update the changelog.

327fe4c

nicoddemus reviewed Jul 31, 2018

View reviewed changes

nicoddemus added 2 commits July 31, 2018 21:08

Improve error message for invalid types passed to pytest.approx()

bf7c188

* Hide the internal traceback * Use !r representation instead of !s (the default for {} formatting)

Create appropriate CHANGELOG entries

8e2ed76

nicoddemus approved these changes Aug 1, 2018

View reviewed changes

Use {!r} for a few other messages as well

098dca3

RonnyPfannschmidt reviewed Aug 1, 2018

View reviewed changes

nicoddemus added 3 commits August 1, 2018 07:01

Improve error checking messages: add position and use pprint

611d254

Use parametrize in repr test for nd arrays

6e32a1f

Fix 'at' string for non-numeric messages in approx()

5003bae

RonnyPfannschmidt reviewed Aug 1, 2018

View reviewed changes

nicoddemus added 3 commits August 1, 2018 07:28

Simplify is_numpy_array as suggested in review

ad5ddaf

Move recursive_map from local to free function

2a2f888

Use ids for parametrized values in test_expected_value_type_error

43664d7

RonnyPfannschmidt approved these changes Aug 1, 2018

View reviewed changes

kalekundert commented Aug 1, 2018

View reviewed changes

Rename recursive_map -> _recursive_list_map as requested in review

a5c0fb7

nicoddemus merged commit 804fc40 into pytest-dev:master Aug 2, 2018

kalekundert deleted the approx_misc_tweaks branch August 2, 2018 04:55

blueyed mentioned this pull request Aug 25, 2018

Issue #3473 #3474

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Miscellaneous improvements to approx() #3741

Miscellaneous improvements to approx() #3741

kalekundert commented Jul 31, 2018 •

edited

nicoddemus commented Jul 31, 2018

coveralls commented Jul 31, 2018 •

edited

nicoddemus Jul 31, 2018

nicoddemus Aug 1, 2018 •

edited

nicoddemus left a comment

RonnyPfannschmidt Aug 1, 2018

nicoddemus Aug 1, 2018

RonnyPfannschmidt Aug 1, 2018

nicoddemus Aug 1, 2018

RonnyPfannschmidt left a comment

RonnyPfannschmidt Aug 1, 2018

nicoddemus Aug 1, 2018

RonnyPfannschmidt Aug 1, 2018

RonnyPfannschmidt Aug 1, 2018

nicoddemus Aug 1, 2018

nicoddemus commented Aug 1, 2018

RonnyPfannschmidt left a comment

kalekundert Aug 1, 2018

nicoddemus Aug 1, 2018

RonnyPfannschmidt Aug 1, 2018

nicoddemus Aug 1, 2018

kalekundert commented Aug 1, 2018

Miscellaneous improvements to approx() #3741

Miscellaneous improvements to approx() #3741

Conversation

kalekundert commented Jul 31, 2018 • edited

nicoddemus commented Jul 31, 2018

coveralls commented Jul 31, 2018 • edited

Choose a reason for hiding this comment

nicoddemus Aug 1, 2018 • edited

Choose a reason for hiding this comment

nicoddemus left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

RonnyPfannschmidt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicoddemus commented Aug 1, 2018

RonnyPfannschmidt left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

kalekundert commented Aug 1, 2018

kalekundert commented Jul 31, 2018 •

edited

coveralls commented Jul 31, 2018 •

edited

nicoddemus Aug 1, 2018 •

edited