pytest.approx abs/rel logic seems to be flipped on np.array() #3695

abrammer · 2018-07-18T20:30:17Z

Hopefully, I'm not missing something obvious here.

Absolute and relative tolerance arguments or logic seem to get flipped when the comparison is against a np.array

Think I've tracked it down to this line.

pytest/src/_pytest/python_api.py

Line 214 in f93995e

return ApproxNumpy(actual, self.abs, self.rel, self.nan_ok) == self.expected

abs and rel arguments should be in the other order according to the ApproxBase.__init__

pytest/src/_pytest/python_api.py

Line 46 in f93995e

def __init__(self, expected, rel=None, abs=None, nan_ok=False):

pytest.__version__
'3.6.3'
np.__version__
'1.13.1'

abs_check = pytest.approx(100, abs=0.1, rel=0)
rel_check = pytest.approx(100, abs=0, rel=0.1)
99 == abs_check
# False
np.array(99) == abs_check
# True

99 == rel_check
# True
np.array(99) == rel_check
# False

The text was updated successfully, but these errors were encountered:

pytestbot · 2018-07-18T20:34:28Z

GitMate.io thinks possibly related issues are #3593 (pytest.approx raises TypeError for 0-d numpy array), #3312 (pytest.approx doesn't support comparison between arrays and scalars), #1994 (Supporting arrays for approx), #3482 (Extend pytest.approx to handle strings. ), and #3164 (TypeError with pytest.approx on nested dictionaries).

This PR updates [pytest](https://pypi.org/project/pytest) from **3.6.4** to **3.7.0**. <details> <summary>Changelog</summary> ### 3.7.0 ``` ========================= Deprecations and Removals ------------------------- - `2639 <https://github.com/pytest-dev/pytest/issues/2639>`_: ``pytest_namespace`` has been deprecated. See the documentation for ``pytest_namespace`` hook for suggestions on how to deal with this in plugins which use this functionality. - `3661 <https://github.com/pytest-dev/pytest/issues/3661>`_: Calling a fixture function directly, as opposed to request them in a test function, now issues a ``RemovedInPytest4Warning``. It will be changed into an error in pytest ``4.0``. This is a great source of confusion to new users, which will often call the fixture functions and request them from test functions interchangeably, which breaks the fixture resolution model. Features -------- - `2283 <https://github.com/pytest-dev/pytest/issues/2283>`_: New ``package`` fixture scope: fixtures are finalized when the last test of a *package* finishes. This feature is considered **experimental**, so use it sparingly. - `3576 <https://github.com/pytest-dev/pytest/issues/3576>`_: ``Node.add_marker`` now supports an ``append=True/False`` parameter to determine whether the mark comes last (default) or first. - `3579 <https://github.com/pytest-dev/pytest/issues/3579>`_: Fixture ``caplog`` now has a ``messages`` property, providing convenient access to the format-interpolated log messages without the extra data provided by the formatter/handler. - `3610 <https://github.com/pytest-dev/pytest/issues/3610>`_: New ``--trace`` option to enter the debugger at the start of a test. - `3623 <https://github.com/pytest-dev/pytest/issues/3623>`_: Introduce ``pytester.copy_example`` as helper to do acceptance tests against examples from the project. Bug Fixes --------- - `2220 <https://github.com/pytest-dev/pytest/issues/2220>`_: Fix a bug where fixtures overridden by direct parameters (for example parametrization) were being instantiated even if they were not being used by a test. - `3695 <https://github.com/pytest-dev/pytest/issues/3695>`_: Fix ``ApproxNumpy`` initialisation argument mixup, ``abs`` and ``rel`` tolerances were flipped causing strange comparsion results. Add tests to check ``abs`` and ``rel`` tolerances for ``np.array`` and test for expecting ``nan`` with ``np.array()`` - `980 <https://github.com/pytest-dev/pytest/issues/980>`_: Fix truncated locals output in verbose mode. Improved Documentation ---------------------- - `3295 <https://github.com/pytest-dev/pytest/issues/3295>`_: Correct the usage documentation of ``--last-failed-no-failures`` by adding the missing ``--last-failed`` argument in the presented examples, because they are misleading and lead to think that the missing argument is not needed. Trivial/Internal Changes ------------------------ - `3519 <https://github.com/pytest-dev/pytest/issues/3519>`_: Now a ``README.md`` file is created in ``.pytest_cache`` to make it clear why the directory exists. ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/pytest - Changelog: https://pyup.io/changelogs/pytest/ - Repo: https://github.com/pytest-dev/pytest/issues - Homepage: http://pytest.org </details>

This PR updates [pytest](https://pypi.org/project/pytest) from **3.6.2** to **3.7.1**. <details> <summary>Changelog</summary> ### 3.7.1 ``` ========================= Bug Fixes --------- - `3473 <https://github.com/pytest-dev/pytest/issues/3473>`_: Raise immediately if ``approx()`` is given an expected value of a type it doesn't understand (e.g. strings, nested dicts, etc.). - `3712 <https://github.com/pytest-dev/pytest/issues/3712>`_: Correctly represent the dimensions of an numpy array when calling ``repr()`` on ``approx()``. - `3742 <https://github.com/pytest-dev/pytest/issues/3742>`_: Fix incompatibility with third party plugins during collection, which produced the error ``object has no attribute '_collectfile'``. - `3745 <https://github.com/pytest-dev/pytest/issues/3745>`_: Display the absolute path if ``cache_dir`` is not relative to the ``rootdir`` instead of failing. - `3747 <https://github.com/pytest-dev/pytest/issues/3747>`_: Fix compatibility problem with plugins and the warning code issued by fixture functions when they are called directly. - `3748 <https://github.com/pytest-dev/pytest/issues/3748>`_: Fix infinite recursion in ``pytest.approx`` with arrays in ``numpy<1.13``. - `3757 <https://github.com/pytest-dev/pytest/issues/3757>`_: Pin pathlib2 to ``>=2.2.0`` as we require ``__fspath__`` support. - `3763 <https://github.com/pytest-dev/pytest/issues/3763>`_: Fix ``TypeError`` when the assertion message is ``bytes`` in python 3. ``` ### 3.7.0 ``` ========================= Deprecations and Removals ------------------------- - `2639 <https://github.com/pytest-dev/pytest/issues/2639>`_: ``pytest_namespace`` has been deprecated. See the documentation for ``pytest_namespace`` hook for suggestions on how to deal with this in plugins which use this functionality. - `3661 <https://github.com/pytest-dev/pytest/issues/3661>`_: Calling a fixture function directly, as opposed to request them in a test function, now issues a ``RemovedInPytest4Warning``. It will be changed into an error in pytest ``4.0``. This is a great source of confusion to new users, which will often call the fixture functions and request them from test functions interchangeably, which breaks the fixture resolution model. Features -------- - `2283 <https://github.com/pytest-dev/pytest/issues/2283>`_: New ``package`` fixture scope: fixtures are finalized when the last test of a *package* finishes. This feature is considered **experimental**, so use it sparingly. - `3576 <https://github.com/pytest-dev/pytest/issues/3576>`_: ``Node.add_marker`` now supports an ``append=True/False`` parameter to determine whether the mark comes last (default) or first. - `3579 <https://github.com/pytest-dev/pytest/issues/3579>`_: Fixture ``caplog`` now has a ``messages`` property, providing convenient access to the format-interpolated log messages without the extra data provided by the formatter/handler. - `3610 <https://github.com/pytest-dev/pytest/issues/3610>`_: New ``--trace`` option to enter the debugger at the start of a test. - `3623 <https://github.com/pytest-dev/pytest/issues/3623>`_: Introduce ``pytester.copy_example`` as helper to do acceptance tests against examples from the project. Bug Fixes --------- - `2220 <https://github.com/pytest-dev/pytest/issues/2220>`_: Fix a bug where fixtures overridden by direct parameters (for example parametrization) were being instantiated even if they were not being used by a test. - `3695 <https://github.com/pytest-dev/pytest/issues/3695>`_: Fix ``ApproxNumpy`` initialisation argument mixup, ``abs`` and ``rel`` tolerances were flipped causing strange comparsion results. Add tests to check ``abs`` and ``rel`` tolerances for ``np.array`` and test for expecting ``nan`` with ``np.array()`` - `980 <https://github.com/pytest-dev/pytest/issues/980>`_: Fix truncated locals output in verbose mode. Improved Documentation ---------------------- - `3295 <https://github.com/pytest-dev/pytest/issues/3295>`_: Correct the usage documentation of ``--last-failed-no-failures`` by adding the missing ``--last-failed`` argument in the presented examples, because they are misleading and lead to think that the missing argument is not needed. Trivial/Internal Changes ------------------------ - `3519 <https://github.com/pytest-dev/pytest/issues/3519>`_: Now a ``README.md`` file is created in ``.pytest_cache`` to make it clear why the directory exists. ``` ### 3.6.4 ``` ========================= Bug Fixes --------- - Invoke pytest using ``-mpytest`` so ``sys.path`` does not get polluted by packages installed in ``site-packages``. (`742 <https://github.com/pytest-dev/pytest/issues/742>`_) Improved Documentation ---------------------- - Use ``smtp_connection`` instead of ``smtp`` in fixtures documentation to avoid possible confusion. (`3592 <https://github.com/pytest-dev/pytest/issues/3592>`_) Trivial/Internal Changes ------------------------ - Remove obsolete ``__future__`` imports. (`2319 <https://github.com/pytest-dev/pytest/issues/2319>`_) - Add CITATION to provide information on how to formally cite pytest. (`3402 <https://github.com/pytest-dev/pytest/issues/3402>`_) - Replace broken type annotations with type comments. (`3635 <https://github.com/pytest-dev/pytest/issues/3635>`_) - Pin ``pluggy`` to ``<0.8``. (`3727 <https://github.com/pytest-dev/pytest/issues/3727>`_) ``` ### 3.6.3 ``` ========================= Bug Fixes --------- - Fix ``ImportWarning`` triggered by explicit relative imports in assertion-rewritten package modules. (`3061 <https://github.com/pytest-dev/pytest/issues/3061>`_) - Fix error in ``pytest.approx`` when dealing with 0-dimension numpy arrays. (`3593 <https://github.com/pytest-dev/pytest/issues/3593>`_) - No longer raise ``ValueError`` when using the ``get_marker`` API. (`3605 <https://github.com/pytest-dev/pytest/issues/3605>`_) - Fix problem where log messages with non-ascii characters would not appear in the output log file. (`3630 <https://github.com/pytest-dev/pytest/issues/3630>`_) - No longer raise ``AttributeError`` when legacy marks can't be stored in functions. (`3631 <https://github.com/pytest-dev/pytest/issues/3631>`_) Improved Documentation ---------------------- - The description above the example for ``pytest.mark.skipif`` now better matches the code. (`3611 <https://github.com/pytest-dev/pytest/issues/3611>`_) Trivial/Internal Changes ------------------------ - Internal refactoring: removed unused ``CallSpec2tox ._globalid_args`` attribute and ``metafunc`` parameter from ``CallSpec2.copy()``. (`3598 <https://github.com/pytest-dev/pytest/issues/3598>`_) - Silence usage of ``reduce`` warning in Python 2 (`3609 <https://github.com/pytest-dev/pytest/issues/3609>`_) - Fix usage of ``attr.ib`` deprecated ``convert`` parameter. (`3653 <https://github.com/pytest-dev/pytest/issues/3653>`_) ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/pytest - Changelog: https://pyup.io/changelogs/pytest/ - Repo: https://github.com/pytest-dev/pytest/issues - Homepage: http://pytest.org </details>

This PR updates [pytest](https://pypi.org/project/pytest) from **3.6.4** to **3.7.2**. <details> <summary>Changelog</summary> ### 3.7.2 ``` ========================= Bug Fixes --------- - `3671 <https://github.com/pytest-dev/pytest/issues/3671>`_: Fix ``filterwarnings`` not being registered as a builtin mark. - `3768 <https://github.com/pytest-dev/pytest/issues/3768>`_, `3789 <https://github.com/pytest-dev/pytest/issues/3789>`_: Fix test collection from packages mixed with normal directories. - `3771 <https://github.com/pytest-dev/pytest/issues/3771>`_: Fix infinite recursion during collection if a ``pytest_ignore_collect`` hook returns ``False`` instead of ``None``. - `3774 <https://github.com/pytest-dev/pytest/issues/3774>`_: Fix bug where decorated fixtures would lose functionality (for example ``mock.patch``). - `3775 <https://github.com/pytest-dev/pytest/issues/3775>`_: Fix bug where importing modules or other objects with prefix ``pytest_`` prefix would raise a ``PluginValidationError``. - `3788 <https://github.com/pytest-dev/pytest/issues/3788>`_: Fix ``AttributeError`` during teardown of ``TestCase`` subclasses which raise an exception during ``__init__``. - `3804 <https://github.com/pytest-dev/pytest/issues/3804>`_: Fix traceback reporting for exceptions with ``__cause__`` cycles. Improved Documentation ---------------------- - `3746 <https://github.com/pytest-dev/pytest/issues/3746>`_: Add documentation for ``metafunc.config`` that had been mistakenly hidden. ``` ### 3.7.1 ``` ========================= Bug Fixes --------- - `3473 <https://github.com/pytest-dev/pytest/issues/3473>`_: Raise immediately if ``approx()`` is given an expected value of a type it doesn't understand (e.g. strings, nested dicts, etc.). - `3712 <https://github.com/pytest-dev/pytest/issues/3712>`_: Correctly represent the dimensions of an numpy array when calling ``repr()`` on ``approx()``. - `3742 <https://github.com/pytest-dev/pytest/issues/3742>`_: Fix incompatibility with third party plugins during collection, which produced the error ``object has no attribute '_collectfile'``. - `3745 <https://github.com/pytest-dev/pytest/issues/3745>`_: Display the absolute path if ``cache_dir`` is not relative to the ``rootdir`` instead of failing. - `3747 <https://github.com/pytest-dev/pytest/issues/3747>`_: Fix compatibility problem with plugins and the warning code issued by fixture functions when they are called directly. - `3748 <https://github.com/pytest-dev/pytest/issues/3748>`_: Fix infinite recursion in ``pytest.approx`` with arrays in ``numpy<1.13``. - `3757 <https://github.com/pytest-dev/pytest/issues/3757>`_: Pin pathlib2 to ``>=2.2.0`` as we require ``__fspath__`` support. - `3763 <https://github.com/pytest-dev/pytest/issues/3763>`_: Fix ``TypeError`` when the assertion message is ``bytes`` in python 3. ``` ### 3.7.0 ``` ========================= Deprecations and Removals ------------------------- - `2639 <https://github.com/pytest-dev/pytest/issues/2639>`_: ``pytest_namespace`` has been deprecated. See the documentation for ``pytest_namespace`` hook for suggestions on how to deal with this in plugins which use this functionality. - `3661 <https://github.com/pytest-dev/pytest/issues/3661>`_: Calling a fixture function directly, as opposed to request them in a test function, now issues a ``RemovedInPytest4Warning``. It will be changed into an error in pytest ``4.0``. This is a great source of confusion to new users, which will often call the fixture functions and request them from test functions interchangeably, which breaks the fixture resolution model. Features -------- - `2283 <https://github.com/pytest-dev/pytest/issues/2283>`_: New ``package`` fixture scope: fixtures are finalized when the last test of a *package* finishes. This feature is considered **experimental**, so use it sparingly. - `3576 <https://github.com/pytest-dev/pytest/issues/3576>`_: ``Node.add_marker`` now supports an ``append=True/False`` parameter to determine whether the mark comes last (default) or first. - `3579 <https://github.com/pytest-dev/pytest/issues/3579>`_: Fixture ``caplog`` now has a ``messages`` property, providing convenient access to the format-interpolated log messages without the extra data provided by the formatter/handler. - `3610 <https://github.com/pytest-dev/pytest/issues/3610>`_: New ``--trace`` option to enter the debugger at the start of a test. - `3623 <https://github.com/pytest-dev/pytest/issues/3623>`_: Introduce ``pytester.copy_example`` as helper to do acceptance tests against examples from the project. Bug Fixes --------- - `2220 <https://github.com/pytest-dev/pytest/issues/2220>`_: Fix a bug where fixtures overridden by direct parameters (for example parametrization) were being instantiated even if they were not being used by a test. - `3695 <https://github.com/pytest-dev/pytest/issues/3695>`_: Fix ``ApproxNumpy`` initialisation argument mixup, ``abs`` and ``rel`` tolerances were flipped causing strange comparsion results. Add tests to check ``abs`` and ``rel`` tolerances for ``np.array`` and test for expecting ``nan`` with ``np.array()`` - `980 <https://github.com/pytest-dev/pytest/issues/980>`_: Fix truncated locals output in verbose mode. Improved Documentation ---------------------- - `3295 <https://github.com/pytest-dev/pytest/issues/3295>`_: Correct the usage documentation of ``--last-failed-no-failures`` by adding the missing ``--last-failed`` argument in the presented examples, because they are misleading and lead to think that the missing argument is not needed. Trivial/Internal Changes ------------------------ - `3519 <https://github.com/pytest-dev/pytest/issues/3519>`_: Now a ``README.md`` file is created in ``.pytest_cache`` to make it clear why the directory exists. ``` </details> <details> <summary>Links</summary> - PyPI: https://pypi.org/project/pytest - Changelog: https://pyup.io/changelogs/pytest/ - Homepage: http://pytest.org </details>

pytestbot added the type: bug problem that needs to be addressed label Jul 18, 2018

abrammer mentioned this issue Jul 18, 2018

bugfix in ApproxNumpy initialisation, use keywords for arguments to fix #3696

Merged

abrammer pushed a commit to abrammer/pytest that referenced this issue Jul 24, 2018

add test wrt pytest-dev#3695 checking numpy array tolerance args

514ca6f

nicoddemus closed this as completed Jul 30, 2018

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

pytest.approx abs/rel logic seems to be flipped on np.array() #3695

pytest.approx abs/rel logic seems to be flipped on np.array() #3695

abrammer commented Jul 18, 2018 •

edited

Loading

pytestbot commented Jul 18, 2018

pytest.approx abs/rel logic seems to be flipped on np.array() #3695

pytest.approx abs/rel logic seems to be flipped on np.array() #3695

Comments

abrammer commented Jul 18, 2018 • edited Loading

pytestbot commented Jul 18, 2018

abrammer commented Jul 18, 2018 •

edited

Loading