Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TestCall.test_non_root_params_not_equal fails #115

Closed
felixonmars opened this issue May 9, 2019 · 12 comments

Comments

@felixonmars
Copy link
Contributor

commented May 9, 2019

Not sure what was the difference, but the exception doesn't raise here:

=================================== FAILURES ===================================
___________________ TestCall.test_non_root_params_not_equal ____________________

self = <testfixtures.tests.test_mock.TestCall object at 0x7fe06a217a58>

    def test_non_root_params_not_equal(self):
        self.check_raises(
            call.foo(x=1).bar(),
            call.foo(x=2).bar(),
>           '\n'
            "'call.foo(x=1)'\n"
            '!=\n'
            "'call.foo(x=2)'"
        )

testfixtures/tests/test_mock.py:31:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <testfixtures.tests.test_mock.TestCall object at 0x7fe06a217a58>
x = call.foo().bar(), y = call.foo().bar()
message = "\n'call.foo(x=1)'\n!=\n'call.foo(x=2)'", regex = None
compare = <function compare at 0x7fe06c0076a8>, kw = {}
args = [call.foo().bar(), call.foo().bar()], value = call.foo().bar()

    def check_raises(self, x=marker, y=marker, message=None, regex=None,
                     compare=compare, **kw):
        args = []
        for value in x, y:
            if value is not marker:
                args.append(value)
        try:
            compare(*args, **kw)
        except Exception as e:
            if not isinstance(e, AssertionError):  # pragma: no cover
                raise
            actual = hexsub(e.args[0])
            if message is not None:
                # handy for debugging, but can't be relied on for tests!
                compare(actual, expected=message, show_whitespace=True)
                assert actual == message
            else:
                if not regex.match(actual):  # pragma: no cover
                    raise AssertionError(
                        '%r did not match %r' % (actual, regex.pattern)
                    )
        else:
>           raise AssertionError('No exception raised!')
E           AssertionError: No exception raised!

testfixtures/tests/test_compare.py:62: AssertionError
==================== 1 failed, 1095 passed in 3.42 seconds =====================

Tested with testfixtures 6.8.2, platform linux -- Python 3.7.3, pytest-4.4.2, py-1.8.0, pluggy-0.11.0. The test passes with Python 2.7 and the rest are the same.

@cjw296

This comment has been minimized.

Copy link
Member

commented May 10, 2019

Please can you explain exactly what you're running and in what environment?
At a higher level, what's most important about what you're looking to achieve?

Most recent CI that's relevant here:
https://circleci.com/gh/Simplistix/testfixtures/1368
https://circleci.com/gh/Simplistix/testfixtures/1369

@felixonmars

This comment has been minimized.

Copy link
Contributor Author

commented May 10, 2019

I am running the test suite with pytest-runner while packaging for Arch.

@cjw296

This comment has been minimized.

Copy link
Member

commented May 10, 2019

You're gonna have to give me a lot more context ;-)
(like a .travis.yaml or a shell session copy'n'paste)

@cjw296

This comment has been minimized.

Copy link
Member

commented May 21, 2019

@felixonmars - closing until you can provide steps to reproduce.

@cjw296 cjw296 closed this May 21, 2019
@felixonmars

This comment has been minimized.

Copy link
Contributor Author

commented May 21, 2019

Sorry for the delay! (I was on a vacation.)

Here is a shell log with latest git master:

$ python setup.py pytest
running pytest
running egg_info
writing testfixtures.egg-info/PKG-INFO
writing dependency_links to testfixtures.egg-info/dependency_links.txt
writing requirements to testfixtures.egg-info/requires.txt
writing top-level names to testfixtures.egg-info/top_level.txt
writing manifest file 'testfixtures.egg-info/SOURCES.txt'
running build_ext
========================================================== test session starts ==========================================================
platform linux -- Python 3.7.3, pytest-4.4.2, py-1.8.0, pluggy-0.11.0
Django settings: testfixtures.tests.test_django.settings (from ini file)
benchmark: 3.2.2 (defaults: timer=time.perf_counter disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/home/felix/projects/testfixtures/.hypothesis/examples')
rootdir: /home/felix/projects/testfixtures, inifile: setup.cfg
plugins: requests-mock-1.6.0, xprocess-0.12.1, xdist-1.28.0, tornado-0.7.0, timeout-1.3.3, mock-1.10.4, httpbin-1.0.0, forked-1.0.2, flake8-1.0.4, django-3.4.7, cov-2.7.1, benchmark-3.2.2, asyncio-0.10.0, aiohttp-0.3.0, hypothesis-4.23.6, flaky-3.5.3, celery-4.2.2, case-1.5.3, aspectlib-1.4.2
collected 1096 items

docs/comparing.txt ............................................................................                                   [  6%]
docs/components.txt ..............                                                                                                [  8%]
docs/datetime.txt ........................................................                                                        [ 13%]
docs/django.txt ..........                                                                                                        [ 14%]
docs/exceptions.txt ................                                                                                              [ 15%]
docs/files.txt ..................................................................................                                 [ 23%]
docs/installation.txt ..                                                                                                          [ 23%]
docs/logging.txt .......................................................                                                          [ 28%]
docs/mocking.txt ..........................................                                                                       [ 32%]
docs/streams.txt ........                                                                                                         [ 32%]
docs/utilities.txt ...............                                                                                                [ 34%]
docs/warnings.txt ..................                                                                                              [ 35%]
testfixtures/tests/configparser-read.txt .....                                                                                    [ 36%]
testfixtures/tests/configparser-write.txt ....                                                                                    [ 36%]
testfixtures/tests/directory-contents.txt ....                                                                                    [ 37%]
testfixtures/tests/test_compare.py .............................................................................................. [ 45%]
.........................................................                                                                         [ 50%]
testfixtures/tests/test_comparison.py ...................................................                                         [ 55%]
testfixtures/tests/test_components.py .                                                                                           [ 55%]
testfixtures/tests/test_date.py .......................                                                                           [ 57%]
testfixtures/tests/test_datetime.py ..........................................                                                    [ 61%]
testfixtures/tests/test_diff.py ..                                                                                                [ 61%]
testfixtures/tests/test_generator.py ..                                                                                           [ 61%]
testfixtures/tests/test_log_capture.py .................                                                                          [ 63%]
testfixtures/tests/test_logcapture.py .............................                                                               [ 66%]
testfixtures/tests/test_mock.py ..F..........                                                                                     [ 67%]
testfixtures/tests/test_outputcapture.py .........                                                                                [ 68%]
testfixtures/tests/test_popen.py .............................................                                                    [ 72%]
testfixtures/tests/test_popen_docs.py ............                                                                                [ 73%]
testfixtures/tests/test_rangecomparison.py .........................................                                              [ 77%]
testfixtures/tests/test_replace.py ................................                                                               [ 80%]
testfixtures/tests/test_replacer.py .........                                                                                     [ 80%]
testfixtures/tests/test_roundcomparison.py ......................................                                                 [ 84%]
testfixtures/tests/test_should_raise.py ......................................                                                    [ 87%]
testfixtures/tests/test_shouldwarn.py .............                                                                               [ 88%]
testfixtures/tests/test_stringcomparison.py .........                                                                             [ 89%]
testfixtures/tests/test_sybil.py ........                                                                                         [ 90%]
testfixtures/tests/test_tempdir.py ........                                                                                       [ 91%]
testfixtures/tests/test_tempdirectory.py ..................................                                                       [ 94%]
testfixtures/tests/test_time.py .............................                                                                     [ 96%]
testfixtures/tests/test_twisted.py ........                                                                                       [ 97%]
testfixtures/tests/test_wrap.py ..............                                                                                    [ 98%]
testfixtures/tests/test_django/test_compare.py .........                                                                          [ 99%]
testfixtures/tests/test_django/test_shouldraise.py ..                                                                             [100%]

=============================================================== FAILURES ================================================================
________________________________________________ TestCall.test_non_root_params_not_equal ________________________________________________

self = <testfixtures.tests.test_mock.TestCall object at 0x7fffdd25dc18>

    def test_non_root_params_not_equal(self):
        self.check_raises(
            call.foo(x=1).bar(),
            call.foo(x=2).bar(),
>           '\n'
            "'call.foo(x=1)'\n"
            '!=\n'
            "'call.foo(x=2)'"
        )

testfixtures/tests/test_mock.py:31:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

self = <testfixtures.tests.test_mock.TestCall object at 0x7fffdd25dc18>, x = call.foo().bar(), y = call.foo().bar()
message = "\n'call.foo(x=1)'\n!=\n'call.foo(x=2)'", regex = None, compare = <function compare at 0x7fffde493730>, kw = {}
args = [call.foo().bar(), call.foo().bar()], value = call.foo().bar()

    def check_raises(self, x=marker, y=marker, message=None, regex=None,
                     compare=compare, **kw):
        args = []
        for value in x, y:
            if value is not marker:
                args.append(value)
        try:
            compare(*args, **kw)
        except Exception as e:
            if not isinstance(e, AssertionError):  # pragma: no cover
                raise
            actual = hexsub(e.args[0])
            if message is not None:
                # handy for debugging, but can't be relied on for tests!
                compare(actual, expected=message, show_whitespace=True)
                assert actual == message
            else:
                if not regex.match(actual):  # pragma: no cover
                    raise AssertionError(
                        '%r did not match %r' % (actual, regex.pattern)
                    )
        else:
>           raise AssertionError('No exception raised!')
E           AssertionError: No exception raised!

testfixtures/tests/test_compare.py:62: AssertionError
===Flaky Test Report===


===End Flaky Test Report===
================================================= 1 failed, 1095 passed in 4.63 seconds =================================================
@cjw296 cjw296 reopened this May 21, 2019
@cjw296

This comment has been minimized.

Copy link
Member

commented May 21, 2019

Okay, but python setup.py pytest isn't something I provide, so what is that doing? More importantly, what environment is it doing it in? It's not going to be possible to make progress on this unless I can get an example list of every package that's installed in that environment.

@felixonmars

This comment has been minimized.

Copy link
Contributor Author

commented May 21, 2019

setup.py pytest invokes pytest-runner, which helps to run under the source tree without the package installed. It should not be the problem because using python -m pytest produces exactly the same output.

The environment is under Arch Linux with the following packages installed:

Packages (78) libnsl-1.2.0-1  python-3.7.3-1  python-apipkg-1.5-1  python-appdirs-1.4.3-2  python-atomicwrites-1.3.0-1
              python-attrs-19.1.0-1  python-automat-0.7.0-2  python-click-7.0-1  python-constantly-15.1.0-3  python-hyperlink-19.0.0-1
              python-idna-2.8-1  python-incremental-17.5.0-2  python-iniconfig-1.0.0-2  python-more-itertools-4.3.0-1
              python-packaging-19.0-1  python-pbr-5.2.0-1  python-pluggy-0.11.0-1  python-py-1.8.0-1  python-pyhamcrest-1.9.0-3
              python-pyparsing-2.4.0-1  python-pytest-4.4.2-1  python-pytz-2019.1-1  python-six-1.12.0-1  python-sqlparse-0.2.4-2
              python-zope-deferredimport-4.3-2  python-zope-deprecation-4.4.0-1  python-zope-event-4.4-1  python-zope-hookable-4.2.0-2
              python-zope-interface-4.6.0-1  python-zope-proxy-4.3.1-1  python2-2.7.16-1  python2-apipkg-1.5-1  python2-appdirs-1.4.3-2
              python2-atomicwrites-1.3.0-1  python2-attrs-19.1.0-1  python2-automat-0.7.0-2  python2-click-7.0-1
              python2-constantly-15.1.0-3  python2-funcsigs-1.0.2-2  python2-hyperlink-19.0.0-1  python2-idna-2.8-1
              python2-incremental-17.5.0-2  python2-iniconfig-1.0.0-2  python2-more-itertools-4.3.0-1  python2-packaging-19.0-1
              python2-pathlib-1.0.1-4  python2-pathlib2-2.3.3-1  python2-pbr-5.2.0-1  python2-pluggy-0.11.0-1  python2-py-1.8.0-1
              python2-pyhamcrest-1.9.0-3  python2-pyparsing-2.4.0-1  python2-pytest-4.4.2-1  python2-pytz-2019.1-1
              python2-scandir-1.10.0-1  python2-six-1.12.0-1  python2-zope-deferredimport-4.3-2  python2-zope-deprecation-4.4.0-1
              python2-zope-event-4.4-1  python2-zope-hookable-4.2.0-2  python2-zope-interface-4.6.0-1  python2-zope-proxy-4.3.1-1
              python-django-2.2.1-1  python-mock-2.0.0-4  python-pytest-django-3.4.7-1  python-pytest-runner-4.4-1
              python-setuptools-1:41.0.1-1  python-sybil-1.2.0-1  python-twisted-18.9.0-1  python-zope-component-4.5-1
              python2-django-1.11.20-1  python2-mock-2.0.0-4  python2-pytest-django-3.4.7-1  python2-pytest-runner-4.4-1
              python2-setuptools-1:41.0.1-1  python2-sybil-1.2.0-1  python2-twisted-18.9.0-1  python2-zope-component-4.5-1

The python2-* ones are irrelevant as they are for building with Python 2.7.

@cjw296

This comment has been minimized.

Copy link
Member

commented May 22, 2019

I'm afraid I know nothing about Arch Linux, and your information is not really helping me understand what python environment you're running python -m pytestin. If this were a virtualenv, I'd be asking you for the output of pip --freeze...

@felixonmars

This comment has been minimized.

Copy link
Contributor Author

commented May 22, 2019

I see... Since this first appears in 6.8.2, I did a bisect to find that it first appears in 62031b5

After further investigation, it is because I have mock installed (the backported version) and at 2.0.0, and also Python 3.7.3 (sys.version_info[:3] > (3, 7, 1)), so the parent_name part falls into the '_mock_parent' path while mock is using the backported version.

The test passes now with mock removed from the test environment, but I guess it's still a bug that will reproduce when mock is present.

@cjw296

This comment has been minimized.

Copy link
Member

commented May 22, 2019

Right, what happens if you use the latest mock backport, rather than one that's 3 years old? ;-)
https://pypi.org/project/mock/3.0.5/

@felixonmars

This comment has been minimized.

Copy link
Contributor Author

commented May 22, 2019

Works fine with new mock installed :)

@cjw296

This comment has been minimized.

Copy link
Member

commented May 22, 2019

Cool :-)

@cjw296 cjw296 closed this May 22, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
2 participants
You can’t perform that action at this time.