Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Python 2.7: tests fail randomly #80

Closed
B4dM4n opened this issue Feb 20, 2020 · 0 comments · Fixed by #81
Closed

Python 2.7: tests fail randomly #80

B4dM4n opened this issue Feb 20, 2020 · 0 comments · Fixed by #81

Comments

@B4dM4n
Copy link

B4dM4n commented Feb 20, 2020

Running python2 setup.py test multiple times yields multiple results. I could observe 3 different error outputs and one success output. Each variant seems to be equally likely.

Error 1 

running test
running egg_info
writing src/decorator.egg-info/PKG-INFO
writing top-level names to src/decorator.egg-info/top_level.txt
writing dependency_links to src/decorator.egg-info/dependency_links.txt
reading manifest file 'src/decorator.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'documentation.pdf'
writing manifest file 'src/decorator.egg-info/SOURCES.txt'
running build_ext
test (tests.test.DocumentationTestCase) ... **********************************************************************
File "/tmp/decorator-4.4.1/src/tests/documentation.py", line 50, in tests.documentation.operation1
Failed example:
    operation1()
Expected:
    operation1 is slow
Got nothing
**********************************************************************
File "/tmp/decorator-4.4.1/src/tests/documentation.py", line 50, in tests.documentation.operation2
Failed example:
    operation2()
Expected:
    operation2 is slow
Got nothing
**********************************************************************
2 items had failures:
   1 of   1 in tests.documentation.operation1
   1 of   1 in tests.documentation.operation2
***Test Failed*** 2 failures.
FAIL
test_singledispatch1 (tests.test.DocumentationTestCase) ... ok
test_singledispatch2 (tests.test.DocumentationTestCase) ... ok
test_add1 (tests.test.ExtraTestCase) ... ok
test_decorator_factory (tests.test.ExtraTestCase) ... ok
test_no_first_arg (tests.test.ExtraTestCase) ... ok
test_qualname (tests.test.ExtraTestCase) ... ok
test_signature (tests.test.ExtraTestCase) ... ok
test_unique_filenames (tests.test.ExtraTestCase) ... ok
test_gen123 (tests.test.GeneratorCallerTestCase) ... ok
test_c_classes (tests.test.TestSingleDispatch) ... ok
test_mro (tests.test.TestSingleDispatch) ... ok
test_mro_conflicts (tests.test.TestSingleDispatch) ... ok
test_register_abc (tests.test.TestSingleDispatch) ... ok
test_register_decorator (tests.test.TestSingleDispatch) ... ok
test_register_error (tests.test.TestSingleDispatch) ... ok
test_simple_overloads (tests.test.TestSingleDispatch) ... ok
test_wrapping_attributes (tests.test.TestSingleDispatch) ... ok

======================================================================
FAIL: test (tests.test.DocumentationTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/decorator-4.4.1/src/tests/test.py", line 81, in test
    self.assertEqual(err, 0)
AssertionError: 2 != 0

----------------------------------------------------------------------
Ran 18 tests in 5.631s

FAILED (failures=1)
Test failed: <unittest.runner.TextTestResult run=18 errors=0 failures=1>
error: Test failed: <unittest.runner.TextTestResult run=18 errors=0 failures=1>

Error 2 

running test
running egg_info
writing src/decorator.egg-info/PKG-INFO
writing top-level names to src/decorator.egg-info/top_level.txt
writing dependency_links to src/decorator.egg-info/dependency_links.txt
reading manifest file 'src/decorator.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'documentation.pdf'
writing manifest file 'src/decorator.egg-info/SOURCES.txt'
running build_ext
test (tests.test.DocumentationTestCase) ... **********************************************************************
File "/tmp/decorator-4.4.1/src/tests/documentation.py", line 50, in tests.documentation.operation1
Failed example:
    operation1()
Expected:
    operation1 is slow
Got nothing
**********************************************************************
1 items had failures:
   1 of   1 in tests.documentation.operation1
***Test Failed*** 1 failures.
FAIL
test_singledispatch1 (tests.test.DocumentationTestCase) ... ok
test_singledispatch2 (tests.test.DocumentationTestCase) ... ok
test_add1 (tests.test.ExtraTestCase) ... ok
test_decorator_factory (tests.test.ExtraTestCase) ... ok
test_no_first_arg (tests.test.ExtraTestCase) ... ok
test_qualname (tests.test.ExtraTestCase) ... ok
test_signature (tests.test.ExtraTestCase) ... ok
test_unique_filenames (tests.test.ExtraTestCase) ... ok
test_gen123 (tests.test.GeneratorCallerTestCase) ... ok
test_c_classes (tests.test.TestSingleDispatch) ... ok
test_mro (tests.test.TestSingleDispatch) ... ok
test_mro_conflicts (tests.test.TestSingleDispatch) ... ok
test_register_abc (tests.test.TestSingleDispatch) ... ok
test_register_decorator (tests.test.TestSingleDispatch) ... ok
test_register_error (tests.test.TestSingleDispatch) ... ok
test_simple_overloads (tests.test.TestSingleDispatch) ... ok
test_wrapping_attributes (tests.test.TestSingleDispatch) ... ok

======================================================================
FAIL: test (tests.test.DocumentationTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/decorator-4.4.1/src/tests/test.py", line 81, in test
    self.assertEqual(err, 0)
AssertionError: 1 != 0

----------------------------------------------------------------------
Ran 18 tests in 5.634s

FAILED (failures=1)
Test failed: <unittest.runner.TextTestResult run=18 errors=0 failures=1>
error: Test failed: <unittest.runner.TextTestResult run=18 errors=0 failures=1>

Error 3 

running test
running egg_info
writing src/decorator.egg-info/PKG-INFO
writing top-level names to src/decorator.egg-info/top_level.txt
writing dependency_links to src/decorator.egg-info/dependency_links.txt
reading manifest file 'src/decorator.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'documentation.pdf'
writing manifest file 'src/decorator.egg-info/SOURCES.txt'
running build_ext
test (tests.test.DocumentationTestCase) ... **********************************************************************
File "/tmp/decorator-4.4.1/src/tests/documentation.py", line 50, in tests.documentation.operation2
Failed example:
    operation2()
Expected:
    operation2 is slow
Got nothing
**********************************************************************
1 items had failures:
   1 of   1 in tests.documentation.operation2
***Test Failed*** 1 failures.
FAIL
test_singledispatch1 (tests.test.DocumentationTestCase) ... ok
test_singledispatch2 (tests.test.DocumentationTestCase) ... ok
test_add1 (tests.test.ExtraTestCase) ... ok
test_decorator_factory (tests.test.ExtraTestCase) ... ok
test_no_first_arg (tests.test.ExtraTestCase) ... ok
test_qualname (tests.test.ExtraTestCase) ... ok
test_signature (tests.test.ExtraTestCase) ... ok
test_unique_filenames (tests.test.ExtraTestCase) ... ok
test_gen123 (tests.test.GeneratorCallerTestCase) ... ok
test_c_classes (tests.test.TestSingleDispatch) ... ok
test_mro (tests.test.TestSingleDispatch) ... ok
test_mro_conflicts (tests.test.TestSingleDispatch) ... ok
test_register_abc (tests.test.TestSingleDispatch) ... ok
test_register_decorator (tests.test.TestSingleDispatch) ... ok
test_register_error (tests.test.TestSingleDispatch) ... ok
test_simple_overloads (tests.test.TestSingleDispatch) ... ok
test_wrapping_attributes (tests.test.TestSingleDispatch) ... ok

======================================================================
FAIL: test (tests.test.DocumentationTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/decorator-4.4.1/src/tests/test.py", line 81, in test
    self.assertEqual(err, 0)
AssertionError: 1 != 0

----------------------------------------------------------------------
Ran 18 tests in 5.632s

FAILED (failures=1)
Test failed: <unittest.runner.TextTestResult run=18 errors=0 failures=1>
error: Test failed: <unittest.runner.TextTestResult run=18 errors=0 failures=1>

Success 

running test
running egg_info
writing src/decorator.egg-info/PKG-INFO
writing top-level names to src/decorator.egg-info/top_level.txt
writing dependency_links to src/decorator.egg-info/dependency_links.txt
reading manifest file 'src/decorator.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching 'documentation.pdf'
writing manifest file 'src/decorator.egg-info/SOURCES.txt'
running build_ext
test (tests.test.DocumentationTestCase) ... ok
test_singledispatch1 (tests.test.DocumentationTestCase) ... ok
test_singledispatch2 (tests.test.DocumentationTestCase) ... ok
test_add1 (tests.test.ExtraTestCase) ... ok
test_decorator_factory (tests.test.ExtraTestCase) ... ok
test_no_first_arg (tests.test.ExtraTestCase) ... ok
test_qualname (tests.test.ExtraTestCase) ... ok
test_signature (tests.test.ExtraTestCase) ... ok
test_unique_filenames (tests.test.ExtraTestCase) ... ok
test_gen123 (tests.test.GeneratorCallerTestCase) ... ok
test_c_classes (tests.test.TestSingleDispatch) ... ok
test_mro (tests.test.TestSingleDispatch) ... ok
test_mro_conflicts (tests.test.TestSingleDispatch) ... ok
test_register_abc (tests.test.TestSingleDispatch) ... ok
test_register_decorator (tests.test.TestSingleDispatch) ... ok
test_register_error (tests.test.TestSingleDispatch) ... ok
test_simple_overloads (tests.test.TestSingleDispatch) ... ok
test_wrapping_attributes (tests.test.TestSingleDispatch) ... ok

----------------------------------------------------------------------
Ran 18 tests in 5.635s

OK

How to reproduce

Dockerfile

FROM debian:buster

RUN apt-get update \
  && apt-get install -y python2 python-setuptools wget \
  && cd /tmp \
  && wget https://files.pythonhosted.org/packages/dc/c3/9d378af09f5737cfd524b844cd2fbb0d2263a35c11d712043daab290144d/decorator-4.4.1.tar.gz \
  && tar xf decorator-4.4.1.tar.gz \
  && rm decorator-4.4.1.tar.gz \
  && apt-get clean \
  && rm -R /var/lib/apt/lists

WORKDIR /tmp/decorator-4.4.1
ENTRYPOINT [ "/usr/bin/python2", "setup.py", "test" ]
for i in {1..10}; do docker run --rm -it <image>; done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Fixed "operation is slow" tests micheles/decorator
1 participant
@B4dM4n