Test failures with Python 3 and enabled warnings #5218

Arfrever · 2014-02-25T13:38:24Z

When warnings are enabled (using e.g. PYTHONWARNINGS="d" environmental variable), then some tests fail with Python 3.* due to PendingDeprecationWarnings / ResourceWarnings triggered by tests.
(Some warnings do not result in failures of tests.)

I use IPython 1.2.1.

Result with Python 3.4:

**********************************************************************
IPython test group: IPython.core
...
======================================================================
ERROR: Test we're not loading modules on startup that we shouldn't.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ipython-1.2.1/build/lib/IPython/core/tests/test_interactiveshell.py", line 452, in test_extraneous_loads
    tt.ipexec_validate(self.fname, out)
  File "/tmp/ipython-1.2.1/build/lib/IPython/testing/tools.py", line 267, in ipexec_validate
    (fname, err))
ValueError: Running file '/tmp/tmpm6efqxch.py' produced error: "/usr/lib64/python3.4/imp.py:32: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses\n  PendingDeprecationWarning)\nsys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='a' encoding='UTF-8'>\n"

======================================================================
ERROR: test_startup_ipy (IPython.core.tests.test_profile.ProfileStartupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ipython-1.2.1/build/lib/IPython/testing/decorators.py", line 233, in skipper_func
    return f(*args, **kwargs)
  File "/tmp/ipython-1.2.1/build/lib/IPython/core/tests/test_profile.py", line 117, in test_startup_ipy
    self.validate('test')
  File "/tmp/ipython-1.2.1/build/lib/IPython/core/tests/test_profile.py", line 106, in validate
    tt.ipexec_validate(self.fname, output, '', options=self.options)
  File "/tmp/ipython-1.2.1/build/lib/IPython/testing/tools.py", line 267, in ipexec_validate
    (fname, err))
ValueError: Running file '/tmp/tmpop9foxvj/test.py' produced error: "/usr/lib64/python3.4/imp.py:32: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses\n  PendingDeprecationWarning)\nsys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='a' encoding='UTF-8'>\n"

======================================================================
ERROR: test_startup_py (IPython.core.tests.test_profile.ProfileStartupTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ipython-1.2.1/build/lib/IPython/testing/decorators.py", line 233, in skipper_func
    return f(*args, **kwargs)
  File "/tmp/ipython-1.2.1/build/lib/IPython/core/tests/test_profile.py", line 112, in test_startup_py
    self.validate('123')
  File "/tmp/ipython-1.2.1/build/lib/IPython/core/tests/test_profile.py", line 106, in validate
    tt.ipexec_validate(self.fname, output, '', options=self.options)
  File "/tmp/ipython-1.2.1/build/lib/IPython/testing/tools.py", line 267, in ipexec_validate
    (fname, err))
ValueError: Running file '/tmp/tmpop9foxvj/test.py' produced error: "/usr/lib64/python3.4/imp.py:32: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses\n  PendingDeprecationWarning)\nsys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='a' encoding='UTF-8'>\n"

======================================================================
ERROR: IPython.core.tests.test_run.TestMagicRunSimple.test_obj_del
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.4/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/tmp/ipython-1.2.1/build/lib/IPython/core/tests/test_run.py", line 245, in test_obj_del
    tt.ipexec_validate(self.fname, 'object A deleted', err)
  File "/tmp/ipython-1.2.1/build/lib/IPython/testing/tools.py", line 267, in ipexec_validate
    (fname, err))
ValueError: Running file '/tmp/tmp8eksyb6y.py' produced error: "/usr/lib64/python3.4/imp.py:32: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses\n  PendingDeprecationWarning)\nsys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='a' encoding='UTF-8'>\n"

======================================================================
ERROR: IPython.core.tests.test_run.TestMagicRunSimple.test_tclass
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/usr/lib64/python3.4/site-packages/nose/case.py", line 198, in runTest
    self.test(*self.arg)
  File "/tmp/ipython-1.2.1/build/lib/IPython/testing/decorators.py", line 233, in skipper_func
    return f(*args, **kwargs)
  File "/tmp/ipython-1.2.1/build/lib/IPython/core/tests/test_run.py", line 303, in test_tclass
    tt.ipexec_validate(self.fname, out, err)
  File "/tmp/ipython-1.2.1/build/lib/IPython/testing/tools.py", line 267, in ipexec_validate
    (fname, err))
ValueError: Running file '/tmp/tmpeadhare7.ipy' produced error: "/usr/lib64/python3.4/imp.py:32: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses\n  PendingDeprecationWarning)\nsys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='a' encoding='UTF-8'>\n"

======================================================================
ERROR: Test that `__file__` is set when running `ipython file.ipy`
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ipython-1.2.1/build/lib/IPython/core/tests/test_shellapp.py", line 46, in test_ipy_script_file_attribute
    tt.ipexec_validate(self.fname, self.fname, err)
  File "/tmp/ipython-1.2.1/build/lib/IPython/testing/tools.py", line 267, in ipexec_validate
    (fname, err))
ValueError: Running file '/tmp/tmpeqvkv9v9.ipy' produced error: "/usr/lib64/python3.4/imp.py:32: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses\n  PendingDeprecationWarning)\nsys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='a' encoding='UTF-8'>\n"

======================================================================
ERROR: Test that `__file__` is set when running `ipython file.py`
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ipython-1.2.1/build/lib/IPython/core/tests/test_shellapp.py", line 35, in test_py_script_file_attribute
    tt.ipexec_validate(self.fname, self.fname, err)
  File "/tmp/ipython-1.2.1/build/lib/IPython/testing/tools.py", line 267, in ipexec_validate
    (fname, err))
ValueError: Running file '/tmp/tmprtfc9c2i.py' produced error: "/usr/lib64/python3.4/imp.py:32: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses\n  PendingDeprecationWarning)\nsys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='a' encoding='UTF-8'>\n"

----------------------------------------------------------------------
Ran 392 tests in 34.207s

FAILED (SKIP=7, errors=7)
...
**********************************************************************
IPython test group: IPython.lib
...
======================================================================
FAIL: Test the IPython runner.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ipython-1.2.1/build/lib/IPython/lib/tests/test_irunner.py", line 136, in testIPython
    self._test_runner(runner,source,output)
  File "/tmp/ipython-1.2.1/build/lib/IPython/lib/tests/test_irunner.py", line 46, in _test_runner
    self.fail(message)
nose.proxy.AssertionError: Mismatch in number of lines

Expected:
...
In [1]: print('hello, this is python')
hello, this is python
# some more code
In [2]: x=1;y=2
In [3]: x+y**2
Out[3]: 5
# An example of autocall functionality
In [4]: from math import *
In [5]: autocall 1
Automatic calling is: Smart
In [6]: cos pi
------> cos(pi)
Out[6]: -1.0
In [7]: autocall 0
Automatic calling is: OFF
In [8]: cos pi
   File "<ipython-input-8-6bd7313dd9a9>", line 1
     cos pi
          ^
SyntaxError: invalid syntax
In [9]: cos(pi)
Out[9]: -1.0
In [10]: for i in range(5):
   ....:     print(i)
   ....:
0
1
2
3
4
In [11]: print("that's all folks!")
that's all folks!
In [12]: exit

Got:
...
In [1]: print('hello, this is python')
hello, this is python
# some more code
In [2]: x=1;y=2
In [3]: x+y**2
Out[3]: 5
# An example of autocall functionality
In [4]: from math import *
In [5]: autocall 1
Automatic calling is: Smart
In [6]: cos pi
------> cos(pi)
Out[6]: -1.0
In [7]: autocall 0
Automatic calling is: OFF
In [8]: cos pi
  File "<ipython-input-8-6bd7313dd9a9>", line 1
    cos pi
         ^
SyntaxError: invalid syntax
In [9]: cos(pi)
Out[9]: -1.0
In [10]: for i in range(5):
   ....:     print(i)
   ....: 
0
1
2
3
4
In [11]: print("that's all folks!")
that's all folks!
In [12]: exit
sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='a' encoding='UTF-8'>
    """Fail immediately, with the given message."""
>>  raise self.failureException('Mismatch in number of lines\n\nExpected:\n~~~~~~~~~\nIn [1]: print(\'hello, this is python\')\nhello, this is python\n# some more code\nIn [2]: x=1;y=2\nIn [3]: x+y**2\nOut[3]: 5\n# An example of autocall functionality\nIn [4]: from math import *\nIn [5]: autocall 1\nAutomatic calling is: Smart\nIn [6]: cos pi\n------> cos(pi)\nOut[6]: -1.0\nIn [7]: autocall 0\nAutomatic calling is: OFF\nIn [8]: cos pi\n   File "<ipython-input-8-6bd7313dd9a9>", line 1\n     cos pi\n          ^\nSyntaxError: invalid syntax\nIn [9]: cos(pi)\nOut[9]: -1.0\nIn [10]: for i in range(5):\n   ....:     print(i)\n   ....:\n0\n1\n2\n3\n4\nIn [11]: print("that\'s all folks!")\nthat\'s all folks!\nIn [12]: exit\n\nGot:\n~~~~~~~~~\nIn [1]: print(\'hello, this is python\')\nhello, this is python\n# some more code\nIn [2]: x=1;y=2\nIn [3]: x+y**2\nOut[3]: 5\n# An example of autocall functionality\nIn [4]: from math import *\nIn [5]: autocall 1\nAutomatic calling is: Smart\nIn [6]: cos pi\n------> cos(pi)\nOut[6]: -1.0\nIn [7]: autocall 0\nAutomatic calling is: OFF\nIn [8]: cos pi\n  File "<ipython-input-8-6bd7313dd9a9>", line 1\n    cos pi\n         ^\nSyntaxError: invalid syntax\nIn [9]: cos(pi)\nOut[9]: -1.0\nIn [10]: for i in range(5):\n   ....:     print(i)\n   ....: \n0\n1\n2\n3\n4\nIn [11]: print("that\'s all folks!")\nthat\'s all folks!\nIn [12]: exit\nsys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=\'/dev/null\' mode=\'a\' encoding=\'UTF-8\'>')


----------------------------------------------------------------------
Ran 57 tests in 4.824s

FAILED (failures=1)
...
**********************************************************************
IPython test group: IPython.testing
...
======================================================================
ERROR: Test with only stdout results.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ipython-1.2.1/build/lib/IPython/testing/tests/test_tools.py", line 100, in test_main_path
    tt.ipexec_validate(self.fname, out)
  File "/tmp/ipython-1.2.1/build/lib/IPython/testing/tools.py", line 267, in ipexec_validate
    (fname, err))
ValueError: Running file '/tmp/tmpjwbwrs6d.py' produced error: "/usr/lib64/python3.4/imp.py:32: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses\n  PendingDeprecationWarning)\nsys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='a' encoding='UTF-8'>\n"

======================================================================
ERROR: Test with only stdout results, expecting windows line endings.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ipython-1.2.1/build/lib/IPython/testing/tests/test_tools.py", line 109, in test_main_path2
    tt.ipexec_validate(self.fname, out)
  File "/tmp/ipython-1.2.1/build/lib/IPython/testing/tools.py", line 267, in ipexec_validate
    (fname, err))
ValueError: Running file '/tmp/tmp_nw4s186.py' produced error: "/usr/lib64/python3.4/imp.py:32: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses\n  PendingDeprecationWarning)\nsys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='a' encoding='UTF-8'>\n"

======================================================================
FAIL: Test exception path in exception_validate.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ipython-1.2.1/build/lib/IPython/testing/tests/test_tools.py", line 122, in test_exception_path
    tt.ipexec_validate(self.fname, expected_out=out, expected_err="C\nD")
  File "/tmp/ipython-1.2.1/build/lib/IPython/testing/tools.py", line 264, in ipexec_validate
    nt.assert_equal("\n".join(err.strip().splitlines()), "\n".join(expected_err.strip().splitlines()))
nose.proxy.AssertionError: "/usr/lib64/python3.4/imp.py:32: PendingD[257 chars]-8'>" != 'C\nD'
- /usr/lib64/python3.4/imp.py:32: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
-   PendingDeprecationWarning)
  C
+ D- D
- sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='a' encoding='UTF-8'>
    """Fail immediately, with the given message."""
>>  raise self.failureException('"/usr/lib64/python3.4/imp.py:32: PendingD[257 chars]-8\'>" != \'C\\nD\'\n- /usr/lib64/python3.4/imp.py:32: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module\'s documentation for alternative uses\n-   PendingDeprecationWarning)\n  C\n+ D- D\n- sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=\'/dev/null\' mode=\'a\' encoding=\'UTF-8\'>')


======================================================================
FAIL: Test exception path in exception_validate, expecting windows line endings.
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/ipython-1.2.1/build/lib/IPython/testing/tests/test_tools.py", line 135, in test_exception_path2
    tt.ipexec_validate(self.fname, expected_out=out, expected_err="C\r\nD")
  File "/tmp/ipython-1.2.1/build/lib/IPython/testing/tools.py", line 264, in ipexec_validate
    nt.assert_equal("\n".join(err.strip().splitlines()), "\n".join(expected_err.strip().splitlines()))
nose.proxy.AssertionError: "/usr/lib64/python3.4/imp.py:32: PendingD[257 chars]-8'>" != 'C\nD'
- /usr/lib64/python3.4/imp.py:32: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
-   PendingDeprecationWarning)
  C
+ D- D
- sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name='/dev/null' mode='a' encoding='UTF-8'>
    """Fail immediately, with the given message."""
>>  raise self.failureException('"/usr/lib64/python3.4/imp.py:32: PendingD[257 chars]-8\'>" != \'C\\nD\'\n- /usr/lib64/python3.4/imp.py:32: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module\'s documentation for alternative uses\n-   PendingDeprecationWarning)\n  C\n+ D- D\n- sys:1: ResourceWarning: unclosed file <_io.TextIOWrapper name=\'/dev/null\' mode=\'a\' encoding=\'UTF-8\'>')


----------------------------------------------------------------------
Ran 57 tests in 6.805s

FAILED (SKIP=6, errors=2, failures=2)
�[?1034h/tmp/ipython-1.2.1/build/lib/IPython/core/displayhook.py:279: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmprgbcotln.py' mode='wb'>
  gc.collect()
/tmp/ipython-1.2.1/build/lib/IPython/core/displayhook.py:279: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmpjwbwrs6d.py' mode='wb'>
  gc.collect()
/tmp/ipython-1.2.1/build/lib/IPython/core/displayhook.py:279: ResourceWarning: unclosed file <_io.FileIO name='/tmp/tmp_nw4s186.py' mode='wb'>
  gc.collect()
/tmp/ipython-1.2.1/build/lib/IPython/core/displayhook.py:279: ResourceWarning: unclosed file <_io.TextIOWrapper name='/tmp/tmpt3pv0ct5.py' mode='w' encoding='UTF-8'>
  gc.collect()

The text was updated successfully, but these errors were encountered:

takluyver · 2014-02-25T18:00:18Z

The warnings appear to be a mixture of:

imp is deprecated in favour of importlib: We still have to support Python 2.7 for now, so it makes sense to use a module that's deprecated in Python 3.3.
Not explicitly closing file handles on /dev/null: Who cares?

So I don't think there's any actual problem to fix. I'll see about making the tests more robust.

The extra warnings in the subprocesses were causing test failures. Closes ipythongh-5218

takluyver · 2014-02-25T18:13:19Z

#5222 fixes the issue for me.

Arfrever · 2014-02-25T19:33:30Z

It might be possible to add something to IPython/utils/py3compat.py...

The extra warnings in the subprocesses were causing test failures. Closes ipythongh-5218

jakirkham · 2015-10-27T15:45:16Z

This PR ( #8929 ) will address the ResourceWarning caused by the unclosed devnull filehandle.

minrk added the prio-low label Feb 25, 2014

minrk added this to the 3.0 milestone Feb 25, 2014

takluyver added a commit to takluyver/ipython that referenced this issue Feb 25, 2014

Unset PYTHONWARNINGS envvar before running subprocess tests.

fdd7e67

The extra warnings in the subprocesses were causing test failures. Closes ipythongh-5218

takluyver mentioned this issue Feb 25, 2014

Unset PYTHONWARNINGS envvar before running subprocess tests. #5222

Merged

minrk closed this as completed in #5222 Feb 25, 2014

minrk modified the milestones: 2.0, 3.0 Feb 25, 2014

mattvonrocketstein pushed a commit to mattvonrocketstein/ipython that referenced this issue Nov 3, 2014

Unset PYTHONWARNINGS envvar before running subprocess tests.

7aec79a

The extra warnings in the subprocesses were causing test failures. Closes ipythongh-5218

jakirkham mentioned this issue Oct 27, 2015

Close devnullat exit to avoid a ResourceWarning (Python 3) #8929

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Test failures with Python 3 and enabled warnings #5218

Test failures with Python 3 and enabled warnings #5218

Arfrever commented Feb 25, 2014

takluyver commented Feb 25, 2014

takluyver commented Feb 25, 2014

Arfrever commented Feb 25, 2014

jakirkham commented Oct 27, 2015

Test failures with Python 3 and enabled warnings #5218

Test failures with Python 3 and enabled warnings #5218

Comments

Arfrever commented Feb 25, 2014

takluyver commented Feb 25, 2014

takluyver commented Feb 25, 2014

Arfrever commented Feb 25, 2014

jakirkham commented Oct 27, 2015