TypeError in error handling in distutils.dir_util.copy_tree

[jaraco]

BPO	22315
Nosy	@jaraco, @merwok, @PCManticore, @dstufft
Files	7304b9b95438.diff

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/jaraco'
closed_at = <Date 2014-09-01.06:10:34.447>
created_at = <Date 2014-08-31.15:15:05.625>
labels = ['type-bug', 'library']
title = 'TypeError in error handling in distutils.dir_util.copy_tree'
updated_at = <Date 2014-09-01.06:10:34.446>
user = 'https://github.com/jaraco'

bugs.python.org fields:

activity = <Date 2014-09-01.06:10:34.446>
actor = 'berker.peksag'
assignee = 'jaraco'
closed = True
closed_date = <Date 2014-09-01.06:10:34.447>
closer = 'berker.peksag'
components = ['Distutils']
creation = <Date 2014-08-31.15:15:05.625>
creator = 'jaraco'
dependencies = []
files = ['36514']
hgrepos = ['272']
issue_num = 22315
keywords = ['patch']
message_count = 7.0
messages = ['226182', '226183', '226187', '226188', '226189', '226194', '226195']
nosy_count = 5.0
nosy_names = ['jaraco', 'eric.araujo', 'Claudiu.Popa', 'python-dev', 'dstufft']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue22315'
versions = ['Python 3.4', 'Python 3.5']

[jaraco]

This morning, I was running an install of a package on Python 3.4.1 when I encountered this error:

C:\Users\jaraco\projects\jaraco.financial [default tip]> ./setup install
running install
running bdist_egg
running egg_info
writing namespace_packages to jaraco.financial.egg-info\namespace_packages.txt
writing jaraco.financial.egg-info\PKG-INFO
writing top-level names to jaraco.financial.egg-info\top_level.txt
writing requirements to jaraco.financial.egg-info\requires.txt
writing dependency_links to jaraco.financial.egg-info\dependency_links.txt
writing entry points to jaraco.financial.egg-info\entry_points.txt
writing manifest file 'jaraco.financial.egg-info\SOURCES.txt'
installing library code to build\bdist.win-amd64\egg
running install_lib
running build_py
copying jaraco\financial\records.py -> build\lib\jaraco\financial
creating build\bdist.win-amd64\egg
creating build\bdist.win-amd64\egg\jaraco
creating build\bdist.win-amd64\egg\jaraco\financial
copying build\lib\jaraco\financial\ledger.py -> build\bdist.win-amd64\egg\jaraco\financial
copying build\lib\jaraco\financial\qif.py -> build\bdist.win-amd64\egg\jaraco\financial
copying build\lib\jaraco\financial\merchant.py -> build\bdist.win-amd64\egg\jaraco\financial
Traceback (most recent call last):
  File "C:\Program Files\Python34\lib\distutils\dir_util.py", line 126, in copy_tree
    names = os.listdir(src)
OSError: [WinError 59] An unexpected network error occurred: 'build\\lib\\jaraco\\financial\\records.py'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "C:\Users\jaraco\projects\jaraco.financial\setup.py", line 72, in <module>
    setuptools.setup(**setup_params)
  File "C:\Program Files\Python34\lib\distutils\core.py", line 148, in setup
    dist.run_commands()
  File "C:\Program Files\Python34\lib\distutils\dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "C:\Program Files\Python34\lib\distutils\dist.py", line 974, in run_command
    cmd_obj.run()
  File "C:\Program Files\Python34\lib\site-packages\setuptools-5.7-py3.4.egg\setuptools\command\install.py", line 67, in run
  File "C:\Program Files\Python34\lib\site-packages\setuptools-5.7-py3.4.egg\setuptools\command\install.py", line 109, in do_egg_install
  File "C:\Program Files\Python34\lib\distutils\cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "C:\Program Files\Python34\lib\distutils\dist.py", line 974, in run_command
    cmd_obj.run()
  File "C:\Program Files\Python34\lib\site-packages\setuptools-5.7-py3.4.egg\setuptools\command\bdist_egg.py", line 161, in run
  File "C:\Program Files\Python34\lib\site-packages\setuptools-5.7-py3.4.egg\setuptools\command\bdist_egg.py", line 147, in call_command
  File "C:\Program Files\Python34\lib\distutils\cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "C:\Program Files\Python34\lib\distutils\dist.py", line 974, in run_command
    cmd_obj.run()
  File "C:\Program Files\Python34\lib\site-packages\setuptools-5.7-py3.4.egg\setuptools\command\install_lib.py", line 10, in run
  File "C:\Program Files\Python34\lib\distutils\command\install_lib.py", line 111, in install
    outfiles = self.copy_tree(self.build_dir, self.install_dir)
  File "C:\Program Files\Python34\lib\site-packages\setuptools-5.7-py3.4.egg\setuptools\command\install_lib.py", line 38, in copy_tree
  File "C:\Program Files\Python34\lib\distutils\cmd.py", line 357, in copy_tree
    not self.force, dry_run=self.dry_run)
  File "C:\Program Files\Python34\lib\distutils\dir_util.py", line 160, in copy_tree
    verbose=verbose, dry_run=dry_run))
  File "C:\Program Files\Python34\lib\distutils\dir_util.py", line 160, in copy_tree
    verbose=verbose, dry_run=dry_run))
  File "C:\Program Files\Python34\lib\distutils\dir_util.py", line 160, in copy_tree
    verbose=verbose, dry_run=dry_run))
  File "C:\Program Files\Python34\lib\distutils\dir_util.py", line 128, in copy_tree
    (errno, errstr) = e
TypeError: 'OSError' object is not iterable

The original traceback indicates a network error occurred. The secondary error occurs when the error is assumed to be an iterable of two elements. I suspect this issue arises from the conversion of WindowsError to OSError or similar.

I plan to look at this in more depth later.

[jaraco]

On a second attempt, the operation completed successfully, so a transient network error was necessary to elicit the bug.

[jaraco]

I've started work on a test to capture the failure. Patch 7304b9 is attached. Any comments on the test before I proceed with the fix? I haven't been able to run the test, so if someone could do that, it would be most appreciated.

[PCManticore]

A similar issue was fixed a couple of days ago in file_util.py, with f01413758114. It would be nice to have similar tests.

[PCManticore]

I'm referring to the "TypeError: 'OSError' object is not iterable" failure.

[jaraco]

Claudiu, thanks for the tip. That was quite helpful. I did not realize that unittest included a mock implementation. I've updated the test to use the same technique. I've furthermore verified that the test captures the failure and adjusted the implementation to match that expectation.

[python-dev]

New changeset 7304b9b95438 by Jason R. Coombs in branch '3.4':
bpo-22315: Add test to capture the failure.
http://hg.python.org/cpython/rev/7304b9b95438

New changeset 300cd36eb25c by Jason R. Coombs in branch '3.4':
bpo-22315: Use technique outlined in test_file_util
http://hg.python.org/cpython/rev/300cd36eb25c

New changeset 3402813338db by Jason R. Coombs in branch '3.4':
bpo-22315: Provide an actual directory during test invocation.
http://hg.python.org/cpython/rev/3402813338db

New changeset ddb0f90620b7 by Jason R. Coombs in branch '3.4':
bpo-22315: Use an existent directory for 'src' to trigger appropriate behavior.
http://hg.python.org/cpython/rev/ddb0f90620b7

New changeset 75a5cc4ef31c by Jason R. Coombs in branch '3.4':
bpo-22315: Use advertised API for OSError
http://hg.python.org/cpython/rev/75a5cc4ef31c

New changeset 640c575ab3e1 by Jason R. Coombs in branch 'default':
Merge with 3.4; Closes bpo-22315
http://hg.python.org/cpython/rev/640c575ab3e1