"python setup.py sdist upload --show-response" can fail with "UnboundLocalError: local variable 'result' referenced before assignment"

[jcea]

BPO	10367
Nosy	@jcea, @pjeby, @tarekziade, @merwok, @briancurtin, @berkerpeksag
Superseder	bpo-12853: global name 'r' is not defined in upload.py
Files	10367.diff: Lib/distutils/command/upload.py 10367.updated.diff: Patch the multiple arguments to self.announce test_10367.diff: The unit test

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = 'https://github.com/merwok'
closed_at = <Date 2016-01-20.06:27:46.382>
created_at = <Date 2010-11-08.21:36:51.074>
labels = ['easy', 'type-bug', 'library']
title = '"python setup.py sdist upload --show-response" can fail with "UnboundLocalError: local variable \'result\' referenced before assignment"'
updated_at = <Date 2016-01-20.06:27:46.371>
user = 'https://github.com/jcea'

bugs.python.org fields:

activity = <Date 2016-01-20.06:27:46.371>
actor = 'berker.peksag'
assignee = 'eric.araujo'
closed = True
closed_date = <Date 2016-01-20.06:27:46.382>
closer = 'berker.peksag'
components = ['Distutils', 'Distutils2']
creation = <Date 2010-11-08.21:36:51.074>
creator = 'jcea'
dependencies = []
files = ['19690', '19913', '19981']
hgrepos = []
issue_num = 10367
keywords = ['patch', 'easy']
message_count = 24.0
messages = ['120804', '120809', '120824', '120891', '120893', '121695', '121714', '121793', '123142', '123193', '123262', '123271', '123295', '123297', '123301', '123303', '123305', '123306', '123307', '123314', '123665', '129858', '129869', '258646']
nosy_count = 9.0
nosy_names = ['jcea', 'pje', 'tarek', 'eric.araujo', 'brian.curtin', 'Priscila.Manhaes', 'daniel.tavares', 'berker.peksag', 'nilovna']
pr_nums = []
priority = 'normal'
resolution = 'duplicate'
stage = 'resolved'
status = 'closed'
superseder = '12853'
type = 'behavior'
url = 'https://bugs.python.org/issue10367'
versions = ['Python 2.7']

[jcea]

Uploading a new release of one of my libraries to PYPI I got this error:

"""
Submitting dist/bsddb3-5.1.1.tar.gz to http://pypi.python.org/pypi
Upload failed (400): A file named "bsddb3-5.1.1.tar.gz" already exists for  bsddb3-5.1.1. To fix problems with that file you should create a new release.
Traceback (most recent call last):
  File "setup.py", line 5, in <module>
    import setup2
  File "/home/pybsddb/setup2.py", line 415, in <module>
    'Programming Language :: Python :: 3.2',
  File "/usr/local/lib/python2.7/distutils/core.py", line 152, in setup
    dist.run_commands()
  File "/usr/local/lib/python2.7/distutils/dist.py", line 953, in run_commands
    self.run_command(cmd)
  File "/usr/local/lib/python2.7/distutils/dist.py", line 972, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python2.7/distutils/command/upload.py", line 60, in run
    self.upload_file(command, pyversion, filename)
  File "/usr/local/lib/python2.7/distutils/command/upload.py", line 189, in upload_file
    self.announce('-'*75, result.read(), '-'*75)
UnboundLocalError: local variable 'result' referenced before assignment
"""

Checking the code, I see this:

"""
# send the data
try:
result = urlopen(request)
status = result.getcode()
reason = result.msg
except socket.error, e:
self.announce(str(e), log.ERROR)
return
except HTTPError, e:
status = e.code
reason = e.msg

        if status == 200:
            self.announce('Server response (%s): %s' % (status, reason),
                          log.INFO)
        else:
            self.announce('Upload failed (%s): %s' % (status, reason),
                          log.ERROR)
        if self.show_response:
            self.announce('-'*75, result.read(), '-'*75)
"""

Here, if we selected "show_response" *AND* some error happens, "result" will be undefined and the last line will fail.

This bug was introduced in Python 2.7. It used to work correctly under Python 2.6.

I didn't check Python 3.x.

I think this bug is trivial to reproduce: just try to upload any file with no permissions for it, using "--show_response" command line option.

[merwok]

Thanks for the report. Can you paste your message and exact Python version on bpo-9199?

[jcea]

This is not bpo-9199, even covering the same line code.

In this case the problem is an exception leaving a local variable undefined, crashing later.

[pjeby]

To better show what the problem is, here's a change that would fix the problem (albeit in an ugly way):

--- upload.py   2010-07-07 20:16:33.000000000 -0400
+++ /home/pje/upload.new        2010-11-09 14:30:21.000000000 -0500
@@ -167,6 +167,9 @@
 
         request = Request(self.repository, data=body,
                           headers=headers)
+
+        result = None
+
         # send the data
         try:
             result = urlopen(request)
@@ -186,4 +189,4 @@
             self.announce('Upload failed (%s): %s' % (status, reason),
                           log.ERROR)
         if self.show_response:
-            self.announce('-'*75, result.read(), '-'*75)
+            self.announce('-'*75, result.read() if result is not None else 'ERROR', '-'*75)

[pjeby]

Btw, a quick review of the 3.x trunk code for this shows that it does *not* have this problem; this is specific to 2.7 and AFAICT *only* 2.7.

(This problem was introduced in r73436, btw, as of the move to urllib2 vs. httplib.)

[PriscilaManhaes]

Well, I fixed the problem moving the "if" that instances "result" even though it getting HTTPError into the "try".

------------------------------------------------------
# send the data
try:
result = urlopen(request)
status = result.getcode()
reason = result.msg
if self.show_response:
self.announce('-'*75, result.read(), '-'*75)
except socket.error, e:
self.announce(str(e), log.ERROR)
return
except HTTPError, e:
status = e.code
reason = e.msg

        if status == 200:
            self.announce('Server response (%s): %s' % (status, reason),
                          log.INFO)
        else:
            self.announce('Upload failed (%s): %s' % (status, reason),
                          log.ERROR)

---

So now it explains the real error:

python setup.py sdist upload
running sdist
running check
warning: sdist: manifest template 'MANIFEST.in' does not exist (using default file list)

writing manifest file 'MANIFEST'
creating Python-2.7
creating Python-2.7/Lib
creating Python-2.7/Tools
creating Python-2.7/Tools/scripts
making hard links in Python-2.7...
hard linking README -> Python-2.7
'_struct.c' not a regular file -- skipping
hard linking setup.py -> Python-2.7
hard linking Lib/smtpd.py -> Python-2.7/Lib
hard linking Tools/scripts/2to3 -> Python-2.7/Tools/scripts
hard linking Tools/scripts/idle -> Python-2.7/Tools/scripts
hard linking Tools/scripts/pydoc -> Python-2.7/Tools/scripts
Creating tar archive
removing 'Python-2.7' (and everything under it)
running upload
Submitting dist/Python-2.7.tar.gz to http://pypi.python.org/pypi
Upload failed (401): Unauthorized

Is this ok?

[merwok]

Thank you for the fix Priscila. Can you submit it as a diff file? (guidelines on http://www.python.org/dev/patches/)

Tarek: I wonder if we should backport pydoc_server from d2 to test behavior bugs like this one.

[pjeby]

It looks to me as though this patch reintroduces bpo-9199, as it passes multiple arguments to self.announce() once again. The patch needs to be made against the SVN version of Python, not the released version.

[danieltavares]

Here's an updated patch, which fixes the passing of multiple arguments to self.announce, based on Priscila's fix.