urlopen breaks when data parameter is used.

[david193]

BPO	11261
Nosy	@birkenfeld, @orsenthil, @ezio-melotti

^{Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.}

Show more details

GitHub fields:

assignee = None
closed_at = <Date 2012-03-15.01:11:55.517>
created_at = <Date 2011-02-20.22:07:29.310>
labels = ['type-bug', 'docs']
title = 'urlopen breaks when data parameter is used.'
updated_at = <Date 2012-03-15.01:11:55.515>
user = 'https://bugs.python.org/david193'

bugs.python.org fields:

activity = <Date 2012-03-15.01:11:55.515>
actor = 'python-dev'
assignee = 'docs@python'
closed = True
closed_date = <Date 2012-03-15.01:11:55.517>
closer = 'python-dev'
components = ['Documentation']
creation = <Date 2011-02-20.22:07:29.310>
creator = 'david193'
dependencies = []
files = []
hgrepos = []
issue_num = 11261
keywords = []
message_count = 5.0
messages = ['128922', '128930', '128935', '128944', '155834']
nosy_count = 6.0
nosy_names = ['georg.brandl', 'orsenthil', 'ezio.melotti', 'docs@python', 'david193', 'python-dev']
pr_nums = []
priority = 'normal'
resolution = 'fixed'
stage = 'resolved'
status = 'closed'
superseder = None
type = 'behavior'
url = 'https://bugs.python.org/issue11261'
versions = ['Python 3.2', 'Python 3.3']

[david193]

The following code works on python 3.1.3 but fails on Python 3.2rc2 (r32rc2:88269, Jan 30 2011, 14:30:28). (I run Mac OS X, version 10.6.6.)

-----------------------------

import urllib, urllib.request, urllib.error, urllib.parse

form = urllib.parse.urlencode({'field1':'Log in'})
try:
	response = urllib.request.urlopen('http://www.google.com/', form)
except urllib.error.HTTPError as exception:
	print (exception)

---

When loaded, the following error messages are generated:

---------------------------------------

Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/urllib/request.py", line 1057, in do_request_
    mv = memoryview(data)
TypeError: cannot make memory view because object does not have the buffer interface

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "<stdin>", line 2, in <module>
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/urllib/request.py", line 138, in urlopen
    return opener.open(url, data, timeout)
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/urllib/request.py", line 364, in open
    req = meth(req)
  File "/Library/Frameworks/Python.framework/Versions/3.2/lib/python3.2/urllib/request.py", line 1062, in do_request_
    data))
ValueError: Content-Length should be specified                                 for iterable data of type <class 'str'> 'field1=Log+in'

---

[orsenthil]

On Sun, Feb 20, 2011 at 10:07:31PM +0000, David Phillips wrote:

The following code works on python 3.1.3 but fails on Python 3.2rc2
(r32rc2:88269, Jan 30 2011, 14:30:28). (I run Mac OS X, version
10.6.6.)

Is that a real world code? (As in used in production)? Because,
things have been a bit tightened in 3.2 and it expects data to be
bytes and throws an exception if it is not bytes.

urlencode will output str and you have explicitly encode it to bytes
(Using the value Accept-Encoding response header) before you send the
data to urlopen. The updated docs reflect this change.

[david193]

Converting the type of my variable "form" from string to bytes did, indeed, allow the code to run, but I have to wonder about changing the inputs to urlopen like this.

The 3.1.3 docs call for the data parameter to be string, and I presume that has been the case all along. Changing the data parameter from string to bytes may be a relatively clean change to make, but this change is going to break a lot of existing code. Are you sure you want to do that?

[birkenfeld]

Quite a few docs still say "string" where in fact bytes are expected in Python 3.x; we're updating these as we go along.

[python-dev]

New changeset fddbe9a59d26 by Senthil Kumaran in branch '3.2':
Fix the wrong urllib exampls which use str for POST data. Closes bpo-11261
http://hg.python.org/cpython/rev/fddbe9a59d26

New changeset 0345dc184e9a by Senthil Kumaran in branch 'default':
cpython:Fix the wrong urllib exampls which use str for POST data. Closes bpo-11261
http://hg.python.org/cpython/rev/0345dc184e9a