New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix POST form submission for Python 3 #222

Merged
merged 1 commit into from Feb 11, 2017

Conversation

Projects
None yet
2 participants
@jwilk
Contributor

jwilk commented Feb 8, 2017

In Python 3, urlencode() returns Unicode string; but urlopen() wants only bytes in data, so it need to be encoded first.

Fixes:

>>> form = lxml.html.fromstring('<form method="POST">', base_url='http://localhost')
>>> lxml.html.submit_form(form)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../lxml/html/__init__.py", line 1119, in submit_form
    return open_http(form.method, url, values)
  File ".../lxml/html/__init__.py", line 1140, in open_http_urllib
    return urlopen(url, data)
  File ".../urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File ".../urllib/request.py", line 464, in open
    req = meth(req)
  File ".../urllib/request.py", line 1183, in do_request_
    raise TypeError(msg)
TypeError: POST data should be bytes or an iterable of bytes. It cannot be of type str.
Fix POST form submission for Python 3
In Python 3, urlencode() returns Unicode string; but urlopen() wants
only bytes in data, so it need to be encoded first.

Fixes:

    >>> form = lxml.html.fromstring('<form method="POST">', base_url='http://localhost')
    >>> lxml.html.submit_form(form)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File ".../lxml/html/__init__.py", line 1119, in submit_form
        return open_http(form.method, url, values)
      File ".../lxml/html/__init__.py", line 1140, in open_http_urllib
        return urlopen(url, data)
      File ".../urllib/request.py", line 163, in urlopen
        return opener.open(url, data, timeout)
      File ".../urllib/request.py", line 464, in open
        req = meth(req)
      File ".../urllib/request.py", line 1183, in do_request_
        raise TypeError(msg)
    TypeError: POST data should be bytes or an iterable of bytes. It cannot be of type str.
@scoder

This comment has been minimized.

Show comment
Hide comment
@scoder

scoder Feb 10, 2017

Member

Thanks. Any reason why you chose ASCII instead of UTF-8 as encoding here?

Member

scoder commented Feb 10, 2017

Thanks. Any reason why you chose ASCII instead of UTF-8 as encoding here?

@jwilk

This comment has been minimized.

Show comment
Hide comment
@jwilk

jwilk Feb 10, 2017

Contributor

urlencode() always returns an ASCII-only string, so ASCII as encoding seemed a natural choice.

Contributor

jwilk commented Feb 10, 2017

urlencode() always returns an ASCII-only string, so ASCII as encoding seemed a natural choice.

@scoder scoder merged commit 6637e88 into lxml:master Feb 11, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment