Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix POST form submission for Python 3 #222

Merged
merged 1 commit into from
Feb 11, 2017
Merged

Fix POST form submission for Python 3 #222

merged 1 commit into from
Feb 11, 2017

Conversation

jwilk
Copy link
Contributor

@jwilk jwilk commented Feb 8, 2017

In Python 3, urlencode() returns Unicode string; but urlopen() wants only bytes in data, so it need to be encoded first.

Fixes:

>>> form = lxml.html.fromstring('<form method="POST">', base_url='http://localhost')
>>> lxml.html.submit_form(form)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File ".../lxml/html/__init__.py", line 1119, in submit_form
    return open_http(form.method, url, values)
  File ".../lxml/html/__init__.py", line 1140, in open_http_urllib
    return urlopen(url, data)
  File ".../urllib/request.py", line 163, in urlopen
    return opener.open(url, data, timeout)
  File ".../urllib/request.py", line 464, in open
    req = meth(req)
  File ".../urllib/request.py", line 1183, in do_request_
    raise TypeError(msg)
TypeError: POST data should be bytes or an iterable of bytes. It cannot be of type str.

In Python 3, urlencode() returns Unicode string; but urlopen() wants
only bytes in data, so it need to be encoded first.

Fixes:

    >>> form = lxml.html.fromstring('<form method="POST">', base_url='http://localhost')
    >>> lxml.html.submit_form(form)
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
      File ".../lxml/html/__init__.py", line 1119, in submit_form
        return open_http(form.method, url, values)
      File ".../lxml/html/__init__.py", line 1140, in open_http_urllib
        return urlopen(url, data)
      File ".../urllib/request.py", line 163, in urlopen
        return opener.open(url, data, timeout)
      File ".../urllib/request.py", line 464, in open
        req = meth(req)
      File ".../urllib/request.py", line 1183, in do_request_
        raise TypeError(msg)
    TypeError: POST data should be bytes or an iterable of bytes. It cannot be of type str.
@scoder
Copy link
Member

scoder commented Feb 10, 2017

Thanks. Any reason why you chose ASCII instead of UTF-8 as encoding here?

@jwilk
Copy link
Contributor Author

jwilk commented Feb 10, 2017

urlencode() always returns an ASCII-only string, so ASCII as encoding seemed a natural choice.

@scoder scoder merged commit 6637e88 into lxml:master Feb 11, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants