UnicodeDecodeError

[chrj]

After upgrading from 0.3.7 to 0.3.8 my tests are failing with UnicodeDecodeError on an URL like:

>>> furl.furl(u"http://www.example.org/?kødpålæg=42")
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/redacted/local/lib/python2.7/site-packages/furl/furl.py", line 826, in __init__
    self.load(url)  # Raises ValueError on invalid url.
  File "/redacted/local/lib/python2.7/site-packages/furl/furl.py", line 851, in load
    self.query.load(tokens.query)
  File "/redacted/local/lib/python2.7/site-packages/furl/furl.py", line 433, in load
    self.params.load(self._items(query))
  File "/redacted/local/lib/python2.7/site-packages/furl/furl.py", line 566, in _items
    items = self._extract_items_from_querystr(items)
  File "/redacted/local/lib/python2.7/site-packages/furl/furl.py", line 596, in _extract_items_from_querystr
    if key.encode('utf8') == urllib.quote_plus(pairstr.encode('utf8')):
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)

I'm running Python 2.7.3 and my sys.stdin.encoding is UTF-8.

[gruns]

Great find. I'll fix this shortly.

Do note, however, that URLs can't contain unicode, as per RFC 1738.

Thus, only alphanumerics, the special characters "$-_.+!*'(),", and
reserved characters used for their reserved purposes may be used
unencoded within a URL.

However, it's furl's duty to do 'the right thing' and coerce a unicode input URL to its encoded ascii form.

[gruns]

This is fixed in furl v0.3.9.

>>> f = furl(u'http://www.example.org/?kødpålæg=42')
>>> f.url
'http://www.example.org/?k%C3%B8dp%C3%A5l%C3%A6g=42'

Update to v0.3.9 with

pip install furl --upgrade

Thank you for bringing this issue to my attention @chrj.

[chrj]

Thank you for the quick response.

We are using furl as a tool for both sanitizing as well as manipulating of user supplied URLs, which is why we sometimes deal with unescaped special characters.

[gruns]

Wonderful to hear.

Don't hesitate to let me know if there's anything else I can do for you.