Improper URL encoding in geocoding queries

[ingalls]

A customer is having problems with queries such as Florianópolis, Brazil not returning relevant results.

The query works as expected here: https://www.mapbox.com/bites/00121/

@sgillies @perrygeo Do you mind taking a look at how unicode is being escaped?

[perrygeo]

@ingalls can you give me some more context about where they're seeing the error. Specifically, are they using the python sdk directly or through the mapbox CLI?

Using the python SDK gives be similar results to the bites

>>> import mapbox
>>> result = mapbox.Geocoder().forward("Florianópolis, Brazil").json()
>>> for x in result['features']:
...     print x['text']
Florianópolis
Jardim Florianópolis
Rua Florianopólis
Rua Florianópolis
Rua Florianopolis

But using the CLI, I get an error

$ mapbox geocoding "Florianópolis, Brazil"
/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py:1303: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
  return ''.join(map(quoter, s))
Traceback (most recent call last):
  File "/Users/mperry/env/mapbox/bin/mapbox", line 9, in <module>
    load_entry_point('mapboxcli==0.2.0', 'console_scripts', 'mapbox')()
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/click/core.py", line 716, in __call__
    return self.main(*args, **kwargs)
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/click/core.py", line 696, in main
    rv = self.invoke(ctx)
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/click/core.py", line 1060, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/click/core.py", line 889, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/click/core.py", line 534, in invoke
    return callback(*args, **kwargs)
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/click/decorators.py", line 17, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/mperry/work/mapbox-cli-py/mapboxcli/scripts/geocoding.py", line 101, in geocoding
    q, types=place_type, lat=lat, lon=lon, country=country)
  File "/Users/mperry/work/mapbox-sdk-py/mapbox/services/geocoding.py", line 51, in forward
    dataset=self.name, query=address)
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/uritemplate/template.py", line 127, in expand
    return self._expand(var_dict, False)
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/uritemplate/template.py", line 89, in _expand
    expanded.update(v.expand(expansion))
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/uritemplate/variable.py", line 333, in expand
    expanded = expansion(name, value, opts['explode'], opts['prefix'])
  File "/Users/mperry/env/mapbox/lib/python2.7/site-packages/uritemplate/variable.py", line 284, in _string_expansion
    return quote(value, self.safe)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py", line 1303, in quote
    return ''.join(map(quoter, s))
KeyError: u'\xf3'

[ingalls]

@perrygeo They are using the CLI

[perrygeo]

OK, I'm moving the issue over to mapbox-cli-py. Hopefully a fix and new release soon.

[perrygeo]

The issue is ultimately solved here in the SDK: We need to explicitly encode strings as utf-8 to construct proper URLs.

Why encode here rather than the CLI? Unicode Sandwich

[sgillies]

@ingalls versions 0.3 (CLI) and 0.7 (SDK) are up on PyPI now. Customer needs to pip install -U mapboxcli and will have the fix.

[stuartwan]

👍