Search using unicode query strings fails.

[bok]

example: search any föö

version: develop 0.6 (a89d89d) with spotify backend.

Backtrace:

  Request from [::ffff:127.0.0.1]:33625 to urn:uuid:bfa91f4c-bc4c-4082-865b-e543ec3291c3: search any "föö"
DEBUG    2011-07-31 21:23:15,872 [27354:PykkaActorThread-3] mopidy.backends.spotify.library
  Spotify search query: föö
ERROR    2011-07-31 21:23:15,873 [27354:PykkaActorThread-21] pykka
  Unhandled exception in MpdSession (urn:uuid:bfa91f4c-bc4c-4082-865b-e543ec3291c3):
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pykka/actor.py", line 166, in _run
    response = self._handle_receive(message)
  File "/usr/lib/python2.7/site-packages/pykka/actor.py", line 246, in _handle_receive
    return self.on_receive(message)
  File "/usr/lib/python2.7/site-packages/mopidy/utils/network.py", line 336, in on_receive
    self.on_line_received(line)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/__init__.py", line 87, in on_line_received
    response = self.dispatcher.handle_request(line)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 51, in handle_request
    return self._call_next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 72, in _call_next_filter
    return next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 81, in _catch_mpd_ack_errors_filter
    return self._call_next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 72, in _call_next_filter
    return next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 92, in _authenticate_filter
    return self._call_next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 72, in _call_next_filter
    return next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 114, in _command_list_filter
    response = self._call_next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 72, in _call_next_filter
    return next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 142, in _idle_filter
    response = self._call_next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 72, in _call_next_filter
    return next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 156, in _add_ok_filter
    response = self._call_next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 72, in _call_next_filter
    return next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 168, in _call_handler_filter
    response = self._format_response(self._call_handler(request))
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 176, in _call_handler
    return handler(self.context, **kwargs)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/protocol/music_db.py", line 327, in search
    return context.backend.library.search(**query).get().mpd_format()
  File "/usr/lib/python2.7/site-packages/pykka/future.py", line 90, in get
    raise self._value  # pylint: disable = E0702
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)

[jodal]

I can't reproduce this using the same revision and either en_DK.utf8 or fr_FR.utf8 as locale. Any ideas?

[bok]

The develop version of pyspotify is the cause, with the commit jodal/pyspotify@5b5fbab.

With this commit, pyspotify now allows for unicode object in search queries ; it is backwards-compatible because it also allows str objects to be passed to the function. But for this it does an implicit conversion to unicode, and I assume this is what is failing here.

[adamcik]

Could you add another comment or update the initial paste with the traceback from mopidy -vv - it should hopefully contain something along the lines of "Exception returned from Foo (urn:uuid:30a1ad23-ba1b-4397-9d8e-3b603c2d820a) to caller: traceback..." which will show us where this is really happening. Using just -v hides the gory details of pykka logging as it was getting to be to verbose most of the time.

[bok]

DEBUG    2011-08-07 01:28:05,786 [24286:PykkaActorThread-4] pykka
  Exception returned from SpotifyBackend (urn:uuid:273f0e44-26d9-46b0-aae3-cb5327b36ade) to caller:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pykka/actor.py", line 166, in _run
    response = self._handle_receive(message)
  File "/usr/lib/python2.7/site-packages/pykka/actor.py", line 237, in _handle_receive
    return callee(*message['args'], **message['kwargs'])
  File "/usr/lib/python2.7/site-packages/mopidy/backends/base/library.py", line 78, in search
    return self.provider.search(**query)
  File "/usr/lib/python2.7/site-packages/mopidy/backends/spotify/library.py", line 58, in search
    self.backend.spotify.search(spotify_query.encode(ENCODING), queue)
  File "/usr/lib/python2.7/site-packages/mopidy/backends/spotify/session_manager.py", line 156, in search
    self.session.search(query, callback)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
WARNING  2011-08-07 01:28:05,788 [24286:MpdThread] mopidy.frontends.mpd.session
  Received invalid data: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)

[adamcik]

So something actually goes wrong someplace inside https://github.com/mopidy/pyspotify/blob/develop/src/session.c#L264 then?

[bok]

Not really. What fails is the implicit conversion of query (which is an utf-8 encoded string) to an unicode object. The implicit conversion tries to use the ascii codec.

I think the fix is to keep the query as unicode (remove .encode(...)) in :

File "/usr/lib/python2.7/site-packages/mopidy/backends/spotify/library.py", line 58, in search
self.backend.spotify.search(spotify_query.encode(ENCODING), queue)

[jodal]

@bok The explanation sounds probable. Can you test that removing .encode() in library.py fixes the problem for you?

[jodal]

As this only applies to pyspotify's develop, Mopidy won't change this before pyspotify 1.4 is released and we can depend on it. 1.4 is probably a bit into the future, so I'm moving this from Mopidy's v0.6 milestone to the v0.7 milestone.