New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Search using unicode query strings fails. #129

Closed
bok opened this Issue Jul 31, 2011 · 8 comments

Comments

3 participants
@bok
Contributor

bok commented Jul 31, 2011

example: search any föö

version: develop 0.6 (a89d89d) with spotify backend.

Backtrace:

  Request from [::ffff:127.0.0.1]:33625 to urn:uuid:bfa91f4c-bc4c-4082-865b-e543ec3291c3: search any "föö"
DEBUG    2011-07-31 21:23:15,872 [27354:PykkaActorThread-3] mopidy.backends.spotify.library
  Spotify search query: föö
ERROR    2011-07-31 21:23:15,873 [27354:PykkaActorThread-21] pykka
  Unhandled exception in MpdSession (urn:uuid:bfa91f4c-bc4c-4082-865b-e543ec3291c3):
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pykka/actor.py", line 166, in _run
    response = self._handle_receive(message)
  File "/usr/lib/python2.7/site-packages/pykka/actor.py", line 246, in _handle_receive
    return self.on_receive(message)
  File "/usr/lib/python2.7/site-packages/mopidy/utils/network.py", line 336, in on_receive
    self.on_line_received(line)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/__init__.py", line 87, in on_line_received
    response = self.dispatcher.handle_request(line)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 51, in handle_request
    return self._call_next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 72, in _call_next_filter
    return next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 81, in _catch_mpd_ack_errors_filter
    return self._call_next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 72, in _call_next_filter
    return next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 92, in _authenticate_filter
    return self._call_next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 72, in _call_next_filter
    return next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 114, in _command_list_filter
    response = self._call_next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 72, in _call_next_filter
    return next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 142, in _idle_filter
    response = self._call_next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 72, in _call_next_filter
    return next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 156, in _add_ok_filter
    response = self._call_next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 72, in _call_next_filter
    return next_filter(request, response, filter_chain)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 168, in _call_handler_filter
    response = self._format_response(self._call_handler(request))
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/dispatcher.py", line 176, in _call_handler
    return handler(self.context, **kwargs)
  File "/usr/lib/python2.7/site-packages/mopidy/frontends/mpd/protocol/music_db.py", line 327, in search
    return context.backend.library.search(**query).get().mpd_format()
  File "/usr/lib/python2.7/site-packages/pykka/future.py", line 90, in get
    raise self._value  # pylint: disable = E0702
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
@jodal

This comment has been minimized.

Member

jodal commented Aug 1, 2011

I can't reproduce this using the same revision and either en_DK.utf8 or fr_FR.utf8 as locale. Any ideas?

@bok

This comment has been minimized.

Contributor

bok commented Aug 6, 2011

The develop version of pyspotify is the cause, with the commit mopidy/pyspotify@5b5fbab.

With this commit, pyspotify now allows for unicode object in search queries ; it is backwards-compatible because it also allows str objects to be passed to the function. But for this it does an implicit conversion to unicode, and I assume this is what is failing here.

@adamcik

This comment has been minimized.

Member

adamcik commented Aug 6, 2011

Could you add another comment or update the initial paste with the traceback from mopidy -vv - it should hopefully contain something along the lines of "Exception returned from Foo (urn:uuid:30a1ad23-ba1b-4397-9d8e-3b603c2d820a) to caller: traceback..." which will show us where this is really happening. Using just -v hides the gory details of pykka logging as it was getting to be to verbose most of the time.

@bok

This comment has been minimized.

Contributor

bok commented Aug 6, 2011

DEBUG    2011-08-07 01:28:05,786 [24286:PykkaActorThread-4] pykka
  Exception returned from SpotifyBackend (urn:uuid:273f0e44-26d9-46b0-aae3-cb5327b36ade) to caller:
Traceback (most recent call last):
  File "/usr/lib/python2.7/site-packages/pykka/actor.py", line 166, in _run
    response = self._handle_receive(message)
  File "/usr/lib/python2.7/site-packages/pykka/actor.py", line 237, in _handle_receive
    return callee(*message['args'], **message['kwargs'])
  File "/usr/lib/python2.7/site-packages/mopidy/backends/base/library.py", line 78, in search
    return self.provider.search(**query)
  File "/usr/lib/python2.7/site-packages/mopidy/backends/spotify/library.py", line 58, in search
    self.backend.spotify.search(spotify_query.encode(ENCODING), queue)
  File "/usr/lib/python2.7/site-packages/mopidy/backends/spotify/session_manager.py", line 156, in search
    self.session.search(query, callback)
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
WARNING  2011-08-07 01:28:05,788 [24286:MpdThread] mopidy.frontends.mpd.session
  Received invalid data: 'ascii' codec can't decode byte 0xc3 in position 1: ordinal not in range(128)
@adamcik

This comment has been minimized.

Member

adamcik commented Aug 6, 2011

So something actually goes wrong someplace inside https://github.com/mopidy/pyspotify/blob/develop/src/session.c#L264 then?

@bok

This comment has been minimized.

Contributor

bok commented Aug 7, 2011

Not really. What fails is the implicit conversion of query (which is an utf-8 encoded string) to an unicode object. The implicit conversion tries to use the ascii codec.

I think the fix is to keep the query as unicode (remove .encode(...)) in :

File "/usr/lib/python2.7/site-packages/mopidy/backends/spotify/library.py", line 58, in search
self.backend.spotify.search(spotify_query.encode(ENCODING), queue)
@jodal

This comment has been minimized.

Member

jodal commented Aug 8, 2011

@bok The explanation sounds probable. Can you test that removing .encode() in library.py fixes the problem for you?

@jodal

This comment has been minimized.

Member

jodal commented Aug 12, 2011

As this only applies to pyspotify's develop, Mopidy won't change this before pyspotify 1.4 is released and we can depend on it. 1.4 is probably a bit into the future, so I'm moving this from Mopidy's v0.6 milestone to the v0.7 milestone.

@jodal jodal closed this in 6b7e7d2 Sep 24, 2011

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment