New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cannot play files with non-ascii characters in path (+possible fix) #353

Closed
herrernst opened this Issue Mar 26, 2013 · 6 comments

Comments

2 participants
@herrernst
Contributor

herrernst commented Mar 26, 2013

Hi,
I am not able to play files which have non-ascii characters in them, like the file 11 Keine Macht für Niemand.mp3 (which has the umlaut "ü" in it).
In the tag_cache the file is referenced as 11%20Keine%20Macht%20f%C3%BCr%20Niemand.mp3 (notice the two-byte encoding for ü, "%C3%BC")
If I search for the file e. g. with mpc, I get this result:

mpc search title macht
file:///[snap]/11%20Keine%20Macht%20f%C3%83%C2%BCr%20Niemand.mp3

(notice the 4-byte representation for "ü")
If I start playing, mopidy throws this:

Traceback (most recent call last):
  File "/usr/lib/python2.7/logging/__init__.py", line 850, in emit
    msg = self.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 723, in format
    return fmt.format(record)
  File "/usr/lib/python2.7/logging/__init__.py", line 464, in format
    record.message = record.getMessage()
  File "/usr/lib/python2.7/logging/__init__.py", line 328, in getMessage
    msg = msg % self.args
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 207: ordinal not in range(128)
Logged from file actor.py, line 244
WARNING  Setting GStreamer state to GST_STATE_PLAYING failed
WARNING  Track is not playable: file:///[snap]/11%20Keine%20Macht%20f%C3%83%C2%BCr%20Niemand.mp3

I have prepared a fix which works for me (i. e. reconstruct the file path as it is in the tag_cache, with 2 bytes representing "ü"), but I don't know if the approach would be working in all cases (fwiw, my terminal encoding is set to utf-8).
Seems to be the same problem as in #347.

@jodal

This comment has been minimized.

Member

jodal commented Mar 30, 2013

This is indeed the same issue as #347, so I'm closing it as a duplicate. Thanks for the debugging effort and good bug report!

@jodal jodal closed this Mar 30, 2013

@jodal

This comment has been minimized.

Member

jodal commented Mar 30, 2013

Didn't look close enough at this yesteday. It's similar to #347, but not exactly the same. Reopening.

@jodal jodal reopened this Mar 30, 2013

@jodal

This comment has been minimized.

Member

jodal commented Mar 30, 2013

I've now fixed #347 which also should fix the traceback and UnicodeDecodeError in this issue. Though, broken playback of local files with non-ASCII chars in the file path is still an open issue.

@jodal

This comment has been minimized.

Member

jodal commented Mar 30, 2013

ü encoded as UTF-8 and urlquoted becomes %C3%BC. If you encode it once more as UTF-8 and urlquote it, it becomes %C3%83%C2%BC. In other words, the path is double-encoded as UTF-8.

The tag_cache file is encoded as UTF-8, thus on mopidy/frontends/mpd/translator.py:90 all values read from the tag cache is decoded from UTF-8 to Unicode. At this point, the path is still urlencoded, so this simply converts the path string from bytes to unicode. Then, on line 149, the path is passed through urllib.unquote() which returns a unicode string because it was given a unicode string. We now have a unicode string with UTF-8 bytes in it. When this string is encoded to an UTF-8 bytestring or printed (and thus implicitly encoded to an UTF-8 bytestring) it becomes double-encoded.

In conclusion, as far as I can see your fix in #354 is correct.

@jodal

This comment has been minimized.

Member

jodal commented Mar 30, 2013

Fixed with the merge of #354.

@jodal jodal closed this Mar 30, 2013

@herrernst

This comment has been minimized.

Contributor

herrernst commented Mar 30, 2013

Yes, my primary problem wasn't the error message. I suspected something like a double-encoding, thanks for investigating!

@ghost ghost assigned jodal Mar 30, 2013

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment