New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

UnicodeDecodeError in MPD frontend on non-english locale #88

Closed
wants to merge 2 commits into
base: develop
from

Conversation

3 participants
@bok
Contributor

bok commented Apr 30, 2011

Hi,

my locale is fr_FR.UTF-8 (french). When starting mopidy while MPD was listening on the same address/port, the exception handling code triggers UnicodeDecodeError (because the errno message is in french), and the message is not shown.

This is a fix.

@bok

This comment has been minimized.

Contributor

bok commented Apr 30, 2011

I also fixed 2 similar errors in mopidy-scan when dealing with non-ascii folder/file names.

@adamcik

This comment has been minimized.

Member

adamcik commented Apr 30, 2011

Either I made some bad assumptions when I wrote these tests or 93262a8 isn't a valid fix. Are paths on your system latin-1?

Anyway, tests don't pass on my system, so won't merge yet :) If nosetests passes on your system then the tests are most definitively broken and need to be fixed/updated depending on what is wrong.

Using sys.getfilesystemencoding() instead of assuming anything would probably be better either way.

======================================================================
ERROR: Failure: UnicodeEncodeError ('ascii' codec can't encode characters in position 36-38: ordinal not in range(128))
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/adamcik/dev/mopidy/lib/python2.6/site-packages/nose/loader.py", line 390, in loadTestsFromName
    addr.filename, addr.module)
  File "/home/adamcik/dev/mopidy/lib/python2.6/site-packages/nose/importer.py", line 39, in importFromPath
    return self.importFromDir(dir_path, fqname)
  File "/home/adamcik/dev/mopidy/lib/python2.6/site-packages/nose/importer.py", line 86, in importFromDir
    mod = load_module(part_fqname, fh, filename, desc)
  File "/home/adamcik/dev/mopidy/tests/backends/local/translator_test.py", line 18, in 
    encoded_uri = path_to_uri(encoded_path)
  File "/home/adamcik/dev/mopidy/mopidy/utils/path.py", line 25, in path_to_uri
    path = os.path.join(*map(f, paths))
  File "/home/adamcik/dev/mopidy/mopidy/utils/path.py", line 24, in 
    f = lambda p: p.decode('latin1').encode('utf8')
UnicodeEncodeError: 'ascii' codec can't encode characters in position 36-38: ordinal not in range(128)

======================================================================
ERROR: test_unicode_in_path (tests.utils.path_test.PathToFileURITest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/adamcik/dev/mopidy/tests/utils/path_test.py", line 68, in test_unicode_in_path
    result = path_to_uri(u'/tmp/æøå')
  File "/home/adamcik/dev/mopidy/mopidy/utils/path.py", line 25, in path_to_uri
    path = os.path.join(*map(f, paths))
  File "/home/adamcik/dev/mopidy/mopidy/utils/path.py", line 24, in 
    f = lambda p: p.decode('latin1').encode('utf8')
UnicodeEncodeError: 'ascii' codec can't encode characters in position 5-7: ordinal not in range(128)

======================================================================
FAIL: test_names_are_unicode (tests.utils.path_test.FindFilesTest)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/adamcik/dev/mopidy/tests/utils/path_test.py", line 137, in test_names_are_unicode
    '%s is not unicode object' % repr(name))
AssertionError: '/home/adamcik/dev/mopidy/tests/data/one.m3u' is not unicode object

----------------------------------------------------------------------
Ran 707 tests in 13.740s

FAILED (SKIP=12, errors=2, failures=1)
@bok

This comment has been minimized.

Contributor

bok commented Apr 30, 2011

You are right. I forgot to make the tests (they don't pass). Paths on my system is utf-8
I double-checked my music folder. I found a file with a messed-up filename, surely an artifact from a windows->linux transfer.
93262a8 fixes the problem for me, because I don't seem to have any utf-8 filenames.

Anyway, I will reset the last commit. As you said on IRC, 5a16b2e is still valid.

To adress problems with messed-up filenames, I think mopidy should catch the exception and skip the file. I'll do another pull request soon, and will be extra careful this time.

@bok bok closed this Apr 30, 2011

@adamcik

This comment has been minimized.

Member

adamcik commented May 1, 2011

We could use chardet or UnicodeDammit, that is if we don't mind an extra dependency. However, I like your suggestion about catching the exception, warning the user and skipping the file much better :)

@jodal jodal reopened this May 5, 2011

@jodal

This comment has been minimized.

Member

jodal commented May 5, 2011

Hum, no way to put a pull request in a milestone. Closing again.

@jodal jodal closed this May 5, 2011

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment