Use kitchen for unicode stuff #12

Closed
ralphbean opened this Issue Feb 20, 2014 · 0 comments

Comments

Projects
None yet
1 participant
Contributor

ralphbean commented Feb 20, 2014

Saw this in the logs:

[2014-02-20 10:55:55][summershum    INFO] Ingesting u'sfk-1.7.0.tar.gz'
[2014-02-20 10:55:56][summershum   ERROR] 'utf8' codec can't decode byte 0xe4 in position 54: invalid continuation byte
Traceback (most recent call last):
  File "/usr/lib/python2.6/site-packages/summershum/core.py", line 30, in ingest
    summershum.utils.calculate_sums(session, msg, tmpdir)
  File "/usr/lib/python2.6/site-packages/summershum/utils.py", line 64, in calculate_sums
    for fname, sha256sum, sha1sum, md5sum in walk_directory(filename):
  File "/usr/lib/python2.6/site-packages/summershum/utils.py", line 103, in walk_directory
    yield (file_path.decode('utf-8'), sha256sum, sha1sum, md5sum)
  File "/usr/lib64/python2.6/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xe4 in position 54: invalid continuation byte
[2014-02-20 10:55:56][summershum   ERROR] Failed to ingest u'sfk-1.7.0.tar.gz' UnicodeDecodeError('utf8', '/tmp/tmpYmhpkD/sfk-1.7.0/testfiles/Formats/22-umlauts-\xe4\xf6\xfc-name.txt', 54, 55, 'invalid continuation byte')

@ralphbean ralphbean closed this in fc3067c Feb 21, 2014

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment