New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

po2sub gets encoding wrong and fails #3827

afranke opened this Issue Sep 11, 2018 · 2 comments


None yet
3 participants

afranke commented Sep 11, 2018

I installed translate-toolkit 2.3.0 via pip but the same problem with 2.0.0b5, and with the Fedora 28 package as well.

po2sub -t po/fr.po fails with

po2sub: WARNING: Error processing: input po/fr.po, output, template 'latin-1' codec can't encode character '\u2019' in position 636: ordinal not in range(256)

For some reason it thinks it’s latin-1 when it should be utf-8.

I’m attaching the files so you can try it yourself.


This comment has been minimized.

Toub commented Sep 25, 2018

Edit: my bad, this is an old bug #3601

I also have the same problem with v3.2.0:

 xliff2po: WARNING: Error processing: input src/assets/i18n/messages.en.xlf, output None, template None: 'ascii' codec can't encode character '\xe9' in position 70: ordinal not in range(128)

My file is UTF-8 encoded, not ascii.

Everything is fine with v2.2.5, so this is a regression.

@Toub Toub referenced this issue Sep 25, 2018


2.3.1 release ? #3829


This comment has been minimized.

vinyldarkscratch commented Dec 18, 2018

Looking into this more, it seems that @afranke's and @Toub's bugs are separate.

@afranke, this seems to be an issue with aeidon.encodings.detect, rather its dependency chardet, assuming the subtitle file is in "ISO-8859-1" (aka latin-1), rather than translate-toolkit itself. It looks like they have an issue since 2017, which describes the issues in detection with only one non-ASCII character. Adding more non-ASCII characters in the subtitle file seemed to fix the issue.

For @Toub's problem, it's an old bug that's affecting more converters than just xliff2po. web2py2po had the same issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment