New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

po2sub gets encoding wrong and fails #3827

Open
afranke opened this Issue Sep 11, 2018 · 2 comments

Comments

Projects
None yet
3 participants
@afranke

afranke commented Sep 11, 2018

I installed translate-toolkit 2.3.0 via pip but the same problem with 2.0.0b5, and with the Fedora 28 package as well.

po2sub -t gnome330.srt po/fr.po fr.srt fails with

po2sub: WARNING: Error processing: input po/fr.po, output fr.srt, template gnome330.srt: 'latin-1' codec can't encode character '\u2019' in position 636: ordinal not in range(256)

For some reason it thinks it’s latin-1 when it should be utf-8.

I’m attaching the files so you can try it yourself.

@Toub

This comment has been minimized.

Toub commented Sep 25, 2018

Edit: my bad, this is an old bug #3601

I also have the same problem with v3.2.0:

 xliff2po: WARNING: Error processing: input src/assets/i18n/messages.en.xlf, output None, template None: 'ascii' codec can't encode character '\xe9' in position 70: ordinal not in range(128)

My file is UTF-8 encoded, not ascii.

Everything is fine with v2.2.5, so this is a regression.

@Toub Toub referenced this issue Sep 25, 2018

Closed

2.3.1 release ? #3829

@vinyldarkscratch

This comment has been minimized.

vinyldarkscratch commented Dec 18, 2018

Looking into this more, it seems that @afranke's and @Toub's bugs are separate.

@afranke, this seems to be an issue with aeidon.encodings.detect, rather its dependency chardet, assuming the subtitle file is in "ISO-8859-1" (aka latin-1), rather than translate-toolkit itself. It looks like they have an issue since 2017, which describes the issues in detection with only one non-ASCII character. Adding more non-ASCII characters in the subtitle file seemed to fix the issue.

For @Toub's problem, it's an old bug that's affecting more converters than just xliff2po. web2py2po had the same issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment