Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

The UTF-8 script code without repository is saved with wrong UTF-8 characters in Windows #1207

Closed
ae-ae opened this issue Feb 22, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@ae-ae
Copy link
Contributor

ae-ae commented Feb 22, 2024

Describe the bug

If you run UTF-8 code without a repository, it will be saved at ClearML Server with wrong UTF-8 characters. The issue is caused by the default encoding of the open() function. The open() function uses the encoding returned by the locale.getencoding() since 3.11 and locale.getpreferredencoding() before 3.11.
Both of these functions return ANSI or LC_CTYPE encoding, but not UTF-8. Reading UTF-8 characters with ANSI or any other encoding results in wrong representation of UTF-8 characters. In some cases it may also cause the charmap exception and no script code will be saved at all.
See #443 for the linked issue.

To reproduce

  1. Run the next script with UTF-8 chars äöüÄÜÖß:
from clearml import Task
# UTF-8 chars here: äöüÄÜÖß.
task = Task.init(project_name="Bug report", task_name="äöüÄÜÖß", output_uri=True)
  1. Open the task in ClearML Web.
  2. Invalid characters are displayed in the UNCOMMITTED CHANGES (depends on LC_CTYPE encoding).

Expected behaviour

The correct UTF-8 characters should be displayed in the UNCOMMITTED CHANGES

Environment

Windows

@ae-ae ae-ae added the bug Something isn't working label Feb 22, 2024
ae-ae added a commit to ae-ae/clearml that referenced this issue Feb 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant