-
-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
UnicodeEncodeError on Windows when there are Unicode chars in the help message #2121
Comments
Please include a minimal reproducible example in the issue itself. Links to projects can be helpful, but it's much easier for contributors and maintainers to address a bug here instead of there. |
Sorry about that. I'll edit the description with an example. Just trying to run it on CI to see if it really breaks. |
Done. |
Running `nene --help` on Windows causes a crash due to the Unicode characters in the app name. For now, remove the Unicode to avoid crashes and add some CI tests to make sure it works. This may be a bug on click (pallets/click#2121).
I believe this can be closed, as it is not an issue caused by click. I wrote up an explanation on another issue and a gist but the tldr is that this is caused by the Windows agent redirecting command output to a file and the default locale code page not being Unicode compatible. While click may be able to solve for this, it is definitely not caused by click. |
The file path in the traceback, |
This was reported to me by a user on Windows and I tested on GitHub Actions since I don't have access to a Windows machine for testing. I'm sure if they were encountering this on Azure or on their own machine, though. |
Here is my repro of the same error. For me this happens when running a click program inside "git bash". https://github.com/rudolfbyker/click-git-bash-unicode-repro I can see how this is not caused by click, but maybe we could treat it as a feature request that click should work around this somehow? |
I'm happy to review a PR that fixes the issue. |
Also note my original comment:
|
A few possible workarounds for those searching:
Depending on the situation, one or more of these could convince Python to use UTF-8 rather than CP1252. |
fwiw, encountered this error for It's on Github Actions Somehow, click still goes into a cp1252 routine in that GHA environment... |
Happy to review a PR. |
could you point me to the point in code where we could set the output encoding based on |
or maybe https://docs.python.org/3/library/sys.html#sys.getdefaultencoding? or some other way to get click to respect Python UTF-8 Mode? |
hmm looks like utf-16? click/src/click/_winconsole.py Line 229 in ca5e1c3
why does the OP and my traceback go into |
as shown in that screenshot, it doesnt happen in every console. would be cool to support Github Actions |
Feel free to review my gist covering this, but just a quick heads up that checking |
would it make sense to simply catch this error in try:
file.write(out)
except UnicodeEncodeError as exc:
if sys.flags.utf8_mode:
raise
msg = "Failed to echo some Unicode character. Try enabling [UTF-8 mode](https://docs.python.org/3/library/os.html#utf8-mode)."
raise UnicodeEncodeError(msg) from exc |
@ddelange +1 I think that's a fantastic idea |
If you think the error needs to be clearer, report that to python. They have been updating many errors in the last few releases. |
I have come across an error when I try to print the help message for my app (
--help
) on Windows (using bash, cmd, and powershell). My help message has unicode characters in it (the project name) which is what seems to be causing the problem:This PR tests running the app with
--help
and it fails on Windows and Python 3.6 and 3.10: leouieda/nene#12Here is a minimum example that fails:
I can confirm that it's the Unicode characters in the docstring of the function wrapped with the main
@click.command
that causes the issue. Removing them fixes the problem (the second CI run on leouieda/nene#12). This issue does not happen on Linux and Mac.For now, I'll remove the unicode characters so I'm not pushing a broken package but it would be great to be able to include the proper spelling of the package name in the future.
Environment:
The text was updated successfully, but these errors were encountered: