Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure log file encoding is utf-8 #646

Merged
merged 3 commits into from
Mar 16, 2021

Conversation

itsayellow
Copy link
Contributor

  • I have added an entry to docs/changelog.md

Summary of changes

Makes sure that utf-8 encoding is used for log files, to prevent UnicodeEncodeErrors from occurring with emoji.

This was a problem on Windows with Python < 3.9.

Closes #645

Test plan

Tested by running

pipx install -e pycowsay

This PR in Windows Powershell Python3.7:

> pipx install -e pycowsay
⚠️  Ignoring --editable install option. pipx disallows it for anything but a local path, to avoid having to create a
    new src/ directory.
  installed package pycowsay 0.0.0.1, Python 3.7.9
  These apps are now globally available
    - pycowsay.exe
done! ✨ 🌟 ✨

pipx 0.16.1.0 in Windows Powershell Python 3.7

> pipx install -e pycowsay
    new src/ directory.
--- Logging error ---
Traceback (most recent call last):
  File "C:\Program Files\Python37\lib\logging\__init__.py", line 1028, in emit
    stream.write(msg + self.terminator)
  File "C:\Program Files\Python37\lib\encodings\cp1252.py", line 19, in encode
    return codecs.charmap_encode(input,self.errors,encoding_table)[0]
UnicodeEncodeError: 'charmap' codec can't encode characters in position 46-47: character maps to <undefined>
Call stack:
  File "C:\Users\mclapp\git\pipx\venv3.7\Scripts\pipx-script.py", line 33, in <module>
    sys.exit(load_entry_point('pipx', 'console_scripts', 'pipx')())
  File "c:\users\mclapp\git\pipx\src\pipx\main.py", line 759, in cli
    return run_pipx_command(parsed_pipx_args)
  File "c:\users\mclapp\git\pipx\src\pipx\main.py", line 202, in run_pipx_command
    suffix=args.suffix,
  File "c:\users\mclapp\git\pipx\src\pipx\commands\install.py", line 67, in install
    suffix=suffix,
  File "c:\users\mclapp\git\pipx\src\pipx\venv.py", line 224, in install_package
    package_or_url, pip_args
  File "c:\users\mclapp\git\pipx\src\pipx\package_specifier.py", line 154, in parse_specifier_for_install
    subsequent_indent=" " * 4,
Message: '⚠️  Ignoring --editable install option. pipx disallows it for anything but a local path, to avoid having to create a\n    new src/ directory.'
Arguments: ()
  installed package pycowsay 0.0.0.1, Python 3.7.9
  These apps are now globally available
    - pycowsay.exe
done! ✨ 🌟 ✨

@uranusjr
Copy link
Member

I wonder if this could cause crashes if there’s an existing log file written in platform encoding.

@itsayellow
Copy link
Contributor Author

I wonder if this could cause crashes if there’s an existing log file written in platform encoding.

I don't think this is a problem because we start a new log file for each command. So we're not appending to an existing file.

@itsayellow
Copy link
Contributor Author

I just did a quick test and things seem to work ok even with append and different encodings:

> python3
Python 3.9.2 (default, Feb 24 2021, 13:26:09) 
[Clang 12.0.0 (clang-1200.0.32.29)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> with open("test.txt","w",encoding="ascii") as test_fh:
...   print("hi", file=test_fh)
... 
>>> with open("test.txt","a",encoding="utf-8") as test_fh:
...   print("hi🙂", file=test_fh)
... 
>>> exit()
birchtree:pipx file test.txt
test.txt: UTF-8 Unicode text
> more test.txt
hi
hi🙂

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Printing log message when overwriting file triggers UnicodeEncodeError
2 participants