Skip to content

UnicodeDecodeError on Windows with pyproject.toml #11058

@domdfcoding

Description

@domdfcoding

Crash Report

On Windows (where the default encoding for open is usually not UTF-8), with a pyproject.toml file containing the mypy configuration, mypy crashes due to being unable to decode unicode characters.

The TOML specification requires files to be encoded in UTF-8, so it is valid to assume tools will correctly parse files containing unicode characters.

Traceback

Traceback (most recent call last):
  File "C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "D:\a\domdf_python_tools\domdf_python_tools\.tox\mypy\Scripts\mypy.EXE\__main__.py", line 7, in <module>
  File "D:\a\domdf_python_tools\domdf_python_tools\.tox\mypy\lib\site-packages\mypy\__main__.py", line 11, in console_entry
    main(None, sys.stdout, sys.stderr)
  File "mypy\main.py", line 87, in main
  File "mypy\main.py", line 165, in run_build
  File "mypy\build.py", line 179, in build
  File "mypy\build.py", line 229, in _build
  File "mypy\build.py", line 475, in load_plugins
  File "mypy\build.py", line 391, in load_plugins_from_config
  File "mypy\build.py", line 511, in find_config_file_line_number
  File "C:\hostedtoolcache\windows\Python\3.6.8\x64\lib\encodings\cp1252.py", line 23, in decode
    return codecs.charmap_decode(input,self.errors,decoding_table)[0]
UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 220: character maps to <undefined>

This is caused by the following function, particularly the call to open on line 511 without an encoding being specified.

mypy/mypy/build.py

Lines 503 to 523 in 5c4aea3

def find_config_file_line_number(path: str, section: str, setting_name: str) -> int:
"""Return the approximate location of setting_name within mypy config file.
Return -1 if can't determine the line unambiguously.
"""
in_desired_section = False
try:
results = []
with open(path) as f:
for i, line in enumerate(f):
line = line.strip()
if line.startswith('[') and line.endswith(']'):
current_section = line[1:-1].strip()
in_desired_section = (current_section == section)
elif in_desired_section and re.match(r'{}\s*='.format(setting_name), line):
results.append(i + 1)
if len(results) == 1:
return results[0]
except OSError:
pass
return -1

To Reproduce

  • Use the following pyproject.toml file content:
[project]
description = "Factory ⸻ A code generator 🏭"

[tool.mypy]
  • Run mypy 0.910 on Windows

Your Environment

  • Mypy version used: 0.910
  • Mypy command-line flags: None
  • Mypy configuration options from mypy.ini (and other config files): None (empty tool.mypy section in pyproject.toml)
  • Python version used: 3.6 and 3.9
  • Operating system and version: Windows 10

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions