python-setuptools: DistutilsPlatformError #5155

danyeaw · 2019-04-08T17:26:24Z

When trying to install a package locally, I am getting a UnicodeDecodeError. It looks like Python is trying to use cp1252 while loading the metadata for PEP517. I would expect that it needs to use UTF-8 in order correctly decode the metadata.

locale.getpreferredencoding() is cp1252.

$ pip3 install --user .
Processing c:/tools/msys64/home/dyeaw/gaphor
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
  Installing backend dependencies ... done
    Preparing wheel metadata ... error
    Complete output from command C:/tools/msys64/mingw64/bin/python3.exe C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py prepare_metadata_for_build_wheel C:/Users/dyeaw/AppData/Local/Temp/tmp237x9yfr:
    Traceback (most recent call last):
      File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 64, in prepare_metadata_for_build_wheel
        hook = backend.prepare_metadata_for_build_wheel
    AttributeError: module 'poetry.masonry.api' has no attribute 'prepare_metadata_for_build_wheel'

    During handling of the above exception, another exception occurred:

    Traceback (most recent call last):
      File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 207, in <module>
        main()
      File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 197, in main
        json_out['return_val'] = hook(**hook_input['kwargs'])
      File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 67, in prepare_metadata_for_build_wheel
        config_settings)
      File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 95, in _get_wheel_metadata_from_wheel
        whl_basename = backend.build_wheel(metadata_directory, config_settings)
      File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-n028mbl_/overlay/lib/python3.7/site-packages\poetry\masonry\api.py", line 40, in build_wheel
        poetry, SystemEnv(Path(sys.prefix)), NullIO(), Path(wheel_directory)
      File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-n028mbl_/overlay/lib/python3.7/site-packages\poetry\masonry\builders\wheel.py", line 48, in make_in
        wb = WheelBuilder(poetry, env, io, target_dir=directory, original=original)
      File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-n028mbl_/overlay/lib/python3.7/site-packages\poetry\masonry\builders\wheel.py", line 38, in __init__
        super(WheelBuilder, self).__init__(poetry, env, io)
      File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-n028mbl_/overlay/lib/python3.7/site-packages\poetry\masonry\builders\builder.py", line 42, in __init__
        self._meta = Metadata.from_package(self._package)
      File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-n028mbl_/overlay/lib/python3.7/site-packages\poetry\masonry\metadata.py", line 50, in from_package
        meta.description = f.read()
      File "C:/tools/msys64/mingw64/lib/python3.7\encodings\cp1252.py", line 23, in decode
        return codecs.charmap_decode(input,self.errors,decoding_table)[0]
    UnicodeDecodeError: 'charmap' codec can't decode byte 0x90 in position 7221: character maps to <undefined>

    ----------------------------------------
Command "C:/tools/msys64/mingw64/bin/python3.exe C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py prepare_metadata_for_build_wheel C:/Users/dyeaw/AppData/Local/Temp/tmp237x9yfr" failed with error code 1 in C:/Users/dyeaw/AppData/Local/Temp/pip-req-build-clnw4_9m

Poetry is not using an encoding when it calls open(), so I forked the project and edited the metadata module. I then tried to install this version:

$ pip3 install --user .
Processing c:/tools/msys64/home/dyeaw/poetry
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  Complete output from command C:/tools/msys64/mingw64/bin/python3.exe C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py get_requires_for_build_wheel C:/Users/dyeaw/AppData/Local/Temp/tmpab8q56px:
  Traceback (most recent call last):
    File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 207, in <module>
      main()
    File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 197, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
    File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 48, in get_requires_for_build_wheel
      backend = _build_backend()
    File "C:/tools/msys64/mingw64/lib/python3.7/site-packages/pep517/_in_process.py", line 34, in _build_backend
      obj = import_module(mod_path)
    File "C:/tools/msys64/mingw64/lib/python3.7\importlib\__init__.py", line 127, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
    File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
    File "<frozen importlib._bootstrap>", line 983, in _find_and_load
    File "<frozen importlib._bootstrap>", line 953, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
    File "<frozen importlib._bootstrap>", line 983, in _find_and_load
    File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
    File "<frozen importlib._bootstrap_external>", line 728, in exec_module
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-93bwkryn/overlay/lib/python3.7/site-packages\setuptools\__init__.py", line 228, in <module>
      monkey.patch_all()
    File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-93bwkryn/overlay/lib/python3.7/site-packages\setuptools\monkey.py", line 101, in patch_all
      patch_for_msvc_specialized_compiler()
    File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-93bwkryn/overlay/lib/python3.7/site-packages\setuptools\monkey.py", line 164, in patch_for_msvc_specialized_compiler
      patch_func(*msvc9('find_vcvarsall'))
    File "C:/Users/dyeaw/AppData/Local/Temp/pip-build-env-93bwkryn/overlay/lib/python3.7/site-packages\setuptools\monkey.py", line 151, in patch_params
      mod = import_module(mod_name)
    File "C:/tools/msys64/mingw64/lib/python3.7\importlib\__init__.py", line 127, in import_module
      return _bootstrap._gcd_import(name[level:], package, level)
    File "<frozen importlib._bootstrap>", line 1006, in _gcd_import
    File "<frozen importlib._bootstrap>", line 983, in _find_and_load
    File "<frozen importlib._bootstrap>", line 967, in _find_and_load_unlocked
    File "<frozen importlib._bootstrap>", line 677, in _load_unlocked
    File "<frozen importlib._bootstrap_external>", line 728, in exec_module
    File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
    File "C:/tools/msys64/mingw64/lib/python3.7\distutils\msvc9compiler.py", line 296, in <module>
      raise DistutilsPlatformError("VC %0.1f is not supported by this module" % VERSION)
  distutils.errors.DistutilsPlatformError: VC 6.0 is not supported by this module

I have setuptools installed via pacman only. I have also removed all of Python 3 and dependencies, along with the full /lib/python3.7 directory to make sure the distutils installed versions were blow away as well. I only used pip3 install --user to prevent any pip vs. pacman conflicts.

The text was updated successfully, but these errors were encountered:

lazka · 2019-04-09T15:26:53Z

Poetry is not using an encoding when it calls open(), so I forked the project and edited the metadata module. I then tried to install this version:

Yeah, it's not passing an encoding here: https://github.com/sdispater/poetry/blob/b4f9e3fc9303bf4039f52ee19a1bf680684000ca/poetry/masonry/metadata.py#L49 Please file a poetry bug.

I have setuptools installed via pacman only.

Try passing --no-build-isolation to pip install

danyeaw · 2019-04-10T01:50:15Z

@lazka Thanks for the feedback. I opened an issue with Poetry, and I'll try to get a PR submitted to get it fixed.

--no-build-isolation only works for non-PEP517 installed packages, and in this case poetry doesn't have a setup.py. I get a FileNotFoundError: [Errno 2] No such file or directory: 'setup.py' error.

lazka · 2019-04-10T06:13:30Z

--no-build-isolation only works for non-PEP517 installed packages, and in this case poetry doesn't have a setup.py. I get a FileNotFoundError: [Errno 2] No such file or directory: 'setup.py' error.

Oh, I see. No idea :/

danyeaw · 2019-04-10T21:02:50Z

@lazka What are your thoughts on an overall strategy on fixing these types of Python issues along with #5001. It seems like Python is changing faster than we can patch downstream. Would it at least be worthwhile to start a discussion on the python-dev mailing list about where there is interest in the Python community providing more official support for MSYS2?

lazka · 2019-04-11T06:59:14Z

What would be the goal of starting a discussion there? No one there is using it and probably doesn't plan to.

We could spin the Python3 package out into a CPython fork with patches on top (+ CI maybe?) and try to push some simple (not MSYS2 specific) things upstream. That would probably make it more visible as its own "platform". But, time.... :)

For the setuptools error, we could try to patch msvc9compiler.py (which fails here because it tries to detect the MSVC version Python was built with) to not fail to import, maybe we can then drop the setuptools patch.

We currently patch setuptools itself to not import distutils.msvc9compiler, which fails because it fails to detect the msvc version and falls back to an unsupported version 6 and raises. This doesn't help much in case setuptools is installed through pip which doesn't contain the fix. This fixes msvc9compiler.py instead to not fail on import but at the compiler instance creation, i.e. the point where the version is actually used. The setuptools fix remains for now to make updates easier. Fixes msys2#5155

lazka · 2019-04-21T14:46:39Z

For the setuptools error, we could try to patch msvc9compiler.py (which fails here because it tries to detect the MSVC version Python was built with) to not fail to import, maybe we can then drop the setuptools patch.

This is what I did now in #5260

python: better compatibility with unpatched setuptools. Fixes #5155

danyeaw · 2019-04-21T17:53:18Z

@lazka Thanks so much for resolving this ❤️

lazka · 2019-04-21T19:25:26Z

Note that installation will still be broken in some cases, not sure when this is relevant for example: https://github.com/msys2/MINGW-packages/blob/master/mingw-w64-python-setuptools/0001-mingw-python-fix.patch

We need to keep the old msys base for binary compatibility with older curl, but that version doesn't have the fix for that issue.

danyeaw mentioned this issue Apr 10, 2019

UnicodeDecodeError in Masonry with MSYS2 python-poetry/poetry#1027

Closed

3 tasks

lazka added the python label Apr 15, 2019

lazka changed the title ~~Python 3 Gives UnicodeDecodeError~~ python-setuptools: DistutilsPlatformError Apr 21, 2019

lazka mentioned this issue Apr 21, 2019

python: better compatibility with unpatched setuptools. Fixes #5155 #5260

Merged

Alexpux closed this as completed in #5260 Apr 21, 2019

Alexpux added a commit that referenced this issue Apr 21, 2019

Merge pull request #5260 from lazka/python-fix-msvc9-import

26c72cb

python: better compatibility with unpatched setuptools. Fixes #5155

glandium added a commit to glandium/git-cinnabar that referenced this issue Oct 2, 2021

[CI] Workaround msys2/MINGW-packages#5155

3b3f653

We need to keep the old msys base for binary compatibility with older curl, but that version doesn't have the fix for that issue.

glandium added a commit to glandium/git-cinnabar that referenced this issue Oct 2, 2021

[CI] Workaround msys2/MINGW-packages#5155

d626585

We need to keep the old msys base for binary compatibility with older curl, but that version doesn't have the fix for that issue.

glandium added a commit to glandium/git-cinnabar that referenced this issue Oct 3, 2021

[CI] Workaround msys2/MINGW-packages#5155

f784c51

We need to keep the old msys base for binary compatibility with older curl, but that version doesn't have the fix for that issue.

glandium added a commit to glandium/git-cinnabar that referenced this issue Oct 3, 2021

[CI] Workaround msys2/MINGW-packages#5155

6a3bf37

We need to keep the old msys base for binary compatibility with older curl, but that version doesn't have the fix for that issue.

glandium added a commit to glandium/git-cinnabar that referenced this issue Oct 3, 2021

[CI] Workaround msys2/MINGW-packages#5155

30cf7a4

We need to keep the old msys base for binary compatibility with older curl, but that version doesn't have the fix for that issue.

sam-s mentioned this issue Aug 15, 2023

pip install tries to use VC instead of gcc and fails #18192

Closed

7 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

python-setuptools: DistutilsPlatformError #5155

python-setuptools: DistutilsPlatformError #5155

danyeaw commented Apr 8, 2019

lazka commented Apr 9, 2019

danyeaw commented Apr 10, 2019

lazka commented Apr 10, 2019

danyeaw commented Apr 10, 2019

lazka commented Apr 11, 2019

lazka commented Apr 21, 2019

danyeaw commented Apr 21, 2019

lazka commented Apr 21, 2019

python-setuptools: DistutilsPlatformError #5155

python-setuptools: DistutilsPlatformError #5155

Comments

danyeaw commented Apr 8, 2019

lazka commented Apr 9, 2019

danyeaw commented Apr 10, 2019

lazka commented Apr 10, 2019

danyeaw commented Apr 10, 2019

lazka commented Apr 11, 2019

lazka commented Apr 21, 2019

danyeaw commented Apr 21, 2019

lazka commented Apr 21, 2019