[Mac] Debugpy 1.6.5 not installable with poetry

[tom-mcclintock]

Before creating a new issue, please check the FAQ to see if your question is answered there.

Environment data

• OS and version: Mac 12.5.1
• poetry 1.3.2

Actual behavior

Poetry is a python package manager that specifies its virtual envs in a pyproject.toml file. Here is an example file to start with:

[tool.poetry]
name = "example"
version = "0.1.0"
description = "Install me."
authors = []

[tool.poetry.dependencies]
python = "^3.9"

[tool.poetry.group.dev.dependencies]
pytest = "^7.2.1"

[build-system]
requires = ["poetry-core"]
build-backend = "poetry.core.masonry.api"

Attempting to install the latest debugpy fails:

$ poetry add debugpy
Using version ^1.6.5 for debugpy

Updating dependencies
Resolving dependencies... (0.1s)

Writing lock file

Package operations: 1 install, 0 updates, 0 removals

  • Installing debugpy (1.6.5): Failed

  CalledProcessError

  Command '['/Users/tom.mcclintock/Repositories/tomsource/.venv/bin/python', '-m', 'pip', 'install', '--disable-pip-version-check', '--isolated', '--no-input', '--prefix', '/Users/tom.mcclintock/Repositories/tomsource/.venv', '--no-deps', '/Users/tom.mcclintock/Library/Caches/pypoetry/artifacts/00/1e/5f/034fcde7ad28e523856bdacb87533e7b40228ca678f31ad37f757684bc/debugpy-1.6.5-cp39-cp39-macosx_11_0_x86_64.whl']' returned non-zero exit status 1.

  at ~/opt/anaconda3/lib/python3.9/subprocess.py:528 in run
       524│             # We don't call process.wait() as .__exit__ does that for us.
       525│             raise
       526│         retcode = process.poll()
       527│         if check and retcode:
    →  528│             raise CalledProcessError(retcode, process.args,
       529│                                      output=stdout, stderr=stderr)
       530│     return CompletedProcess(process.args, retcode, stdout, stderr)
       531│ 
       532│ 

The following error occurred when trying to handle this error:


  EnvCommandError

  Command ['/Users/tom.mcclintock/Repositories/tomsource/.venv/bin/python', '-m', 'pip', 'install', '--disable-pip-version-check', '--isolated', '--no-input', '--prefix', '/Users/tom.mcclintock/Repositories/tomsource/.venv', '--no-deps', '/Users/tom.mcclintock/Library/Caches/pypoetry/artifacts/00/1e/5f/034fcde7ad28e523856bdacb87533e7b40228ca678f31ad37f757684bc/debugpy-1.6.5-cp39-cp39-macosx_11_0_x86_64.whl'] errored with the following return code 1, and output: 
  ERROR: debugpy-1.6.5-cp39-cp39-macosx_11_0_x86_64.whl is not a supported wheel on this platform.
  

  at ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/poetry/utils/env.py:1540 in _run
      1536│                 output = subprocess.check_output(
      1537│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1538│                 )
      1539│         except CalledProcessError as e:
    → 1540│             raise EnvCommandError(e, input=input_)
      1541│ 
      1542│         return decode(output)
      1543│ 
      1544│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:


  PoetryException

  Failed to install /Users/tom.mcclintock/Library/Caches/pypoetry/artifacts/00/1e/5f/034fcde7ad28e523856bdacb87533e7b40228ca678f31ad37f757684bc/debugpy-1.6.5-cp39-cp39-macosx_11_0_x86_64.whl

  at ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/poetry/utils/pip.py:58 in pip_install
       54│ 
       55│     try:
       56│         return environment.run_pip(*args)
       57│     except EnvCommandError as e:
    →  58│         raise PoetryException(f"Failed to install {path.as_posix()}") from e
       59│ 

But installing the previous version works: poetry add debugpy=1.6.4.

I believe the crux of the error is: ERROR: debugpy-1.6.5-cp39-cp39-macosx_11_0_x86_64.whl is not a supported wheel on this platform

Expected behavior

I'd like to be able to install the latest debugpy :).

[int19h]

Does the error only show up with poetry, but not when pip install is used directly? And if it does show up for the latter, do you get any additional output beyond "not a supported wheel on this platform"?

[tom-mcclintock]

@int19h pip installs debugpy successfully.

poetry install
poetry shell
pip install debugpy

I should clarify though -- the goal is to make debugpy 1.6.5 installable with poetry because it is a dependency of IPyKernel as of a couple days ago. I suspect I'm just the first person to report this issue ;).

[int19h]

As far as platform support goes, we publish a py3-none-any wheel that should serve as a fallback for any platform that is not served by the more specific wheels. But we also publish debugpy-1.6.5-cp39-cp39-macosx_11_0_x86_64.whl specifically - and if pip install worked, it sounds like that wheel was fine?

Looking at the log, the wheel that Poetry is trying to install is coming from a path that is its cache. Is that a cached version of the PyPI wheel, or did it try to build its own instead, perhaps in some way that does not involve setup.py? While we do have pyproject.toml, we still rely on setup.py to properly package the binary bits in vendored pydevd, which precluded this last time we tried a more declarative approach.

[tom-mcclintock]

Hmm I think I am following.

RE "is that a cached version of ...?" - how can I help answer that? Are you asking if it's a local cache, or a cache somewhere that poetry looks for wheels?

Thanks again for working with me here!

[int19h]

I was referring to this path:
.../Caches/pypoetry/artifacts/00/1e/5f/034fcde7ad28e523856bdacb87533e7b40228ca678f31ad37f757684bc/debugpy-1.6.5-cp39-cp39-macosx_11_0_x86_64.whl
It would be nice to figure out where exactly this comes from. If it's the same wheel that we publish on PyPI, it should work (or else we have a bigger problem!). My concern is that it might be a wheel produced by Poetry itself from debugpy sources, and that something went wrong in the process of building it. If you can compare that file and its contents to the official wheels on PyPI, the difference might be visible.

BTW, is this actually an x86-64 Mac, or is it ARM? We don't publish wheels for specifically ARM, so the expectation is that the py3-none-any wheel would be used there; but, perhaps, Poetry starts with the assumption that only macOS wheels are relevant, and ends up with the wrong architecture since there's no right one among the candidates it's considering?

[rantav]

I'm seeing the same issue. I'm new to poetry, so I hope I'm not missing something obvious. Anyway to your question @int19h it seems that the whl file is the same as the published one

My local failing file:

$ md5  /Users/rantavory/Library/Caches/pypoetry/artifacts/13/33/e8/f28b845f350c201a4998591d33980c1ce073de4d7d13ad138a8e2d3a94/debugpy-1.6.6-cp39-cp39-macosx_11_0_x86_64.whl
MD5 (/Users/rantavory/Library/Caches/pypoetry/artifacts/13/33/e8/f28b845f350c201a4998591d33980c1ce073de4d7d13ad138a8e2d3a94/debugpy-1.6.6-cp39-cp39-macosx_11_0_x86_64.whl) = 892defd64f3763fcd5401529a6701977

It's: 892defd64f3763fcd5401529a6701977

MD5 Pypi file from https://pypi.org/project/debugpy/1.6.6/#copy-hash-modal-3bdbf641-4208-4348-8b49-64073fcda391
MD5: 892defd64f3763fcd5401529a6701977

[tom-mcclintock]

@int19h sorry for the delayed response. My laptop is x86 (see attached).

I went into the cache and deleted the offending wheel, and attempted to poetry add debugpy again but it still failed. Here is the new error report:

Click me

$ poetry add debugpy
Using version ^1.6.6 for debugpy

Updating dependencies
Resolving dependencies... (0.1s)

Writing lock file

Package operations: 0 installs, 1 update, 0 removals

  • Updating debugpy (1.6.5 -> 1.6.6): Failed

  CalledProcessError

  Command '['/Users/tom.mcclintock/Repositories/tomsource/.venv/bin/python', '-m', 'pip', 'install', '--disable-pip-version-check', '--isolated', '--no-input', '--prefix', '/Users/tom.mcclintock/Repositories/tomsource/.venv', '--upgrade', '--no-deps', '/Users/tom.mcclintock/Library/Caches/pypoetry/artifacts/13/33/e8/f28b845f350c201a4998591d33980c1ce073de4d7d13ad138a8e2d3a94/debugpy-1.6.6-cp39-cp39-macosx_11_0_x86_64.whl']' returned non-zero exit status 1.

  at ~/opt/anaconda3/lib/python3.9/subprocess.py:528 in run
       524│             # We don't call process.wait() as .__exit__ does that for us.
       525│             raise
       526│         retcode = process.poll()
       527│         if check and retcode:
    →  528│             raise CalledProcessError(retcode, process.args,
       529│                                      output=stdout, stderr=stderr)
       530│     return CompletedProcess(process.args, retcode, stdout, stderr)
       531│ 
       532│ 

The following error occurred when trying to handle this error:


  EnvCommandError

  Command ['/Users/tom.mcclintock/Repositories/tomsource/.venv/bin/python', '-m', 'pip', 'install', '--disable-pip-version-check', '--isolated', '--no-input', '--prefix', '/Users/tom.mcclintock/Repositories/tomsource/.venv', '--upgrade', '--no-deps', '/Users/tom.mcclintock/Library/Caches/pypoetry/artifacts/13/33/e8/f28b845f350c201a4998591d33980c1ce073de4d7d13ad138a8e2d3a94/debugpy-1.6.6-cp39-cp39-macosx_11_0_x86_64.whl'] errored with the following return code 1, and output: 
  ERROR: debugpy-1.6.6-cp39-cp39-macosx_11_0_x86_64.whl is not a supported wheel on this platform.
  

  at ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/poetry/utils/env.py:1540 in _run
      1536│                 output = subprocess.check_output(
      1537│                     command, stderr=subprocess.STDOUT, env=env, **kwargs
      1538│                 )
      1539│         except CalledProcessError as e:
    → 1540│             raise EnvCommandError(e, input=input_)
      1541│ 
      1542│         return decode(output)
      1543│ 
      1544│     def execute(self, bin: str, *args: str, **kwargs: Any) -> int:

The following error occurred when trying to handle this error:


  PoetryException

  Failed to install /Users/tom.mcclintock/Library/Caches/pypoetry/artifacts/13/33/e8/f28b845f350c201a4998591d33980c1ce073de4d7d13ad138a8e2d3a94/debugpy-1.6.6-cp39-cp39-macosx_11_0_x86_64.whl

  at ~/Library/Application Support/pypoetry/venv/lib/python3.9/site-packages/poetry/utils/pip.py:58 in pip_install
       54│ 
       55│     try:
       56│         return environment.run_pip(*args)
       57│     except EnvCommandError as e:
    →  58│         raise PoetryException(f"Failed to install {path.as_posix()}") from e
       59│ 

We can see that the wheel is different than last week (now it's downloading debugpy 1.6.6). This means that poetry downloads the wheel, puts it in the cache, and then attempts to install from there, hence the error message pointing to the cache location.

[mpas]

Having the same issue, the versions 1.6.5 and 1.6.6 are not installable with Poetry. The only thing that works is installing versions 1.6.4 (poetry add debugpy@1.6.4). Would be great if we could use the latest versions.

[int19h]

It looks like we got tripped by this change in Big Sur. Basically, the problem is that when the wheel is built, the toolchain assigns the platform tag according to the actual version of macOS used to build, which is macos-11.0. But then when pip running on Python 3.9 is determining the valid platform tags when installing the package, when it asks the OS about its version number, it lies and says that it's macOS 10.16 for backwards compatibility reasons. Consequently, the wheel is deemed not supported.

(The reason why this only happened now is that we were building older versions of Python on macOS 10.x before. But we can't do that anymore because it's been deprecated in Azure Pipelines for some time now and will be completely removed soon.)

The official workaround per Apple is to tell the OS to not lie by setting environment variable SYSTEM_VERSION_COMPAT=0 for the Python process that will be installing packages. A long-term fix would be for Python 3.9 to build with the new macOS SDK, so that it gets correct OS version reported regardless of the compat setting; but if I remember correctly, they don't do major SDK revisions across minor releases within a single major Python version. Thus, I would recommend using Python 3.10 or later if at all possible. If you're just using whatever version comes with macOS out of the box, pyenv is usually the easiest way to upgrade.

[int19h]

Another possibility would be for us to take down the macos-11.0 wheels for Python 3.8 and 3.9, thus forcing pip to install the generic wheel instead. It'll mean that debugging is somewhat slower as it's running as pure Python code, without Cython speedups. On the other hand, it would "just work", while the speedups are still available for those on Python 3.10+.

[tom-mcclintock]

@int19h thank you for digging so deep into this! Me and my team are content with moving to python 3.10. It's been on the docket for a while now so it's as good a time as any. And, I can confirm that debugpy 1.6.6 is installable with poetry for python 3.10.

Feel free to handle the status of this issue as you see fit. I really appreciate your quick responses. 👍

[int19h]

The offending wheel for Python 3.9 has been removed; please let me know if this hasn't solved the problem!

[erichulburd]

@int19h - thanks for tracking that down. FWIW, I wasn't able to install with python 3.10 either. I had to install debugpy==1.6.4. 1.6.5 and 1.6.6 resulted in the same error the OP described.

% poetry --version
Poetry (version 1.3.2)
% python --version
Python 3.10.9
% uname -a
Darwin xxxx 21.6.0 Darwin Kernel Version 21.6.0: Thu Sep 29 20:12:57 PDT 2022; root:xnu-8020.240.7~1/RELEASE_X86_64 x86_64

[kpeez]

I'm getting the same error installing version 1.6.5 or 1.6.6 with python 3.10.8

❯ poetry --version
Poetry (version 1.3.2)
❯ python --version
Python 3.10.8
❯ uname -a
Darwin xxxx 22.3.0 Darwin Kernel Version 22.3.0: Thu Jan  5 20:48:54 PST 2023; root:xnu-8792.81.2~2/RELEASE_ARM64_T6000 x86_64

[int19h]

I wonder if Poetry itself is still using the system-provided Python (3.9) in your case. Can you share the output of poetry debug info?

[tom-mcclintock]

Poetry uses whatever version of python it finds first that fits the pyproject.toml's version constraints. I.e. when it creates the virtual env it symlinks to the python installation.

In contrast, when creating a new virtual env, conda actually fetches a fresh python distribution and installs it.

Not sure if that helps @int19h.

[int19h]

I was rather thinking about the version of Python that is used to run Poetry itself, before it even gets to find and parse a pyproject.toml somewhere. Although what should matter here is the version of Python used by pip - but I'm not sure if this behavior wrt version number virtualization also applies to subprocesses once the parent process triggers it.

Alternatively, if it picks the first Python version that fits, could it be that pyproject.toml has something like:

[tool.poetry.dependencies]
python = "^3.9"

and the env that was created ended up linking to system Python? It would be interesting to see what poetry run python -V says.

[mpas]

I had the exact mentioned problem. Updating to Python 3.10 on my machine solved the issue!

[kpeez]

@int19h it doesn't seem like poetry is using the system python. My pyproject.toml has the following:

[tool.poetry.dependencies]
python = ">=3.10,<3.11"

and the output of poetry debug info gives:

Poetry
Version: 1.3.2
Python:  3.10.9

Virtualenv
Python:         3.10.9
Implementation: CPython
Path:           /Users/xxx/xxx/xxx/.venv
Executable:     /Users/xxx/xxx/xxx/.venv/bin/python
Valid:          True

System
Platform:   darwin
OS:         posix
Python:     3.10.9
Path:       /opt/miniconda3
Executable: /opt/miniconda3/bin/python3.10

[int19h]

Hm, I wonder if this has something to do with it being conda - they build the Python binaries themselves, it's not the same stuff as from python.org. If it's built with an older Apple SDK, it would explain this behavior.

To check, try this and see if it gives you 11.0 or 10.16:

/opt/miniconda3/bin/python3.10 -c "import platform; print(platform.mac_ver())"

[kpeez]

@int19h I think you're on to something -- here's what I get:

> /opt/miniconda3/bin/python3.10 -c "import platform; print(platform.mac_ver())"
('10.16', ('', '', ''), 'x86_64')

[int19h]

It sounds like you'll have to use the SYSTEM_VERSION_COMPAT=0 workaround in this case, or if you don't need specific conda packages, switch to a different Python provider (e.g. pyenv will normally install official builds from python.org). I'd rather not take down the Python 3.10 binary wheels, since they seem to work for non-conda interpreters.

[kpeez]

Thanks! I re-installed conda (and switched from x86_64 to arm64) and am now no longer getting this error.

[jacobbridges]

Hi all, not meaning to necro the thread but I ran into this error today while attempting to install debugpy with poetry. Any suggestions?

poetry debug info

Poetry
Version: 1.4.1
Python:  3.10.6

Virtualenv
Python:         3.10.6
Implementation: CPython
Path:           /Users/.../venv
Executable:     /Users/.../venv/bin/python
Valid:          True

System
Platform:   darwin
OS:         posix
Python:     3.10.6
Path:       /Users/xxx/.pyenv/versions/3.10.6
Executable: /Users/xxx/.pyenv/versions/3.10.6/bin/python3.10

Platform Mac Version

python3.10 -c "import platform; print(platform.mac_ver())"
('13.2.1', ('', '', ''), 'arm64')

Other Debug Info

% poetry --version
Poetry (version 1.4.1)
% python --version
Python 3.10.6
% uname -a
Darwin 00000764494mac 22.3.0 Darwin Kernel Version 22.3.0: Mon Jan 30 20:38:37 PST 2023; root:xnu-8792.81.3~2/RELEASE_ARM64_T6000 arm64

Full Error

  • Installing debugpy (1.6.6): Failed

  _WheelFileValidationError

  ["In /Users/JAB215/Library/Caches/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/_pydevd_bundle/pydevd_cython.cp39-win_amd64.pyd didn't match RECORD", "In /Users/JAB215/Library/Caches/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/_pydevd_frame_eval/pydevd_frame_evaluator.cp39-win_amd64.pyd didn't match RECORD", "In /Users/JAB215/Library/Caches/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/pydevd_attach_to_process/attach_amd64.dll didn't match RECORD", "In /Users/JAB215/Library/Caches/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/pydevd_attach_to_process/attach_x86.dll didn't match RECORD", "In /Users/JAB215/Library/Caches/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/pydevd_attach_to_process/inject_dll_amd64.exe didn't match RECORD", "In /Users/JAB215/Library/Caches/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/pydevd_attach_to_process/inject_dll_x86.exe didn't match RECORD", "In /Users/JAB215/Library/Caches/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/pydevd_attach_to_process/run_code_on_dllmain_amd64.dll didn't match RECORD", "In /Users/JAB215/Library/Caches/pypoetry/artifacts/f2/96/74/9c8a5447a1d805edc0c82f964602703c805d10cae4b7b14e5d5d546ff0/debugpy-1.6.6-py2.py3-none-any.whl, hash / size of debugpy/_vendored/pydevd/pydevd_attach_to_process/run_code_on_dllmain_x86.dll didn't match RECORD"]

  at venv/lib/python3.10/site-packages/installer/sources.py:289 in validate_record
      285│                         f"In {self._zipfile.filename}, hash / size of {item.filename} didn't match RECORD"
      286│                     )
      287│ 
      288│         if issues:
    → 289│             raise _WheelFileValidationError(issues)
      290│ 
      291│     def get_contents(self) -> Iterator[WheelContentElement]:
      292│         """Sequential access to all contents of the wheel (including dist-info files).
      293│ 

[mpas]

Hi all, not meaning to necro the thread but I ran into this error today while attempting to install debugpy with poetry. Any suggestions?

poetry debug info
Platform Mac Version
Other Debug Info
Full Error

@jacobbridges
Try with a more recent 3.10 version of Python, in my case that solved the problem.

[R2D34]

Stumbled on a similar issue today, while trying to install ipykernel on mac with python3.11 (installed via Brew)...

Mac Platform

python3.11 -c "import platform; print(platform.mac_ver())"
('13.0', ('', '', ''), 'arm64')

Poetry debug info

Poetry
Version: 1.4.1
Python:  3.11.2

System
Platform:   darwin
OS:         posix
Python:     3.11.2
Path:       /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11
Executable: /opt/homebrew/opt/python@3.11/Frameworks/Python.framework/Versions/3.11/bin/python3.11

Shortened Error while installing debugpy 1.6.6

_WheelFileValidationError

[....
"debugpy/_vendored/pydevd/pydevd_attach_to_process/run_code_on_dllmain_x86.dll didn't match RECORD"]

  at ~/Library/Application Support/pypoetry/venv/lib/python3.11/site-packages/installer/sources.py:289 in validate_record
      285│                         f"In {self._zipfile.filename}, hash / size of {item.filename} didn't match RECORD"
      286│                     )
      287│
      288│         if issues:
    → 289│             raise _WheelFileValidationError(issues)
      290│
      291│     def get_contents(self) -> Iterator[WheelContentElement]:
      292│         """Sequential access to all contents of the wheel (including dist-info files).
      293│

Tested also with Python 3.10 and while the SYSTEM_VERSION_COMPAT=0 was passed to command. Didn't work.

[onuralpszr]

Looks like it is related to this issue : #1246

It needs a fix before downgrade poetry or turn off certain config written in here.