Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[bug] platform_system == "Darwin"|"Linux" used on Win10 ⇒ pdm add tensorflow>=2.16 fails #2867

Closed
1 task done
goyalyashpal opened this issue May 5, 2024 · 10 comments
Closed
1 task done
Labels
🐛 bug Something isn't working

Comments

@goyalyashpal
Copy link

goyalyashpal commented May 5, 2024

  • I have searched the issue tracker and believe that this is not a duplicate.

Summaries & Titles

  • [bug] platform_system == "Darwin"|"Linux" used on Win10 ⇒ pdm add tensorflow>=2.16 fails
  • [bug] pdm add uses platform_system == "Darwin" or "Linux" on Win10 os
    hence pdm fails to resolve tensorflow to tensorflow-intel package
  • pdm add tensorflow>=2.16 fails, pip install tensorflow>=2.16 succeeds on windows

Steps to reproduce

  • run pdm lock with pyproject.toml having specifics as given:
# file: pyproject.toml
[project]
dependencies = [
    # "keras",
    "tensorflow>=2.16",
]
requires-python = "<3.13,>=3.12"

[build-system]
requires = ["pdm-backend"]
build-backend = "pdm.backend"

Actual behavior

> pdm lock -v
... STATUS ...
... NORMAL OUTPUT ...
... TRACEBACK ...
normal output
🔒 Lock failed
ERROR: Unable to find a resolution for tensorflow-cpu-aws
because of the following conflicts:
  tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or 
  platform_machine == "aarch64") and platform_system == "Linux" (from 
  tensorflow@2.16.1)
To fix this, you could loosen the dependency version constraints in
pyproject.toml. See
https://pdm-project.org/en/latest/usage/lockfile/#solve-the-locking-failure 
for more details.
See $LOCALAPPDATA\pdm\pdm\Logs\pdm-lock-kt_3eb6f.log for
detailed debug log.
[ResolutionImpossible]: Unable to find a resolution
WARNING: Add '-v' to see the detailed traceback
verbose status

STATUS: Resolving dependencies
pdm.termui: ======== Start resolving requirements ========
pdm.termui:   tensorflow>=2.16
pdm.termui:   python<3.13,>=3.12
pdm.termui:   Adding requirement tensorflow>=2.16
pdm.termui:   Adding requirement python<3.13,>=3.12
pdm.termui: ======== Starting round 0 ========
STATUS: Resolving: new pin python<3.13,>=3.12
pdm.termui: Pinning: python None
pdm.termui: ======== Ending round 0 ========
pdm.termui: ======== Starting round 1 ========
pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui:   Adding requirement tensorflow==2.16.1; platform_system == "Darwin" and platform_machine == "arm64"(from tensorflow 2.16.1)
pdm.termui:   Adding requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux"(from
tensorflow 2.16.1)
pdm.termui: Candidate rejected: tensorflow@2.16.1 because it introduces a new requirement tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" that conflicts with other requirements:

pdm.termui: Conflicts detected:
  tensorflow-cpu-aws==2.16.1; (platform_machine == "arm64" or platform_machine == "aarch64") and platform_system == "Linux" (from tensorflow@2.16.1)
verbose traceback

Traceback (most recent call last):
  File "<frozen runpy>", line 198, in _run_module_as_main
  File "<frozen runpy>", line 88, in _run_code
  File "$USERPROFILE\.local\bin\pdm.exe\__main__.py", line 7, in <module>
  File "D:\UserFiles\pipx\venvs\pdm\Lib\site-packages\pdm\core.py", line 358, in main
    return core.main(args or sys.argv[1:])
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "D:\UserFiles\pipx\venvs\pdm\Lib\site-packages\pdm\core.py", line 276, in main
    raise cast(Exception, err).with_traceback(traceback) from None
  File "D:\UserFiles\pipx\venvs\pdm\Lib\site-packages\pdm\core.py", line 271, in main
    self.handle(project, options)
  File "D:\UserFiles\pipx\venvs\pdm\Lib\site-packages\pdm\core.py", line 207, in handle
    command.handle(project, options)
  File "D:\...\.pdm-plugins\Lib\site-packages\pdm_conda\cli\commands\lock.py", line 31, in handle
    super().handle(project=project, options=options)
  File "D:\UserFiles\pipx\venvs\pdm\Lib\site-packages\pdm\cli\commands\lock.py", line 92, in handle
    actions.do_lock(
  File "D:\UserFiles\pipx\venvs\pdm\Lib\site-packages\pdm\cli\actions.py", line
125, in do_lock
    raise ResolutionImpossible("Unable to find a resolution") from None
resolvelib.resolvers.ResolutionImpossible: Unable to find a resolution

Expected behavior

  • if pip is able to install tensorflow>=2.16 then why can't pdm fetch that?
  • pip does so with wheels: tensorflow_intel-2.16.1-cp310-cp310-win_amd64.whl (376.9 MB)
  • ... which corresponds to package tensorflow-intel but with automatic resolution from package tensorflow
  • this automatic resolution can be seen from output of pip install below: tensorflow-intel->tensorflow
> pip install tensorflow
Collecting tensorflow
  Downloading tensorflow-2.16.1-cp310-cp310-win_amd64.whl.metadata (3.5 kB)
Collecting tensorflow-intel==2.16.1 (from tensorflow)
  Downloading tensorflow_intel-2.16.1-cp310-cp310-win_amd64.whl.metadata (5.0 kB)
Collecting absl-py>=1.0.0 (from tensorflow-intel==2.16.1->tensorflow)
  ...
Collecting mdurl~=0.1 (from markdown-it-py>=2.2.0->rich->keras>=3.0.0->tensorflow-intel==2.16.1->tensorflow)
  Using cached mdurl-0.1.2-py3-none-any.whl.metadata (1.6 kB)
Downloading tensorflow-2.16.1-cp310-cp310-win_amd64.whl (2.1 kB)
Downloading tensorflow_intel-2.16.1-cp310-cp310-win_amd64.whl (376.9 MB)
  ...
Using cached mdurl-0.1.2-py3-none-any.whl (10.0 kB)
Installing collected packages: namex, libclang, flatbuffers, wrapt, typing-extensions, termcolor, tensorflow-io-gcs-filesystem, tensorboard-data-server, six, pygments, protobuf, numpy, mdurl, MarkupSafe, markdown, grpcio, gast, absl-py, werkzeug, optree, opt-einsum, ml-dtypes, markdown-it-py, h5py, google-pasta, astunparse, tensorboard, rich, keras, tensorflow-intel, tensorflow
Successfully installed MarkupSafe-2.1.5 absl-py-2.1.0 astunparse-1.6.3 flatbuffers-24.3.25 gast-0.5.4 google-pasta-0.2.0 grpcio-1.63.0 h5py-3.11.0 keras-3.3.3 libclang-18.1.1 markdown-3.6 markdown-it-py-3.0.0 mdurl-0.1.2 ml-dtypes-0.3.2 namex-0.0.8 numpy-1.26.4 opt-einsum-3.3.0 optree-0.11.0 protobuf-4.25.3 pygments-2.18.0 rich-13.7.1 six-1.16.0 tensorboard-2.16.2 tensorboard-data-server-0.7.2 tensorflow-2.16.1 tensorflow-intel-2.16.1 tensorflow-io-gcs-filesystem-0.31.0 termcolor-2.4.0 typing-extensions-4.11.0 werkzeug-3.0.2 wrapt-1.16.0

Environment Information

> which pip
/d/UserFiles/scoop/apps/mambaforge/current/scripts/pip

> pip --version
pip 24.0 from D:\UserFiles\scoop\apps\mambaforge\current\lib\site-packages\pip (python 3.10)
> # Paste the output of `pdm info && pdm info --env` below:
(project_1-3.12)
> pdm info
PDM version:
  2.15.1
Python Interpreter:
  D:\...\.venv\Scripts\python.exe (3.12)
Project Root:
  D:/...
Local Packages:

(project_1-3.12)
> pdm info --env
{
  "implementation_name": "cpython",
  "implementation_version": "3.12.2",
  "os_name": "nt",
  "platform_machine": "AMD64",
  "platform_release": "10",
  "platform_system": "Windows",
  "platform_version": "10.0.19045",
  "python_full_version": "3.12.2",
  "platform_python_implementation": "CPython",
  "python_version": "3.12",
  "sys_platform": "win32"
}
@goyalyashpal goyalyashpal added the 🐛 bug Something isn't working label May 5, 2024
@goyalyashpal goyalyashpal changed the title pdm add tensorflow>=2.16 fails, pip install tensorflow>=2.16 succeeds on windows [bug] platform_system == "Darwin"|"Linux" used on Win10 os ➡ pdm add tensorflow>=2.16 fails May 5, 2024
@goyalyashpal goyalyashpal changed the title [bug] platform_system == "Darwin"|"Linux" used on Win10 os ➡ pdm add tensorflow>=2.16 fails [bug] platform_system == "Darwin"|"Linux" used on Win10 ⇒ pdm add tensorflow>=2.16 fails May 5, 2024
@frostming
Copy link
Collaborator

  • if pip is able to install tensorflow>=2.16 then why can't pdm fetch that?

pip is resolving for the current platform but PDM is resolving for multiple platforms. Read more: https://frostming.com/en/2024/pdm-lockfile/

You can do pdm lock -S no_cross_platform to replicate pip's behavior.

@frostming frostming closed this as not planned Won't fix, can't repro, duplicate, stale May 7, 2024
@goyalyashpal
Copy link
Author

thanks for "closing as not planned" 😃

@frostming
Copy link
Collaborator

thanks for "closing as not planned" 😃

you are welcome. or what should I do otherwise?

@goyalyashpal
Copy link
Author

goyalyashpal commented May 7, 2024

i have seen many projects arrogantly close even non planned issues as completed.

so i always thanks those who dont do such weird things.

@pawamoy
Copy link
Sponsor Contributor

pawamoy commented May 7, 2024

I am myself often unsure if I should close an issue as "completed" or "not planned" 🤔
Sometimes won't fix, can't repro, duplicate, or stale do not really apply 🤔
What's your reasoning when choosing not planned or completed @goyalyashpal 🙂? Curious to see how others handle that.

@goyalyashpal
Copy link
Author

if a proper solution exists, and author agrees (indicated by yes, this is what i needed etc) then "completed"

if workaround etc is required (like here), out of scope, out of plan, not enough resources (will, time, power, knowledge), technical or code debt, etc etc etc then "not planned"

@goyalyashpal
Copy link
Author

but PDM is resolving for multiple platforms

just sharing for reference the following para of how micromamba addresses this:

Using conda-lock, you can generate lockfiles which,
- ..., pin precisely and include a checksum for each package for reproducibility.
- [... also] , these “unified” lock files are multi-platform.

- #conda-lock-yaml-spec-files | Micromamba User Guide

@frostming
Copy link
Collaborator

frostming commented May 8, 2024

Not planned means no offense, if so, GitHub should remove this option.

I close as completed when there is any action taken(docs/code) on the maintainers' side, otherwise not planned, including Q&A

@goyalyashpal
Copy link
Author

goyalyashpal commented May 21, 2024

You can do pdm lock -S no_cross_platform to replicate pip's behavior.
- @ frostming at #2867 (comment)

can that be specified in pyproject.toml ?

@frostming
Copy link
Collaborator

can that be specified in pyproject.toml ?

[tool.pdm.options]
lock = ["-S", "no_cross_platform"]

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🐛 bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants