Using pyproject.toml when some dependencies import packages in their setup.py

[illeatmyhat]

Some dependencies of dependencies import packages, e.g. torch, in their setup.py, and this means that it is impossible to use the regular [project.dependencies] field in the pyproject.toml.
Attempting to pip install this package in a new environment will fail because torch is not yet installed.

Hence, there is a need for being able to install packages in "waves".
pyproject.toml has a similar concept in [build-system.requires] but does not do anything when pip installing

How should we reconcile this if we just want to use pyproject.toml without hacks?

The way I've gotten around it is with a setup.py like so:

#!/usr/bin/env python

import pip
import setuptools

pip.main(['install', 'torch'])    # some dependencies use torch in their setup.py

if __name__ == '__main__':
    setuptools.setup()

however this isn't ideal because installing stuff in setup.py without the user knowing sounds like a cancerous thing to do.

[abravalheri]

Hi @illeatmyhat , what I would do first is to try to open a PR to the dependency in question adding a pyproject.toml with build-system.requires.

Have you tried to add pytorch in your packages build-system.reques? Does the build fail anyway?

[abravalheri]

Your setup.py file will run in an environment that has access to the packages in your build-system.requires.

Given your description I thought that you were having problems when building your package...

If that is not the case, can't you add pytorch (and all missing dependencies) to your package dependencies?

(Of course this is just a temporary stop-gap... the best is still trying to PR the project that does not specify the requirements directly)

[illeatmyhat]

can't you add pytorch to your package dependencies

No, during the pip install process, it will attempt to download all of the dependencies, including torch. Then, when basicsr is downloaded, it executes the setup.py and attempts to import torch which is not yet installed.
see

Collecting basicsr>=1.3.4.0
  Using cached basicsr-1.4.1.tar.gz (171 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error
  
  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [6 lines of output]
      Traceback (most recent call last):
        File "<string>", line 2, in <module>
        File "<pip-setuptools-caller>", line 34, in <module>
        File "/run/flatpak/app/com.jetbrains.PyCharm-Community/pip-install-8d0wh6jz/basicsr_05ddfa1214814f02be731c0dfa304744/setup.py", line 8, in <module>
          import torch
      ModuleNotFoundError: No module named 'torch'
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

[abravalheri]

Well if adding the missing packages to your project’s build-system.requires doesn't solve the problem there is not really a good way of solving this, I am afraid.

You night have to contact the maintainers and describe the problems you are facing 😓.

[illeatmyhat]

sounds like a deadlock then, because the maintainers are gone and won't merge the fix people already made
I added the source repo as a git+https install but it's inconvenient that this is the only way to resolve the problem

[abravalheri]

Sorry to hear that. I am afraid that currently this is the only guaranteed way of making the installation via pip work.
There is not much what setuptools can do in this case, since the dependency resolution is something governed by standards and PEPs. Ultimately, setuptools is not the installer.

[jwodder]

If the setup.py for BasicSR imports torch, then torch needs to be listed in the build-system.requires of BasicSR.