conda skeleton pypi does not work if the project's setup.py has build system deps (defined in pyproject.toml)

[shoemakerdr]

Actual Behavior

If a project has a build system dependency (as defined in pyproject.toml), the recipe generation can fail. For example, the following package (makefun) uses pyproject.toml to define extra dependencies that are needed:

# pyproject.toml
[build-system]
requires = [
    "pytest-runner",
    "setuptools",
    "setuptools_scm",
    "six",
]

When I invoke conda skeleton, it is missing host dependencies (specifically setuptools_scm) in order to generate the recipe:

$ conda skeleton pypi makefun
Warning, the following versions were found for makefun
0.1.0
0.2.0
0.3.0
0.4.0
0.5.0
1.0.0
1.0.1
1.0.2
1.1.0
1.1.1
1.1.2
1.2.0
1.3.0
1.4.0
1.5.0
1.5.1
1.6.0
1.6.1
1.6.2
1.6.3
1.6.5
1.6.6
1.6.7
1.6.8
1.6.9
1.6.10
1.6.11
1.7.0
1.8.0
1.9.0
1.9.1
1.9.2
Using 1.9.2
Use --version to specify a different version.
Using url https://files.pythonhosted.org/packages/79/d6/6bf8a56091f18235fe23e6424426c07014bd4e7a317cc37fa202e1aaa32f/makefun-1.9.2.tar.gz (56 KB) for makefun.
Downloading makefun
PyPI URL:  https://files.pythonhosted.org/packages/79/d6/6bf8a56091f18235fe23e6424426c07014bd4e7a317cc37fa202e1aaa32f/makefun-1.9.2.tar.gz
Unpacking makefun...
done
working in /tmp/tmpa5x5vxk4conda_skeleton_makefun-1.9.2.tar.gz
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... done

## Package Plan ##

  environment location: /home/shoemakd/conda-bld/skeleton_1596228888169/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_


The following NEW packages will be INSTALLED:

    _libgcc_mutex:    0.1-conda_forge
    _openmp_mutex:    4.5-0_gnu
    ca-certificates:  2020.6.24-0
    certifi:          2020.6.20-py36_0
    ld_impl_linux-64: 2.34-hc38a660_9
    libffi:           3.2.1-he1b5a44_1007
    libgcc-ng:        9.3.0-h24d8f2e_12
    libgomp:          9.3.0-h24d8f2e_12
    libstdcxx-ng:     9.3.0-hdf63c60_12
    ncurses:          6.2-he1b5a44_1
    openssl:          1.1.1g-h516909a_1
    pip:              20.2-py_0
    python:           3.6.11-h425cb1d_1_cpython
    python_abi:       3.6-1_cp36m
    pyyaml:           5.3.1-py36h7b6447c_1
    readline:         8.0-he28a2e2_2
    setuptools:       49.2.0-py36_0
    sqlite:           3.32.3-hcee41ef_1
    tk:               8.6.10-hbc83047_0
    wheel:            0.34.2-py_1
    xz:               5.2.5-h516909a_1
    yaml:             0.2.5-h7b6447c_0
    zlib:             1.2.11-h516909a_1006

Preparing transaction: ...working... done
Verifying transaction: ...working... done
Executing transaction: ...working... done
Applying patch: '/tmp/tmpa5x5vxk4conda_skeleton_makefun-1.9.2.tar.gz/pypi-distutils.patch'
Trying to apply patch as-is
INFO:conda_build.source:Trying to apply patch as-is
INFO conda_build.source:apply_patch(596): Trying to apply patch as-is
patching file core.py
Hunk #1 succeeded at 167 with fuzz 2 (offset 1 line).
Traceback (most recent call last):
  File "setup.py", line 11, in <module>
    pkg_resources.require("setuptools_scm")
  File "/home/shoemakd/conda-bld/skeleton_1596228888169/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_/lib/python3.6/site-packages/pkg_resources/__init__.py", line 899, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/home/shoemakd/conda-bld/skeleton_1596228888169/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_/lib/python3.6/site-packages/pkg_resources/__init__.py", line 785, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'setuptools_scm' distribution was not found and is required by the application
$PYTHONPATH = /tmp/tmpa5x5vxk4conda_skeleton_makefun-1.9.2.tar.gz/makefun-1.9.2

Leaving build/test directories:
  Work:
 /home/shoemakd/conda-bld/skeleton_1596228888169/work
  Test:
 /home/shoemakd/conda-bld/skeleton_1596228888169/test_tmp
Leaving build/test environments:
  Test:
source activate  /home/shoemakd/conda-bld/skeleton_1596228888169/_test_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_place
  Build:
source activate  /home/shoemakd/conda-bld/skeleton_1596228888169/_build_env


Error: command failed: /home/shoemakd/conda-bld/skeleton_1596228888169/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_/bin/python setup.py install

Expected Behavior

conda skeleton should inspect pyproject.toml for build dependencies.

Steps to Reproduce

Run $ conda skeleton pypi makefun

Output of conda info

conda info

     active environment : /home/shoemakd/dev_mount/sde/conda_recipe_feedstocks/venv/conda_recipe_feedstocks
    active env location : /home/shoemakd/dev_mount/sde/conda_recipe_feedstocks/venv/conda_recipe_feedstocks
            shell level : 2
       user config file : /home/shoemakd/.condarc
 populated config files : /ctc/sw/v7/conda/2020.06.09__13.52.24/.condarc
          conda version : 4.8.3
    conda-build version : 3.18.11
         python version : 3.6.10.final.0
       virtual packages : __glibc=2.17
       base environment : /ctc/sw/v7/conda/2020.06.09__13.52.24  (read only)
           channel URLs : http://tsi-public-read:AP4t34DoXSdds9H9rvCDgT8YSuuSiNZ5nLV2y@artifactory/artifactory/api/conda/ctc-conda-prod-gsd/linux-64
                          http://tsi-public-read:AP4t34DoXSdds9H9rvCDgT8YSuuSiNZ5nLV2y@artifactory/artifactory/api/conda/ctc-conda-prod-gsd/noarch
                          http://tsi-public-read:AP4t34DoXSdds9H9rvCDgT8YSuuSiNZ5nLV2y@artifactory/artifactory/api/conda/ctc-conda-prod-3p/linux-64
                          http://tsi-public-read:AP4t34DoXSdds9H9rvCDgT8YSuuSiNZ5nLV2y@artifactory/artifactory/api/conda/ctc-conda-prod-3p/noarch
                          http://tsi-public-read:AP4t34DoXSdds9H9rvCDgT8YSuuSiNZ5nLV2y@artifactory/artifactory/api/conda/anaconda-conda-upstream-main/linux-64
                          http://tsi-public-read:AP4t34DoXSdds9H9rvCDgT8YSuuSiNZ5nLV2y@artifactory/artifactory/api/conda/anaconda-conda-upstream-main/noarch
                          http://tsi-public-read:AP4t34DoXSdds9H9rvCDgT8YSuuSiNZ5nLV2y@artifactory/artifactory/api/conda/anaconda-conda-upstream-r/linux-64
                          http://tsi-public-read:AP4t34DoXSdds9H9rvCDgT8YSuuSiNZ5nLV2y@artifactory/artifactory/api/conda/anaconda-conda-upstream-r/noarch
                          http://tsi-public-read:AP4t34DoXSdds9H9rvCDgT8YSuuSiNZ5nLV2y@artifactory/artifactory/api/conda/anaconda-conda-upstream-condaforge/linux-64
                          http://tsi-public-read:AP4t34DoXSdds9H9rvCDgT8YSuuSiNZ5nLV2y@artifactory/artifactory/api/conda/anaconda-conda-upstream-condaforge/noarch
          package cache : /home/shoemakd/.conda/pkgs
                          /ctc/sw/v7/conda/LATEST/pkgs
       envs directories : /home/shoemakd/.conda/envs
                          /ctc/sw/v7/conda/2020.06.09__13.52.24/envs
               platform : linux-64
             user-agent : conda/4.8.3 requests/2.23.0 CPython/3.6.10 Linux/3.10.0-957.el7.x86_64 rhel/7.4 glibc/2.17
                UID:GID : 71501:10000
             netrc file : None
           offline mode : False

[shoemakerdr]

Also, would definitely be willing to submit a PR if offered a bit of guidance. Thanks!

[shoemakerdr]

Seems like my issue is related to #3507

[github-actions]

Hi there, thank you for your contribution!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.

If you would like this issue to remain open please:

1. Verify that you can still reproduce the issue at hand
2. Comment that the issue is still reproducible and include:
   - What OS and version you reproduced the issue on
   - What steps you followed to reproduce the issue

NOTE: If this issue was closed prematurely, please leave a comment.

Thanks!

[LourensVeen]

This seems to be breaking the packaging tutorial over at https://docs.conda.io/projects/conda-build/en/stable/user-guide/tutorials/building-conda-packages.html. The sep package has moved to pyproject.toml with release 1.1.0, as a result of which I get

Traceback (most recent call last):
  File "/tmp/tmpj27pk3bjconda_skeleton_sep-1.1.0.tar.gz/sep-1.1.0/setup.py", line 32, in <module>
    from Cython.Build import cythonize
ModuleNotFoundError: No module named 'Cython'

when running conda skeleton pypi sep. Doing conda skeleton pypi --version=1.0.3 sep does work, or at least it doesn't give any errors.

conda info:

     active environment : package-building
    active env location : /home/lourens/.miniconda3/envs/package-building
            shell level : 1
       user config file : /home/lourens/.condarc
 populated config files : /home/lourens/.condarc
          conda version : 23.9.0
    conda-build version : 3.27.0
         python version : 3.9.18.final.0
       virtual packages : __archspec=1=x86_64
                          __glibc=2.35=0
                          __linux=5.15.0=0
                          __unix=0=0
       base environment : /home/lourens/.miniconda3  (writable)
      conda av data dir : /home/lourens/.miniconda3/etc/conda
  conda av metadata url : None
           channel URLs : https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
          package cache : /home/lourens/.miniconda3/pkgs
                          /home/lourens/.conda/pkgs
       envs directories : /home/lourens/.miniconda3/envs
                          /home/lourens/.conda/envs
               platform : linux-64
             user-agent : conda/23.9.0 requests/2.31.0 CPython/3.9.18 Linux/5.15.0-83-generic ubuntu/22.04.3 glibc/2.35
                UID:GID : 1000:1000
             netrc file : None
           offline mode : False