Migration from a fully configured setup.py to pyproject.toml destroys entry_points

[MatthewRalston]

Problem description

What is your environment

python 3.10.1
pip 22.3.1
setuptools 61.0

What is your pyproject.toml?

• pyproject.toml

[project.scripts]
kmerdb = "kmerdb:cli"
[options.entry_points]
console_scripts = {kmerdb= "kmerdb:cli"}

What is the issue?

After successful install into a .venv:

$ python
>>>import kmerdb # works fine
$ python -m kmerdb -h # Also works fine.
$ kmerdb -h
Traceback (most recent call last):
  File "path/to/.venv/bin/kmerdb", line 5, in <module>
    from kmerdb import cli
ModuleNotFoundError: No module named 'kmerdb'

$ cat path/to/.venv/bin/kmerdb
#!/ffast2/kdb/.venv/bin/python
# -*- coding: utf-8 -*-
import re
import sys
from kmerdb import cli
if __name__ == '__main__':
    sys.argv[0] = re.sub(r'(-script\.pyw|\.exe)?$', '', sys.argv[0])
    sys.exit(cli())

It seems the bin script create does not have an appropriate PythonPath set.

[merwok]

It is strange that you’re using both project.scripts, which is a standardized way of declaring console script entry points, and setuptools options for entry points at the same time. Can you keep only the first one, build a wheel in a clean environment, install it in a fresh virtualenv and see what you get?

[MatthewRalston]

Thanks I thought of that, and I get the same result regardless of which strategy from the official tutorials I use. Maybe I should possibly downgrade pip?

Not sure how but when I updated from setup.py to pyproject.toml, it broke my entry point. Relevant commit is this open issue in MatthewRalston/kmerdb/issues/97

[MatthewRalston]

It may also be worth noting that I've tried using console_scripts = {kmerdb = 'kmerdb:cli'} as well in both setup.cfg and setup.py, independently, as well as in combination and nothing seems to be setting the pythonpath correctly with the bin script when installing from a python -m build (build module on PyPI`) wheel.

[MatthewRalston]

In the past my build process was as follows:

python setup.py sdist bdist_wheel
auditwheel repair --plat linux_x86_64 dist/kmerdb-*linux_x86_64.whl

pip install dist/kmerdb-***.whl
# obviously replacing the info to the output from audtiwheel 

[merwok]

I don’t know if we can trust that the install will clean up previous script, hence why I recommended a clean build (don’t keep temporary build dirs) and a clean install in a new venv. Limit the possibility of wrong observation!

[MatthewRalston]

I'm asking why python -m build i.e. the recommended way to produce a wheel, gets installed with a bonk bin script with my setuptools stack. I've even offered the pyproject.toml permutations I've tried both in the linked issue MatthewRalston/kmerdb/issues/97 and in this issue here.

My repository remains open so you can see the difference between the legacy setup.py and the pyproject.toml on my master branch. It's not the virtualenv, I've blown it away many times.

I've referenced both the legacy setup.py that produces a correct entry point, and I've commented that in a FRESH virtualenv, .venv including the versions of virtualenv, setuptools, pip, and Python the installation does not produce a correct bin script when using pyproject.toml

I don't expect this is the easiest issue to debug, and I really don't need any special considerations here. It's genuinely an issue with my setuptools stack and I've offered both my configurations that I've tried, as well as the version numbers of the tools I'm using.

I haven't been asked a specific question I can provide more details on, and I've explained the issue pretty well.

Thanks again for looking at this.

[henryiii]

The PEP explicitly disallows setting console_scripts inside options.entry_points. One focus of the PEP is "one way to do things"; this would have been two ways to do the same thing, so it's disallowed.

This should be up to the installer (pip) - this script gets generated when you install (that's the only time it would know what paths to inject). The console script in both cases just fills a metadata slot in the wheel, and when you install, it just reads this metadata entry and generates the script. Do you have a before and after wheel? Looking at the metadata differences of both would be enlightening, I think.

[MatthewRalston]

No, I generate one wheel in this iteration of the build cycle with the new python -m build recommended system. I've had to revert my repository to standard setup.py, as the entry_points seem to be misconfigured when I use pyproject.toml. The odd thing, is I essentially have 3 config files total: setup.cfg, setup.py, and pyproject.toml, some of which are required by TravisCI and/or ReadTheDocs. Removing either setup.py and setup.cfg did not resolve this problem; I'm not entirely sure what component of the setuptools, python -m build, and pip is creating the problematic bin script.

Again:

• python -m mymodule works in my fresh virtualenv
• import mymodule works in my fresh virtualenv
• mymodule -h invokes path/to/.venv/bin/mymodule which doesn't work, complaining that mymodule doesn't exist

[henryiii]

I meant can we get a “broken” (pyproject.toml) wheel and a working (setup.py) wheel and compare them? Even if the working one is slightly older. All we need to do is look in the scripts directory of the wheel & at the entry_points.txt file. Wheels are just zip files.

[MatthewRalston]

Ahhh!!! Yes I understand. I'll upload asap

[MatthewRalston]

From kmerdb 0.7.0 (the older setup.py referenced above with console_scripts.

• entry_points.txt (0.7.0 | setup.py)

[console_scripts]
kmerdb = kmerdb:cli

• entry_points.txt (0.7.2 | pyproject.toml)

[console_scripts]
kmerdb = kmerdb:cli

It seems like the pyproject.toml wheel does not include my module in the wheel file, in addition to the dist-info

Again, here is the relevant pyproject.toml

[MatthewRalston]

Adding the following to pyproject.toml satisfies the package discovery. I'm not sure why pyproject.toml alone does not include my package.

[tool.setuptools]
include-package-data = true
packages = ['kmerdb']