The tests directory is no longer distributed

[penguinpee]

Fedora recently updated to Python 3.12 in rawhide (development branch). As part of that all Python packages were rebuild. This brought to light that the tests directory is no longer distributed. Neither the GitHub tarball nor the PyPI tarball contain it.

Since tests are important in order to catch regressions early, e.g. when upgrading to a new Python release, I would like to ask you to include the tests in the tarball either here or on PyPI, preferably both.

Looking at #71 it seems they were included in the past.

[hroncok]

The github trarball actually includes tests. It's only the PyPI sdist that does not.

[thebjorn]

The packaging code has been stable for "a while" (setup.py only lists the package, and the github release uses ncipollo/release-action@v1). Can you work with the tarball, or do you want me to include the tests in the PyPI package also? (I don't think any of my PyPI packages include tests...)

[penguinpee]

The github trarball actually includes tests. It's only the PyPI sdist that does not.

It does not. The package uses the forge macros. So, we are pulling the tarball from GitHub. And it does not contain a tests directory.

Tarball used by spec file: https://github.com/thebjorn/pydeps/archive/v1.12.8/pydeps-1.12.8.tar.gz

This is for the PR updating the package.

The GitHub tarball for 1.11.1 does include it, indeed. However the tarball in the side cache does not. Maybe that was downloaded from PyPI. I don't know. But simple to verify:

fedpkg clone pydeps
cd pydeps
fedpkg sources
tar tzf pydeps-1.11.1.tar.gz | grep tests

[hroncok]

The Feodora package is in an inconsistent state, I've commented the details in https://src.fedoraproject.org/rpms/pydeps/pull-request/2# because I didn't want to bother upstream with Fedora-specific things.

[thebjorn]

The gitlab release .whl and tar.gz contains the same as released on PyPI (i.e. no tests), the Source code (zip) and (tar.gz) do contain the tests.

[penguinpee]

(dev38) go|c:\srv\tmp> tar xvf pydeps-1.12.8.tar
x pydeps-1.12.8/
x pydeps-1.12.8/LICENSE
x pydeps-1.12.8/PKG-INFO
x pydeps-1.12.8/README.rst
x pydeps-1.12.8/pydeps/
x pydeps-1.12.8/pydeps/init.py
x pydeps-1.12.8/pydeps/main.py
x pydeps-1.12.8/pydeps/arguments.py
x pydeps-1.12.8/pydeps/cli.py
x pydeps-1.12.8/pydeps/colors.py
x pydeps-1.12.8/pydeps/configs.py
x pydeps-1.12.8/pydeps/depgraph.py
x pydeps-1.12.8/pydeps/depgraph2dot.py
x pydeps-1.12.8/pydeps/dot.py
x pydeps-1.12.8/pydeps/dummymodule.py
x pydeps-1.12.8/pydeps/mf27.py
x pydeps-1.12.8/pydeps/mfimp.py
x pydeps-1.12.8/pydeps/package_names.py
x pydeps-1.12.8/pydeps/py2depgraph.py
x pydeps-1.12.8/pydeps/pycompat.py
x pydeps-1.12.8/pydeps/pydeps.py
x pydeps-1.12.8/pydeps/pystdlib.py
x pydeps-1.12.8/pydeps/render_context.py
x pydeps-1.12.8/pydeps/target.py
x pydeps-1.12.8/pydeps.egg-info/
x pydeps-1.12.8/pydeps.egg-info/PKG-INFO
x pydeps-1.12.8/pydeps.egg-info/SOURCES.txt
x pydeps-1.12.8/pydeps.egg-info/dependency_links.txt
x pydeps-1.12.8/pydeps.egg-info/entry_points.txt
x pydeps-1.12.8/pydeps.egg-info/requires.txt
x pydeps-1.12.8/pydeps.egg-info/top_level.txt
x pydeps-1.12.8/setup.cfg
x pydeps-1.12.8/setup.py

I don't see a tests directory listed for the tarball.

Can you work with the tarball

Yes, I can work with the GitHub tarball.

The gitlab release .whl and tar.gz contains the same as released on PyPI (i.e. no tests), the Source code (zip) and (tar.gz) do contain the tests.

Well, what is downloaded and what I checked are the GitHub and PyPI release tarballs. I could probably check to see if we can use the source tarball. But if it's easy to include tests in the GitHub release tarball, I'd prefer that.

[thebjorn]

Ok, give me a couple of minutes..

[penguinpee]

Hold on. It seems we are using the source tarball after all. I must have made a mistake when uploading to the side cache. So, we should be good without any modifications.

Sorry for all the fuzz.

[thebjorn]

Lolz. Well, 1.12.9 is out and it contains the tests :-)

[penguinpee]

Thanks for the quick action. It doesn't hurt. I will need to step away from the keyboard for a bit. I must have taken a wrong turn somewhere. My apologies for all the fuzz.

But it's all settled now. I've updated to 1.12.9 right away and made sure everything is in place. Feel free to revert your changes again, if you prefer.

I did see an error running tests with the 1.12.9 release. I'll look into it and file a new issue if it turns out to be an upstream issue, not a Fedora quirk.

[thebjorn]

There is an mf directory containing snapshots of the standard python modulefinder module at various versions (including the 2.7 version which is causing your issue). They're just stored for reference they're not imported anywhere... Looks like your build step is trying to compile everything?

[thebjorn]

Excluding files/directories from packaging was a bit more trouble than I anticipated, so I've moved the pydeps/mf directory to docs. Version 1.12.10 will be available on PyPI/github as soon as CI/CD har run.

[penguinpee]

Excluding files/directories from packaging was a bit more trouble than I anticipated, so I've moved the pydeps/mf directory to docs.

With packages_find() introduced in 1.12.9 the tests directory will be included and installed as a top level module. There's several options to make that work:

1. Remove the packages option entirely. Enables autodiscovery, which will exclude tests by default.
2. Add include=['pydeps*'] in packages_find()
3. Add exclude=['tests*'] in packages_find()

With pydeps/mf moved to docks, that should be all. If you wanted to exclude that directory in it's original place, I'd go for option 3 and expand the exclude list: exclude=['tests*', 'pydeps.mf*'].

Let me know what you prefer and I can provide a patch.

[thebjorn]

The problem I was having was that python -m build didn't pay attention to the packages_find(exclude=...) - it may well be because I was "doing it wrogn". (I was using python setup.py sdist bdist_wheel previously.

Upon closer inspection, the tar.gz file contains the tests, but the .whl file does not (with packages_find(exclude=["tests*"])), so I'm guessing that is correct.

Version 1.12.12 is available on PyPI and github with this setup.

[penguinpee]

exclude and include are globs. The asterisk is easily overlooked and forgotten making the glob fail. I've been there... 😉