Release tarballs don't include all required files/directories (e.g. missing tests dir)

[coreycb]

In working on Ubuntu packaging I noticed that the pytest.main() call in setup.py isn't able to find the tests directory. We need to run tests during package builds in order to keep python-hypothesis in main.

It seems there are a number of files/directories not included in the tarball.

For example, the root directory in the v1.11.0 tag of the git repository is:

appveyor.yml docs hypothesis-extra notebooks scripts src tox.ini
CONTRIBUTING.rst examples LICENSE.txt README.rst setup.py tests

whereas the root directory of the released tar [1] is:

PKG-INFO README.rst setup.cfg setup.py src

[1] https://pypi.python.org/packages/source/h/hypothesis/hypothesis-1.11.0.tar.gz

[DRMacIver]

Pypi releases contain what is needed to install the package from pypi, as is right and proper. If you want release tarballs of the whole repo there's the github releases page which has complete tarballs.

But... I've got to be honest, if you're trying to package 1.11.0 I'd rather you simply didn't. It's 5 minor versions behind already, which includes a lot of bug fixes, performance improvements and new features, and I'm not really looking forward to the prospects of having people complaining about bridges Hypothesis being broken because they're stuck on an old version.

[tomprince]

Having the tests in the package allow them to be run as part of the package build process. Having them in the tarball doesn't necessitate them being installed. (See https://github.com/pyca/cryptography/blob/master/MANIFEST.in#L12 for example).

[DRMacIver]

Packaging them in github release tarballs also allows you to do that and I don't see any particularly compelling reason to include them on pypi when that's not part of pypi's purpose.

[tomprince]

Pypi releases contain what is needed to install the package from pypi, as is right and proper.

Clearly people disagree with what is "right and proper". Certainly one purpose is to provide the means to required
to install the package (but if you want to optimize that case, then it would be better to provide a wheel in addition to the sdist anyway).

Looking at some popular packages on PyPI, all the following packages include tests or docs or both in their sdists:

• pip
• setuptools
• requests
• cryptography
• django
• boto
• lxml
• python-dateutil
• simplejson
• six
• setuptools
• twisted

If you want release tarballs of the whole repo there's the github releases page which has complete tarballs.

At least one issue with this is that tags on github can be changed. By putting the release on PyPI avoids this issue. in addition (and somewhat relatedly), Fedora prefers packaging from upstream provided tarballs, rather than autotgenerated ones from github. If the packages on PyPI don't contain tests or documentation then it is impossible to include them in distribution packages.

[kxepal]

I think here two categories of users clashes with different requirements.

First group is regular users which installs package via direct pip install as third party dependency and would like to use it right now. They don't interested in internal tests as they will never run them (and it's a shame that installation process doesn't involves any checks if package is not broken, but that's another sad story) and they also doesn't interested in builtin docs as they prefer read them on some service like RTD or PythonHosted where they are pretty rendered. So it's indeed sounds unreasonable to boat tarballs with the bits that will never be used.

Second group is package maintainers who want to install things right: check tarball signature (if any), run tests before make a package, pregenerate docs and put them into specific directory etc. These people needs all the bits all the time to provide high quality of their work and service for their users.

So two groups needs different tarballs, but unlikely you would like to maintain two versions of releases. Responsibility reduction is a good thing and that's why to provide one release tarball that suites everyone.

[tomprince]

I think here two categories of users clashes with different requirements.

If one provides wheels as well as sdists, you can serve both groups with files that serve both groups optimally. And it is just as easy to upload a wheel as well as an sdist. From the documentation of twine[1]:

$ python setup.py sdist bdist_wheel
$ twine upload dist/*

[1] twine is preferred as it always verifies HTTPS when uploading (and also allows you to sign the uploaded files).

[DRMacIver]

I really don't care, and I'm really not interested in arguing the point.