Tarballs containing a setup.py and tests

[jayvdb]

This is a bit like the common source tarball problem of including a built ./configure script - and then should the repo also include the generated ./configure script because it is a very important build artifact.

Here, it would be very useful for this repo to have a generated setup.py in the repo. Then the GitHub tag tarballs are installable using setuptools, and include the tests so that the installation can be verified.

At the moment I can not use the sdist, as it doesnt include tests .

I also can not use the github archive, as it lacks a setup.py .

And openSUSE maintainers dont like having a .spec which uses two tarballs (the sdist + github archive for the tests).

Once I have a tested & installed dephell, I can fix this problem by using dephell to generate the setup.py , but that is not possible while I am trying to build dephell. My objective is to include dephell in the build process of other packages which lack a setup.py .

fwiw, it isnt so important that the dephell plugins have tests running, if I can get a testable archive of the main dephell project. The test for the main dephell project should implicitly cover most of the plugins.

[orsinium]

I have mixed feelings about it.

Let's talk not about "how dephell should be built", but "how project build command should work", because dephell is built by dephell.

Very often, tests aren't included in sdist. For example, pipenv and pip. This is because none of tools in python ecosystem can run tests for downloaded package, so tests in sdist is redundant and just spent users traffic.

However, I got your point and understand that for some tools outside python ecosystem tests can be helpful. I think, best decision here is also make "dephell-0.7.0-with-tests.tar.gz". So, pip will choose package without tests, but package with tests can be downloaded manually if needed. What do you think?

[orsinium]

I forgot important point not to include tests in main sdist: some tools (like pipenv) has really fat "tests" dir because of fixtures.

[jayvdb]

none of tools in python ecosystem can run tests for downloaded package

What? setup.py test is integrated into setuptools. pytest-runner integrates tests into setuptools. etc.

pip and pipenv have huge test suites, so huge that the repo size is now a pain point. pypa/pipenv#3680 I hope this project thinks ahead and avoids that problem. fwiw, I just finished packaging pipenv for openSUSE https://build.opensuse.org/package/show/devel:languages:python/python-pipenv (oh lovely, it is now broken by some other dependency changing).

Typically projects of that size/importance will have a separate test data repo, and still ship the tests with the sdist (rarely larger than a few meg of .py files), and usually a large percentage of the test suite (the unit tests) will work fine without the test data , and only the integration tests will be skipped.)

And in both cases, distro packagers build from their GitHub archive, using a setup.py . The pipenv project does use a Pipfile and a pyproject.toml, but they recognise the importance of having a setup.py in the repo , because they are a build tool, and build tools usually include bootstrap files to avoid the catch-22 situations.

so tests in sdist is redundant and just spent users traffic.

No, increasingly the normal users are using wheels, not sdists, where available.

And please note I did not suggest tests in the sdist as the only option.

The other very simple approach would be to have a generated setup.py in git, removing one barrier to getting started. There are very few tools which support pyproject.toml at the moment, and the implementations are all in flux and are often using new libraries which are also rapidly changing. e.g. at the moment I can't use poetry in system RPMs because it depends on a newer version of jsonschema than the stable version that OpenStack requires, and OpenStack wins. And it also has the same problem -- its repo doesnt include a setup.py

Let's talk not about "how dephell should be built", ..

IMO that is important to talk about, as dephell can be an important tool in bootstraping other projects, and as such should be adding a setup.py to its repo , so that the other projects do not need to. Also it is a dev tool, not a user tool.

[orsinium]

What?

Don't panic, I open for changes. In my mind and in my projects. Sorry, if my messages sound uncompromising.

The other very simple approach would be to have a generated setup.py in git, removing one barrier to getting started.

Yeah, no problems, I'll add it.

The other very simple approach would be to have a generated setup.py in git

Anyway, should we include tests in the main sdist or it's better to generate other sdist with -with-tests prefix? dephell project build should work in the best way, we can't leave it broken :)

[orsinium]

Another possibility is to specify some threshold size for tests after that we won't include tests in sdist.

[jayvdb]

As far as I know, there is only one slot for an sdist per release in PyPI, but I havent tried to upload multiple for any project I am a maintainer of. Another way to see if this is possible is to query the BigQuery dataset for multiple files per release using queries a bit like psincraian/pepy#128 (comment) .

However, extra tarballs could be created and projects could choose to upload them as extra artifacts on each GitHub tag.

That has been bugging me a bit lately, as I regularly get push back on adding tests to the sdist.

I have some other more crazy ideas, sort of sparked by amazon-ion/ion-python#76 (comment)

As there can be many wheels or eggs per PyPI release, and wheels and egg filenames have a few tag components used to indicate what platforms should use which wheels/eggs, extra sdists could be uploaded to PyPI as wheels/eggs with an "invalid" platform tag, such as 'sdist' or 'sdist-really-large', which pip will never try to install because the tags would never match the local host platform identifiers. The wheels could also be valid sdists by using data dirs of ../foo-bar-1.2.3/. I suspect the egg structure can also be abused sufficiently to include a valid sdist.

Another possibility is to specify some threshold size for tests after that we won't include tests in sdist.

I really like your idea of a threshold to determine whether the tests should be automatically included because they are small, and excluded if too large. The chosen threshold isnt very important to me, and it could even be relative to the size of the core files. It creates a feature around which sensible discussion can occur about balancing priorities, and adjusting them as other factors change (e.g. as the PyPI stats show percentage of wheels increases, the need for small sdists diminishes and the threshold can be adjusted accordingly) . Allowing packages to override the threshold, and/or force inclusion or exclusion, would likely be helpful.

[orsinium]

I really like your idea of a threshold to determine whether the tests should be automatically included because they are small, and excluded if too large.

Ok, let's do it in this way. Thank you :)