Testsuite for src/setup.py

[mkoeppe]

We should have a testsuite that tests the correct operation and interaction of various commands that a distutils-based setup.py should support.

Does something like this exist in the Python world already?

CC: @jdemeyer @embray @vbraun

Component: build

Issue created by migration from https://trac.sagemath.org/ticket/21678

[vbraun]

comment:1

Not that I know of; Usually you realize pretty quickly that you can't install your project.

Any non-trivial functionality of setup.py should be implemented in the sage_setup library which can be unit tested in the usual way.

[mkoeppe]

comment:2

Replying to @vbraun:

Not that I know of; Usually you realize pretty quickly that you can't install your project.

Any non-trivial functionality of setup.py should be implemented in the sage_setup library which can be unit tested in the usual way.

Yes, but I am thinking of higher-level tests, for example to ensure that making an sdist and then unpacking-building-cleaning succeeds and gives a tree that is the same as in the sdist archive. This is currently not tested at all and broken (#21516).

[embray]

comment:3

That's not a bad idea. astropy-helper has a number of functional tests to this effect: https://github.com/astropy/astropy-helpers/tree/master/astropy_helpers/tests

They're specifically geared toward testing the code in astropy-helpers itself, but this tests a lot of the other expected outputs of setup.py in the process.

[mkoeppe]

comment:4

Ticket #21516 (Fix sagelib sdist) now has a test (sdistcheck) but it currently fails.

[jdemeyer]

comment:6

Just a pointer: I implemented a pretty extensive installation test suite for cysignals. It tests installation with --prefix, --root and --user. It tests setup.py sdist and tests distclean.

See https://github.com/sagemath/cysignals/blob/master/Makefile#L71 and following.

[mkoeppe]

comment:7

Moving some tickets to 9.2. This is not a promise that I will be working on them.

[mkoeppe]

comment:8

As of #29950, we test it with tox.