Tests try to install to system directory

[toddrme2178]

I am packaging scikit-build for openSUSE Linux and we are running into issues where the tests try to install to a system purelib directory that the build system doesn't have write access to. It would be helpful if the tests were configured to install to a temporary directory rather than a system directory.

For example tests/test_hello_cpp.py::test_hello_develop installs to:

/usr/lib/python3.5/site-packages/test-easy-install-1853.write-test

The specific tests that are causing this problem are:

tests/test_hello_cpp.py::test_hello_develop
tests/test_issue274_support_default_package_dir.py::test_install_command
tests/test_issue274_support_default_package_dir.py::test_test_command
tests/test_issue274_support_one_package_without_package_dir.py::test_install_command
tests/test_issue274_support_one_package_without_package_dir.py::test_test_command
tests/test_issue334_configure_cmakelists_non_cp1252_encoding.py::test_install_command

[bnavigator]

What exactly are you trying to accomplish by packaging it yourself?

The 'official' specfile for python-scikit-build on openSUSE leaves out these tests for exactly this reason:

%check
# setup.py install, develop, etc default to writing to /usr .
#  Tests need enhancing upstream or patching.
%pytest -k 'not (test_install_command or test_develop_command or test_test_command or test_hello_develop)'

The tests are not needed for a proper rpm package

[toddrme2178]

@bnavigator I am trying to get the tests working in the official openSUSE packages so we can catch issues early. We are having a build issue with blosc, for example, and having the tests working can help us figure out where exactly the problem lies. That is why we try to have tests running whenever possible, and when we can't we try to work with upstream to get them working.

[bnavigator]

Okay here is how I proposed to fix it in the openSUSE package:

• Create an empty custom sitelib in /tmp/fakepythonroot/usr/lib/python*/site-packages
• Add the following setup.cfg to the three failing sample projects

[develop]
prefix=/tmp/fakepythonroot/usr/

[install]
prefix=/tmp/fakepythonroot/usr/

• Adjust test_hello_cpp::test_hello_sdist to find the additional file.

Simply adding a --prefix argument to all the calls of setup.py develop and setup.py install would also work for most tests. But it is not possible for the (deprecated) calls of setup.py test because scikit-build calls develop without any arguments in its test wrapper.

I considered writing a PR for this, but I am struggling how to implement the creation of the fake sitelib. Maybe a module- or session-wide pytest fixture? However, passing the path to the projects would need to be done in code instead of the static setup.cfg then. That again is blocked by the wrapped call of develop in the test command.

[mgorny]

Can't the tests be modified to use a temporary install root?

Even when you're running in a dedicated test root, it gets "tainted" after a single test run since the installed packages are left there. If I diff the contents of .nox/tests-3-10 before and after pytest runs, the following extra files appear:

+.nox/tests-3-10/lib/python3.10/site-packages/easy-install.pth
+.nox/tests-3-10/lib/python3.10/site-packages/fail_outside_project_root-0.0.1-py3.10-linux-x86_64.egg
+.nox/tests-3-10/lib/python3.10/site-packages/fail_outside_project_root-0.0.1-py3.10-linux-x86_64.egg/EGG-INFO
+.nox/tests-3-10/lib/python3.10/site-packages/fail_outside_project_root-0.0.1-py3.10-linux-x86_64.egg/EGG-INFO/dependency_links.txt
+.nox/tests-3-10/lib/python3.10/site-packages/fail_outside_project_root-0.0.1-py3.10-linux-x86_64.egg/EGG-INFO/not-zip-safe
+.nox/tests-3-10/lib/python3.10/site-packages/fail_outside_project_root-0.0.1-py3.10-linux-x86_64.egg/EGG-INFO/PKG-INFO
+.nox/tests-3-10/lib/python3.10/site-packages/fail_outside_project_root-0.0.1-py3.10-linux-x86_64.egg/EGG-INFO/SOURCES.txt
+.nox/tests-3-10/lib/python3.10/site-packages/fail_outside_project_root-0.0.1-py3.10-linux-x86_64.egg/EGG-INFO/top_level.txt
+.nox/tests-3-10/lib/python3.10/site-packages/hello-1.2.3-py3.10-linux-x86_64.egg
+.nox/tests-3-10/lib/python3.10/site-packages/hello-1.2.3-py3.10-linux-x86_64.egg/EGG-INFO
+.nox/tests-3-10/lib/python3.10/site-packages/hello-1.2.3-py3.10-linux-x86_64.egg/EGG-INFO/dependency_links.txt
+.nox/tests-3-10/lib/python3.10/site-packages/hello-1.2.3-py3.10-linux-x86_64.egg/EGG-INFO/not-zip-safe
+.nox/tests-3-10/lib/python3.10/site-packages/hello-1.2.3-py3.10-linux-x86_64.egg/EGG-INFO/PKG-INFO
+.nox/tests-3-10/lib/python3.10/site-packages/hello-1.2.3-py3.10-linux-x86_64.egg/EGG-INFO/SOURCES.txt
+.nox/tests-3-10/lib/python3.10/site-packages/hello-1.2.3-py3.10-linux-x86_64.egg/EGG-INFO/top_level.txt
+.nox/tests-3-10/lib/python3.10/site-packages/hello.egg-link

Not only this is a hassle for downstream packagers who need to clean up after the test suite but I can well imagine that this could cause false test results in the future, when someone runs tests more than once.