Tests require the installed docs directory

[yurivict]

Describe the bug
Currently the testsuite requires nest to be installed including its docs directory.

This line runs test scripts from installed docs directory.

Problems:
(1) Some users prefer to not install docs and they can't run tests
(2) Tests should not be installed. They should run in the build directort after the project is built and should not require nest to be installed.

[github-actions]

Issue automatically marked stale!

[jougs]

Sorry for taking so long to respond to this.

1. You don't need to build or install any documentation in order to run the tests. I just checked current master with the CMake flags -Dwith-devdoc=OFF -Dwith-userdoc=OFF as well as -Dwith-devdoc=ON -Dwith-userdoc=ON and could run make installcheck without problems in both cases (after calling make install).

2. We beg to differ here because of two reasons: First, the tests are installed to the documentation directory mostly because we think that they can also serve as some sort of documentation in that they show how to do certain things with NEST. Second, we need the tests installed in cases where NEST comes packaged and the source directory is not even there anymore. The name of the make target to run the tests (installcheck) also implies that our tests test that the installed version works, not that the compilation went fine.

That said, no one stops you from running do_tests.sh with the --prefix parameter pointing to whichever NEST sources' testsuite directory you want to test without any need to call make install first.

[StefanBruens]

Sorry for taking so long to respond to this.

1. You don't need to build or install any documentation in order to run the tests. I just checked current master with the CMake flags -Dwith-devdoc=OFF -Dwith-userdoc=OFF as well as -Dwith-devdoc=ON -Dwith-userdoc=ON and could run make installcheck without problems in both cases (after calling make install).

You are missing some important aspects:

1. Builds on Linux distributions and *BSDs are done as unprivileged user, including the make install step.
2. Unprivileged users are not allowed to install into the system directories, i.e. a staging area aka DESTDIR is required.

So, please try the following:

cmake -CMAKE_INSTALL_PREFIX=/usr <other options>
make
make DESTDIR=/tmp/staging/ installcheck

DESTDIR is automatically handled correctly by CMake for install(), but ignored in most other places, e.g. as part of add_test(... COMMAND ...).

2. We beg to differ here because of two reasons: First, the tests are installed to the documentation directory mostly because we think that they can also serve as some sort of documentation in that they show how to do certain things with NEST. Second, we need the tests installed in cases where NEST comes packaged and the source directory is not even there anymore. The name of the make target to run the tests (installcheck) also implies that our tests test that the installed version works, not that the compilation went fine.

Actually, using CMAKE_INSTALL_FULL_DOCDIR has not benefit, and CMAKE_INSTALL_FULL_BINDIR only very limited:

1. Calling from CMakelists.txt (more specifically, the generated Makefile) implies the sources are still there, as well as the built binary
2. The binary may or may not work in a user setup. make installcheck only implies nest is working with all sources and the just built binaries still around. But you can't do something like make; make install; make clean; make check.

[jougs]

What exactly is it that you want to do and why? I am afraid I don't get the full picture of what you want to achieve.

Running either PyNEST or the stand-alone version of NEST requires a bunch of dynamic libraries (in <prefix>/lib/nest/) and SLI libraries (in <prefix>share/nest/sli/) for starting up the built-in interpreter that is at the very core of the simulation kernel. If those files are not installed, NEST does not know how to find them and just won't work.

The testsuite runner (testsuite/do_tests.sh) heavily relies on the SLI interpreter (<prefix>/bin/sli) for executing test scripts and as that won't work without being installed, it does not make a difference in my opinion if the testsuite is installed or not.

As a consequence of this, make installcheck does not work without having called make install first.

This all will change quite dramatically in the not-so-far-future -- we're currently working on removing SLI and all of its dependencies, but I'm afraid that right now, you can't run the testsuite without having run make install first.

Actually, using CMAKE_INSTALL_FULL_DOCDIR has not benefit, and CMAKE_INSTALL_FULL_BINDIR only very limited:

1. Calling from CMakelists.txt (more specifically, the generated Makefile) implies the sources are still there, as well as the built binary
2. The binary may or may not work in a user setup. make installcheck only implies nest is working with all sources and the just built binaries still around. But you can't do something like make; make install; make clean; make check.

I might have not been completely precise, but users who run a installed version of NEST but don't have the source or build directory anymore can always run

source <prefix>/bin/nest_vars.sh
<prefix>/share/nest/testsuite/do_tests.sh --prefix=<prefix> --report-dir=<whatever> --with-python=python3 

to run the testsuite on their own.