Add external ringtest to CTest suite #997
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This suppresses a rather verbose message: /usr/bin/which: no xcrun in (...long list of paths...)
This is conditional on the NEURON tests being enabled. This is likely to be a sensible setup for developers working on NEURON and CoreNEURON.
Previously multiple jobs would try to write to the same `.coverage` file, causing a data race and job failures if the tests were run in parallel. This forces each job to write to a separate output file.
Add several CMake helpers (nrn_add_test_group, nrn_add_test, nrn_add_test_group_comparison) and use them to integrate tests from the external neuronsimulator/ringtest repository into the CTest suite of NEURON. NEURON and CoreNEURON are run in several configurations and the results are compared against each other and a reference file. Note that, because of small numerical differences when TABLE statements are removed for CoreNEURON compatibility, the tests are not run if neither CoreNEURON nor NRN_ENABLE_MOD_COMPATIBILITY is enabled.
Many lines in this file had already been touched in recent commits.
- Support older (v3.8.2) CMake versions. - Fix a trivial error in a coverage report filename, fixing parallel runs of some tests. - Add more workarounds for issues with Python in MPI tests, which seem to be related to #894. - Fix dependencies of test targets. - Disable {Python, MPI} tests when those features are disabled. - Use the CORENEURONLIB environment variable so tests run reliably from working directories that are *not* where `nrnivmodl` was run. - Make {build_dir}/bin/nrnivmodl executable, previously it was only made executable when it was installed.
olupton
force-pushed
the
olupton_integration_ctests
branch
from
5c80c46
to
79c9965
Compare
pramodk
reviewed
Feb 18, 2021
Codecov Report
@@ Coverage Diff @@
## master #997 +/- ##
==========================================
+ Coverage 27.31% 27.37% +0.05%
==========================================
Files 574 574
Lines 109074 109072 -2
==========================================
+ Hits 29790 29854 +64
+ Misses 79284 79218 -66
Continue to review full report at Codecov.
|
iomaganaris
reviewed
Feb 18, 2021
- Remove some unnecessary $ENV{SHELL} now nrnivmodl is executable in the
build directory.
- Reorganise some comments.
nrnhines
reviewed
Feb 18, 2021
nrnhines
approved these changes
Feb 18, 2021
Remove extraneous Python imports and tweak a comment.
This was referenced Feb 22, 2021
alexsavulescu
pushed a commit
that referenced
this pull request
Apr 13, 2021
* Add external `ringtest` to CTest suite.
- Add several CMake helpers (nrn_add_test_group, nrn_add_test,
nrn_add_test_group_comparison) and use them to integrate tests from the
external neuronsimulator/ringtest repository into the CTest suite of
NEURON.
- NEURON and CoreNEURON are run in several configurations and the
results are compared against each other and a reference file. Note that,
because of small numerical differences when TABLE statements are removed
for CoreNEURON compatibility, the tests are not run if neither CoreNEURON
nor NRN_ENABLE_MOD_COMPATIBILITY is enabled.
* Replace `which` with `command -v`.
This suppresses a rather verbose message:
/usr/bin/which: no xcrun in (...long list of paths...)
* Run CoreNEURON tests if it is built as a submodule.
This is conditional on the NEURON tests being enabled. This is likely to
be a sensible setup for developers working on NEURON and CoreNEURON.
* Allow code coverage jobs to run in parallel.
Previously multiple jobs would try to write to the same `.coverage`
file, causing a data race and job failures if the tests were run in
parallel. This forces each job to write to a separate output file.
* cmake-format the remainder of test/CMakeLists.txt
alexsavulescu
pushed a commit
that referenced
this pull request
Apr 30, 2021
* Add external `ringtest` to CTest suite.
- Add several CMake helpers (nrn_add_test_group, nrn_add_test,
nrn_add_test_group_comparison) and use them to integrate tests from the
external neuronsimulator/ringtest repository into the CTest suite of
NEURON.
- NEURON and CoreNEURON are run in several configurations and the
results are compared against each other and a reference file. Note that,
because of small numerical differences when TABLE statements are removed
for CoreNEURON compatibility, the tests are not run if neither CoreNEURON
nor NRN_ENABLE_MOD_COMPATIBILITY is enabled.
* Replace `which` with `command -v`.
This suppresses a rather verbose message:
/usr/bin/which: no xcrun in (...long list of paths...)
* Run CoreNEURON tests if it is built as a submodule.
This is conditional on the NEURON tests being enabled. This is likely to
be a sensible setup for developers working on NEURON and CoreNEURON.
* Allow code coverage jobs to run in parallel.
Previously multiple jobs would try to write to the same `.coverage`
file, causing a data race and job failures if the tests were run in
parallel. This forces each job to write to a separate output file.
* cmake-format the remainder of test/CMakeLists.txt
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The addition in this PR is the set of
nrn_add_test*(...)CMake helpers, which are used to configure integration tests from https://github.com/neuronsimulator/ringtest as part of the CTest suite. These helpers are documented incmake/NeuronTestHelper.cmake. Note that, because of small numerical differences whenTABLEstatements are removed for CoreNEURON compatibility, theringtesttests are not run if neither CoreNEURON norNRN_ENABLE_MOD_COMPATIBILITYis enabled.This PR also enables the CoreNEURON CTest suite if the NEURON CTests are enabled and CoreNEURON is built as a submodule. This means all the tests can be run with a single
make testin this configuration, which is likely to be a sensible setup for developers working on NEURON and CoreNEURON.Minor other changes:
whichwithcommand -vto suppress a rather verbose message (/usr/bin/which: no xcrun in (...long list of paths...))cmake-formatthe remainder oftest/CMakeLists.txtas many lines were already touched in this PR.make test ARGS='-j 8'), previously there was a data race due to multiple jobs writing the same output file.