Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Tests: convert quick tests into regular tests #14558

Merged
merged 7 commits into from Dec 12, 2022

Conversation

tamiko
Copy link
Member

@tamiko tamiko commented Dec 10, 2022

This has irked me quite a bit: our quick tests duplicate a lot of
functionality that is already available in the testsuite proper. So,
instead of reinventing the wheel, let us simply use the testsuite itself.

The following pull request:

  • refactors all quick tests into regular tests that are configured for all
    available build modes.

  • modifies the test target to properly depend on the library
    and the setup_tests_quick_tests target. This ensures that when calling
    make test or ninja test the library is compiled and quick tests are
    properly configured.

  • the test target will now execute run_quick_tests.cmake which does
    little more than invoking

    ctest -JN -R "quick_tests/(step.debug|step.release|affinity|.*.${_build_type})"

    which mimicks the old behavior, meaning, we only run quick tests in one
    configuration with the exception of the step and affinity tests.

Requires #14559

@tamiko
Copy link
Member Author

tamiko commented Dec 10, 2022

/rebuild

# If we used message(SEND_ERROR ...) at the top (with the actual error
# message) then subsequent messages (i.e., the test specific help) would not
# be printed. Hence, do it down here.
message(SEND_ERROR "")
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This workaround has been unnecessary for a really long time...

@tamiko
Copy link
Member Author

tamiko commented Dec 10, 2022

tamiko@peregrine dealii/build % ninja test          
[2/2] Running quicktests...
-- Running quick_tests in DEBUG mode with -j16:
Test project /home/tamiko/workspace/dealii/build
      Start  9: quick_tests/step.debug
      Start 13: quick_tests/umfpack.debug
      Start  5: quick_tests/gmsh.debug
      Start 10: quick_tests/sundials-ida.debug
      Start  4: quick_tests/boost_zlib.debug
      Start  3: quick_tests/assimp.debug
      Start  8: quick_tests/lapack.debug
      Start  2: quick_tests/affinity.debug
      Start 12: quick_tests/tbb.debug
      Start  7: quick_tests/kokkos.debug
      Start 11: quick_tests/symengine.debug
      Start  6: quick_tests/gsl.debug
      Start  1: quick_tests/adolc.debug
 1/13 Test  #7: quick_tests/kokkos.debug .........   Passed    0.07 sec
 2/13 Test  #1: quick_tests/adolc.debug ..........   Passed    0.08 sec
 3/13 Test #12: quick_tests/tbb.debug ............   Passed    0.09 sec
 4/13 Test  #6: quick_tests/gsl.debug ............   Passed    0.09 sec
 5/13 Test #11: quick_tests/symengine.debug ......   Passed    0.09 sec
 6/13 Test  #5: quick_tests/gmsh.debug ...........   Passed    0.10 sec
 7/13 Test  #4: quick_tests/boost_zlib.debug .....   Passed    0.10 sec
 8/13 Test  #2: quick_tests/affinity.debug .......   Passed    0.10 sec
 9/13 Test  #3: quick_tests/assimp.debug .........   Passed    0.10 sec
10/13 Test #13: quick_tests/umfpack.debug ........   Passed    0.11 sec
11/13 Test  #8: quick_tests/lapack.debug .........   Passed    0.10 sec
12/13 Test #10: quick_tests/sundials-ida.debug ...   Passed    0.11 sec
13/13 Test  #9: quick_tests/step.debug ...........***Failed    6.65 sec

92% tests passed, 1 tests failed out of 13

Total Test time (real) =   6.65 sec

The following tests FAILED:
          9 - quick_tests/step.debug (Failed)
Errors while running CTest
Output from these tests are in: /home/tamiko/workspace/dealii/build/Testing/Temporary/LastTest.log
Use "--rerun-failed --output-on-failure" to re-run the failed cases verbosely.


**************************************************************************
**                                                                      **
**                Error: Some of the quick tests failed.                **
**                                                                      **
**************************************************************************

Check the files quick_tests.log and Testing/Temporary/LastTest.log located
in your build directory for error messages. Alternatively, you can run all
failing quick tests again via

        $ ctest --rerun-failed --output-on-failure -R quick_tests/

If you are unable to fix this problem, write to the mailing list linked at
https://www.dealii.org

@tamiko
Copy link
Member Author

tamiko commented Dec 11, 2022

oops I garbled the force push. Reponening

@tamiko tamiko reopened this Dec 11, 2022
@tamiko tamiko force-pushed the cmake_improve_quick_tests branch 2 times, most recently from 11e78d4 to 84b4288 Compare December 11, 2022 07:01
@tamiko
Copy link
Member Author

tamiko commented Dec 11, 2022

In reference to #1921

@masterleinad
Copy link
Member

I guess this needs #14559?

@tamiko
Copy link
Member Author

tamiko commented Dec 12, 2022

@masterleinad Yes, this requires #14559, I will rebase as soon as the other one is merged.

@tamiko
Copy link
Member Author

tamiko commented Dec 12, 2022

@masterleinad @peterrum Alright. This one is now ready for review (and ready to pass the testsuite ^^).

@masterleinad masterleinad merged commit 7d2e969 into dealii:master Dec 12, 2022
@masterleinad masterleinad deleted the cmake_improve_quick_tests branch December 12, 2022 22:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants