Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add performance testing framework: preparatory refactoring #13350

Merged
merged 9 commits into from Feb 9, 2022

Conversation

tamiko
Copy link
Member

@tamiko tamiko commented Feb 9, 2022

@peterrum and I are currently working on bringing back performance tests to deal.II. Our idea is to integrate performance tests as a part of the testsuite with the following rationale:

  • By default performance tests will be compiled and linked but do not run (with expected stage RUN or DIFF). This will ensure that compilation and linking of performance test code is part of our CI and does not diverge from the library API.
  • We will use ctest (and our testsuite infrastructure) to compile and run performance tests that simply report (a best of three) timing result to the output file. These output files can then further be postprocessed by a script to create a table. All of this will be chained together in a run_performance_tests.cmake ctest script.

This PR adds some necessary features to the current testsuite infrastructure:

  • implement thread concurrency annotation via [...].threads=XY.output
  • the ability to set MPI rank and thread limits via TEST_THREAD_LIMIT and TEST_MPI_RANK_LIMIT. Tests that specify a higher number of MPI ranks or thread concurrency will be dropped.
  • allow to use maximal concurrency by setting .threads=max. or .mpirun=max..

@tamiko
Copy link
Member Author

tamiko commented Feb 9, 2022

/rebuild

Copy link
Member

@peterrum peterrum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me! Thanks for making the first steps!

Since I am too much involved here, I would appreciate a second opinion!

Copy link
Member

@masterleinad masterleinad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me.

Copy link
Member

@peterrum peterrum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's merge. More is coming!

@peterrum peterrum merged commit 276c665 into dealii:master Feb 9, 2022
@tamiko tamiko deleted the performance_tests_01 branch November 30, 2022 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants