Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TEST_MPI_RANKS=auto does not account for ctest parallelization #706

Closed
LecrisUT opened this issue Sep 15, 2023 · 1 comment
Closed

TEST_MPI_RANKS=auto does not account for ctest parallelization #706

LecrisUT opened this issue Sep 15, 2023 · 1 comment

Comments

@LecrisUT
Copy link
Contributor

LecrisUT commented Sep 15, 2023

For example, if you have ctest -j 2, there will be 2 tests running in parallel. But the auto calculation gets the maximum number of processors through ProcessorCount and defines it statically to be used in mpiexec. This leads to the cores being oversubscribed and potentially switching between the tests.

if (TEST_MPI_RANKS STREQUAL "auto")
include(ProcessorCount)
ProcessorCount(nproc)
math(EXPR num_ranks "(${nproc}+${TEST_OMP_THREADS}-1)/${TEST_OMP_THREADS}"
)# get 1/$TEST_OMP_THREADS the number of procs (rounded up)

On the cmake side, I am not sure if there is clear way to define this. The documentation is a bit vague on how to design this. Maybe there is some internal handling when using the PROCESSORS property. This should be investigated with the cmake team (discussion).

@alazzaro
Copy link
Member

For your case I would suggest to set the number of ranks via -DTEST_MPI_RANKS. As you said there is not a clean solution for this problem, but please let us know if you can find something. Note that tests are really meant to check correctness and not performance.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants