Skip to content

Commit

Permalink
This commit introduces the following environment variables that affect
Browse files Browse the repository at this point in the history
`make test` (or `ctest`) behavior of ALPSCore:

| Variable                    | Default                 | Usual value    | Meaning |
|-----------------------------|-------------------------|----------------|---------|
| `ALPS_TEST_MPIEXEC`           | `${MPIEXEC}`             | `mpiexec`      | MPI launcher                                                      |
| `ALPS_TEST_MPI_NROC_FLAG`     | `${MPIEXEC_NUMPROC_FLAG}`| `-n`           | flag to specify the number of MPI processes                       |
| `ALPS_TEST_MPI_NPROC`         | 1                        | 1              | How many MPI processes to launch in MPI-enabled tests             |
| `ALPS_TEST_MPIEXEC_PREFLAGS`  | `${MPIEXEC_PREFLAGS}`    | (empty string) | MPI launcher arguments preceding the executable name              |
| `ALPS_TEST_MPIEXEC_POSTFLAGS` | `${MPIEXEC_POSTFLAGS}`   | (empty string) | MPI launcher arguments preceding the arguments for the executable |

The `${...}` above are CMake variables, normally set by `FindMPI` module.

Related: issue #211.

This should close #296.

Intended use:

**Case 1: Vanilla MPI-enabled environment.**

The command to run an MPI program using 2 processes: `mpiexec -n 2 some_test`

Setting the variables to run each MPI-enabled tests on 2 processes: `ALPS_TEST_MPI_NPROC=2 make test`

**Case 2: NERSC Cori**

(*Disclaimer:* not tested with an actual Cori run)

Users are not supposed to run `mpiexec`. One has to allocate interactive nodes first.

Allocating 2 Haswell nodes for 30 minutes: `salloc -N 2 -C haswell -q interactive -t 0:30:00`

Command to run on the allocated nodes: `srun some_test`

Setting the variables to run each MPI-enabled tests on the allocated nodes:
`ALPS_TEST_MPIEXEC=srun ALPS_TEST_MPI_NPROC=' ' ALPS_TEST_MPI_NPROC_FLAG=' ' make test`

(note the variables are assigned spaces, not empty strings!)

**Case 3: Blue Waters**

(*Disclaimer:* not tested on actual Blue Waters machine)

The `aprun` command is supposed to be used to launch parallel processes from an interactive node
(see https://bluewaters.ncsa.illinois.edu/using-aprun ).

Command to run on 16 cores, using 8 cores per node (that is, 2 nodes), placing the processes on adjacent cores:
`aprun -N 8 -d 1 -n 16 some_test`

Setting the variables to run each MPI-enabled tests with this configuration:
`ALPS_TEST_MPIEXEC=aprun ALPS_TEST_MPI_NPROC=16 ALPS_TEST_MPI_NPROC_FLAG='-N 8 -d 1 -n' make test`
  • Loading branch information
galexv committed Mar 29, 2019
1 parent c1be569 commit 9b464ee
Showing 1 changed file with 13 additions and 4 deletions.
17 changes: 13 additions & 4 deletions common/cmake/ALPSEnableTests.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -120,10 +120,19 @@ function(alps_add_gtest test)
set(run_ok_cmd_ "$<TARGET_FILE:${test}> ${test_xml_output_}")
set(shell_cmd_ "if ${build_cmd_}\; then ${run_fail_cmd_}\; false\; else ${run_ok_cmd_}\; fi")
set(cmd_ "/bin/sh" "-c" "${shell_cmd_}")
elseif (arg_PARTEST AND MPIEXEC)
# FIXME: if compiler supports MPI directly, the MPIEXEC program is not deduced!
# FIXME: in the MPI test command, POSIX shell is assumed
set(cmd_ "/bin/sh" "-c" "${MPIEXEC} ${MPIEXEC_NUMPROC_FLAG} \${ALPS_TEST_MPI_NPROC:-1} ${MPIEXEC_PREFLAGS} $<TARGET_FILE:${test}> ${MPIEXEC_POSTFLAGS} ${test_xml_output_}")
elseif (arg_PARTEST AND ALPS_HAVE_MPI)
# if unspecified, we assume the standard mpi launcher, according to MPI-3.1 specs, Chapter 8.8
# (see https://www.mpi-forum.org/docs/mpi-3.1/mpi31-report/node228.htm#Node228)
if (NOT MPIEXEC)
set(MPIEXEC "mpiexec")
endif()
if (NOT MPIEXEC_NUMPROC_FLAG)
set(MPIEXEC_NUMPROC_FLAG "-n")
endif()

# we also allow test-time override of the MPI launcher and its arguments
# (NOTE: POSIX shell is assumed)
set(cmd_ "/bin/sh" "-c" "\${ALPS_TEST_MPIEXEC:-${MPIEXEC}} \${ALPS_TEST_MPI_NPROC_FLAG:-${MPIEXEC_NUMPROC_FLAG}} \${ALPS_TEST_MPI_NPROC:-1} \${ALPS_TEST_MPIEXEC_PREFLAGS:-${MPIEXEC_PREFLAGS}} $<TARGET_FILE:${test}> \${ALPS_TEST_MPIEXEC_POSTFLAGS:-${MPIEXEC_POSTFLAGS}} ${test_xml_output_}")
else()
set(cmd_ $<TARGET_FILE:${test}> ${test_xml_output_})
endif()
Expand Down

0 comments on commit 9b464ee

Please sign in to comment.