Building DGSWEM HPX on Stampede

Build HPX

Building HPX on Stampede

Load Modules

module swap intel/15.0.2 gcc/4.9.1

module load boost/1.55.0

module load cmake/3.1.0

Build and Install LibGeoDecomp

git clone https://github.com/gentryx/libgeodecomp.git

Download build script: build_lgd_release_stampede.sh

Clone and Build DGSWEM/HPX

git clone https://github.com/foci/dgswem.git

cd dgswem

git checkout -b dgswem-hpx-dev remotes/origin/dgswem-hpx-dev

mkdir dgswem-hpx-dev/build_release

cd dgswem-hpx-dev/build_release

cmake -DCMAKE_CXX_COMPILER=/opt/apps/gcc/4.9.1/bin/g++ -DCMAKE_C_COMPILER=/opt/apps/gcc/4.9.1/bin/gcc -DCMAKE_PREFIX_PATH=$WORK/local_install/ -DCMAKE_BUILD_TYPE=Release ..

make dgswem_lgd_dataflow

Running dgswem_hpx

You need to build and run adcprep (make adcprep) and run it in the directory as you normally would for a parallel DGSWEM run. However, the number of subdomains you divide the grid into does not need to match the number of compute cores you plan to use (but generally you should break it into at least that number).

/PATH/TO/dgswem-hpx-dev/build_release/src/dgswem_lgd_dataflow

The application will run into problems if the ulimit -n is not set high enough. (This hasn't been a problem on Stampede so far.)

Sample slurm batch script

When running an HPX application, instead of using -n , use -N -n . Also, there is no restriction on the number of compute nodes or cores and the number of subdomains the problem is divided into.

run_hpx.slurm