DASH, the C++ Template Library for Distributed Data Structures with Support for Hierarchical Locality for HPC and Data-Driven Science
Switch branches/tags
Clone or download
fmoessbauer Merge pull request #599 from dash-project/sup-cxx14-593-fix
Remove enforcement of C++11 to enable C++14
Latest commit 1c75574 Oct 18, 2018
Permalink
Failed to load latest commit information.
.circleci remove enforcement of C++11 to enable C++14 Oct 18, 2018
CMakeExt Merge branch 'development' into dash-bump-c++14 Oct 14, 2018
config Remove misleading comment Mar 7, 2017
contrib added .clang-tidy file May 23, 2018
dart-if Merge branch 'development' into feat-dart-memallocator Sep 28, 2018
dart-impl Merge branch 'development' into feat-dart-memallocator Sep 28, 2018
dash ci:minimal use papi 5.5.1 Oct 15, 2018
doc Update Doxyfile.in Mar 24, 2017
vendor fixed globiter to be trivially copyable and added test, remove member… Feb 9, 2018
.codecov.yml fix codecov status when coverage drops slightly Jun 22, 2018
.codedocs Relative path format in codedocs configuration Oct 14, 2016
.gitignore Git: Ignore all hidden files by default Sep 21, 2018
.gitmodules Move content of `.gitignore` to `.gitmodules`. Mar 2, 2018
CHANGELOG.md Mention compiler wrapper in changelog Oct 1, 2017
CMakeLists.txt use dash-ci for coverage builds, fixed formatting of test-log output Jun 1, 2018
CONTRIBUTING.md Update CONTRIBUTING.md Oct 18, 2017
COPYING.md Change HLRS to 'University of Stuttgart (HLRS)' (IHR has no contribut… Mar 23, 2017
README.md added codecov badge, re-enabled Halo tests May 4, 2018
build.analyze.sh Introduce macro DART_MPI_DYNAMIC_WINDOWS and CMake var ENABLE_DYNAMIC… Jul 14, 2017
build.cov.sh enable debug in coverage tests May 7, 2018
build.debug.sh Introduce macro DART_MPI_DYNAMIC_WINDOWS and CMake var ENABLE_DYNAMIC… Jul 14, 2017
build.dev.sh Introduce macro DART_MPI_DYNAMIC_WINDOWS and CMake var ENABLE_DYNAMIC… Jul 14, 2017
build.mic.sh Introduce macro DART_MPI_DYNAMIC_WINDOWS and CMake var ENABLE_DYNAMIC… Jul 14, 2017
build.minimal.sh Introduce macro DART_MPI_DYNAMIC_WINDOWS and CMake var ENABLE_DYNAMIC… Jul 14, 2017
build.nasty.sh Introduce macro DART_MPI_DYNAMIC_WINDOWS and CMake var ENABLE_DYNAMIC… Jul 14, 2017
build.sh Introduce macro DART_MPI_DYNAMIC_WINDOWS and CMake var ENABLE_DYNAMIC… Jul 14, 2017
mkdocs.yml added documentation of DASH's atomic interface in userguide Feb 13, 2017
release.pl Add files in include/dash/std/*.h to release Sep 22, 2018

README.md

CI Status Build Status Documentation Status Documentation codecov CII Best Practices

DASH

A C++ Template Library for Distributed Data Structures with Support for Hierarchical Locality for HPC and Data-Driven Science.

Summary

Exascale systems are scheduled to become available in 2018-2020 and will be characterized by extreme scale and a multilevel hierarchical organization.

Efficient and productive programming of these systems will be a challenge, especially in the context of data-intensive applications. Adopting the promising notion of Partitioned Global Address Space (PGAS) programming the DASH project develops a data-structure oriented C++ template library that provides hierarchical PGAS-like abstractions for important data containers (multidimensional arrays, lists, hash tables, etc.) and allows a developer to control (and explicitly take advantage of) the hierarchical data layout of global data structures.

In contrast to other PGAS approaches such as UPC, DASH does not propose a new language or require compiler support to realize global address space semantics. Instead, operator overloading and other advanced C++ features are used to provide the semantics of data residing in a global and hierarchically partitioned address space based on a runtime system with one-sided messaging primitives provided by MPI or GASNet.

As such, DASH can co-exist with parallel programming models already in widespread use (like MPI) and developers can take advantage of DASH by incrementally replacing existing data structures with the implementation provided by DASH. Efficient I/O directly to and from the hierarchical structures and DASH-optimized algorithms such as map-reduce are also part of the project. Two applications from molecular dynamics and geoscience are driving the project and are adapted to use DASH in the course of the project.

Funding

DASH is funded by the German Research Foundation (DFG) under the priority programme "Software for Exascale Computing - SPPEXA" (2013-2018).

Community

Project Website:

http://www.dash-project.org

GitHub:

https://github.com/dash-project

Documentation Wiki

http://doc.dash-project.org

Repository:

Contact:

Contributing

See guidelines in CONTRIBUTING.md.

Installation

DASH installations are available as Docker containers or build from source using CMake.

Docker Containers

For pre-build Docker container images, see the DASH project on Docker Hub.

Building from Source

DASH is built using CMake.

Build scripts are provided for typical DASH configurations and can serve as starting points for custom builds:

Script file name Description
build.sh Standard release build
build.dev.sh Development / debug build
build.mic.sh Release build for Intel MIC (Xeon Phi)

Prerequisites

  • CMake version 2.8.5 or greater (3.0.0 or greater recommended)
  • C compiler supporting the C99 standard
  • C++ compiler supporting the C++11 standard

Optional third-party libraries directly supported by DASH:

  • PAPI
  • libnuma
  • hwloc
  • BLAS implementation like Intel MKL, ATLAS
  • LIKWID
  • HDF5

Getting the sources

DASH is hosted on Github at https://github.com/dash-project/dash and makes use of git submodules to include third-party software (mainly the GoogleTest framework required for building the tests). The build environment will take care of cloning the submodules upon first invocation. However, there might be cases where recursive cloning is required, e.g., if there is no internet access available during the configuration step.

In that case, please use a recursive clone:

(dash/)$ git clone --recursive https://github.com/dash-project/dash.git

Building DASH from Source

To build the DASH project using CMake with default build settings, run:

(dash/)$ cmake --build .

Or, specify a new directory used for the build:

(dash/)$ mkdir build && cd ./build

For a list of available CMake parameters:

(build/)$ cmake .. -L

Parameters can be set using -D flags. As an example, these parameters will configure the build process to use icc as C compiler instead of the default compiler:

(build/)$ cmake -DCMAKE_C_COMPILER=icc ..

To configure build parameters using ccmake:

(build/)$ ccmake ..

1. Choosing a DASH runtime (DART)

DASH provides the following variants:

  • MPI: The Message Passing Interface, requiring a MPI 3.0 compliant implementation
  • CUDA: nNvidia's Compute Unified Device Architecture (contributor distribution only)
  • SHMEM: Symmetric Hierarchical Memory access (contributor distribution only)

The build process creates the following libraries:

  • libdart-mpi
  • libdart-cuda
  • libdart-shmem

By default, DASH is configured to build all variants of the runtime. You can specify which implementations of DART to build using the cmake option

(build/)$ cmake -DDART_IMPLEMENTATIONS=mpi,shmem ...

Programs using DASH select a runtime implementation by linking against the respective library.

Specifying the MPI implementation for the DART-MPI runtime

The most reliable method to build DART-MPI with a specific MPI installation is to specify the CMake options MPI_<lang>_COMPILER:

(build/) $ cmake -DMPI_C_COMPILER=/path/to/mpi/bin/mpicc \
                 -DMPI_CXX_COMPILER=/path/to/mpi/bin/mpiCC \
                 ...

3. Examples and Unit Tests

Source code of usage examples of DASH are located in dash/examples/. Examples each consist of a single executable and are built by default. Binaries from examples and unit tests are deployed to the build direcory but will not be installed. To disable building of examples, specify the cmake option

(build/)$ cmake -DBUILD_EXAMPLES=OFF ...

To disable building of unit tests, specify the cmake option

(build/)$ cmake -DBUILD_TESTS=OFF ...

The example applications are located in the bin/ folder in the build directory.

4. Installation

The default installation path is $HOME/opt as users on HPC systems typically have install permissions in their home directory only.

To specify a different installation path, use

(build/)$ cmake -DINSTALL_PREFIX=/your/install/path ../

The option -DINSTALL_PREFIX=<DASH install path> can also be given in Step 1.

The installation process copies the 'bin', 'lib', and 'include' directories in the build directory to the specified installation path.

(dash/)$ cmake --build . --target install

Or manually using make:

(build/)$ cmake <build options> ../
(build/)$ make
(build/)$ make install

Running DASH Applications

Building a DASH application

DASH provides wrapper scripts that ensure correct include paths as well as linking for all required DASH, DART, and third-party libraries.

The wrappers are named as dash-<variant>c++, depending on the DASH variant activated at build time, e.g., for the MPI variant the wrapper will be called dash-mpic++ (and its aliases dash-mpiCC and dash-mpicxx).

To build a DASH (MPI) application, replace the call to the MPI C++ compiler with a call to dash-mpicxx (or its alias dash-mpiCC and dash-mpic++):

$ CXX=dash-mpicxx make

The compiler wrapper currently provides two options:

  • --dash:verbose will cause all invocations of the underlying compiler to be printed to the console.
  • --dash:nocppflags disables passing DASH-related precompiler flags to the underlying compiler, including flags that control DASH verbosity (if enabled during DASH build) and assertions.

All other parameters will be passed to the underlying compiler, allowing you to control optimization flags and pass precompiler options.

Note that the compiler wrappers also set the language standard to C++11.

Running a DASH application

With the MPI variant, applications are spawn by MPI:

$ mpirun <MPI args> <app>-mpi

For CUDA and SHMEM, use

$ dartrun-cuda <dartrun-args> <app>-cuda

and respectively

$ dartrun-shmem <dartrun-args> <app>-shmem

Running Tests

Launch the DASH unit test suite using dash-test-shmem or dash-test-mpi:

(dash/shmem/bin/)$ dartrun-shmem <dartrun args> dash-test-shmem <gtest args>

or

(dash/mpi/bin/)$ mpirun <MPI args> dash-test-mpi <gtest args>

For example, you would all unit tests of matrix data structures on 4 units using the MPI runtime with:

(dash/mpi/bin/)$ mpirun -n 4 dash-test-mpi --gtest_filter="MatrixTest*"

or all tests except for the Array test suite:

(dash/mpi/bin/)$ mpirun -n 4 dash-test-mpi --gtest_filter="-ArrayTest*"

Profiling DASH Applications using IPM

Use LD_PRELOAD to run a DASH application built with the DART-MPI backend:

$ LD_PRELOAD=/$IPM_HOME/lib/libipm.so mpirun -n <nproc> <DASH executable>

Available options for IPM are documented in the IPM user guide.

Links

The DASH project homepage: http://www.dash-project.org

The Munich Network Management homepage: http://www.mnm-team.org