KaMPIng: Karlsruhe MPI next generation 🚀

This is KaMPIng [kampɪŋ], a (near) zero-overhead MPI wrapper for modern C++.

It covers the whole range of abstraction levels from low-level MPI calls to convenient STL-style bindings, where most parameters are inferred from a small subset of the full parameter set. This allows for both rapid prototyping and fine-tuning of distributed code with predictable runtime behavior and memory management.

Using template-metaprogramming, only code paths required for computing parameters not provided by the user are generated at compile time, which results in (near) zero-overhead bindings.

🏃 Quick Start: We provide a wide range of usage and simple applications examples (start with allgatherv). Or checkout the documentation for a description of KaMPIng's core concepts and a full reference.

KaMPIng is developed at the Algorithm Engineering Group at Karlsruhe Institute of Technology.

If you use KaMPIng in the context of an academic publication, we kindly ask you to cite our technical report:

@misc{kamping2024,
  title={KaMPIng: Flexible and (Near) Zero-overhead C++ Bindings for MPI},
  author={Demian Hespe and Lukas Hübner and Florian Kurpicz and Peter Sanders and Matthias Schimek and Daniel Seemaier and Christoph Stelz and Tim Niklas Uhl},
  year={2024},
  eprint={2404.05610},
  archivePrefix={arXiv},
  primaryClass={cs.DC}
}

Features ✨

Named Parameters 💬

Using plain MPI, operations like MPI_Allgatherv often lead to verbose and error-prone boilerplate code:

std::vector<T> v = ...; // Fill with data
int size;
MPI_Comm_size(comm, &size);
int n = static_cast<int>(v.size());
std::vector<int> rc(size), rd(size);
MPI_Allgather(&n, 1, MPI_INT, rc.data(), 1, MPI_INT, comm);
std::exclusive_scan(rc.begin(), rc.end(), rd.begin(), 0);
int n_glob = rc.back() + rd.back();
std::vector<T> v_glob(v_global_size);
MPI_Allgatherv(v.data(), v_size, MPI_TYPE, v_glob.data(), rc.data(), rd.data(), MPI_TYPE, comm);

In contrast, KaMPIng introduces a streamlined syntax inspired by Python's named parameters. For example, the allgatherv operation becomes more intuitive and concise:

std::vector<T> v = ...; // Fill with data
std::vector<T> v_glob = comm.allgatherv(send_buf(v));

Empowered by named parameters, KaMPIng allows users to name and pass parameters in arbitrary order, computing default values only for the missing ones. This not only improves readability but also streamlines the code, providing a user-friendly and efficient way of writing MPI applications.

Controlling memory allocation 💾

KaMPIng's resize policies allow for fine-grained control over when allocation happens:

resize policy
kamping::resize_to_fit	resize the container to exactly accommodate the data
kamping::no_resize	assume that the container has enough memory available to store the data
kamping::grow_only	only resize the container if it not large enough

// easy to use with sane defaults
std::vector<int> v = comm.recv<int>(source(kamping::rank::any));

// flexible memory control
std::vector<int> v_out;
v_out.resize(enough_memory_to_fit);
// already_known_counts are the recv_counts that may have been computed already earlier and thus do not need to be computed again
comm.recv<int>(recv_buf<kamping::no_resize>(v_out), recv_count(i_know_already_know_that), source(kamping::rank::any));

STL support 📚

• KaMPIng works with everything that is a std::contiguous_range, everywhere.
• Builtin C++ types are automatically mapped to their corresponding MPI types.
• All internally used containers can be altered via template parameters.

Expandability 🧩

• Don't like the performance of your MPI implementation's reduce algorithm? Just override it using our plugin architecture.
• Add additional functionality to communicator objects, without altering any application code.
• Easy to integrate with existing MPI code.
• Flexible core library for a new toolbox 🧰 of distributed datastructures and algorithms

And much more ... ↗️

• Safety guarantees for non-blocking communication and easy handling of multiple requests via request pools
• Compile time and runtime error checking (which can be completely deactivated).
• Collective hierarchical timers to speed up your evaluation workflow.
• ...

Dive into the documentation or tests to find out more ...

(Near) zero overhead - for development and performance 📈

Using template-metaprogramming, KaMPIng only generates the code paths required for computing parameters not provided by the user. The following shows a complete implementation of distributed sample sort with KaMPIng.

void sort(MPI_Comm comm_, std::vector<T>& data, size_t seed) {
    Communicator<> comm(comm_);
    size_t const   oversampling_ratio = 16 * static_cast<size_t>(std::log2(comm.size())) + 1;
    std::vector<T> local_samples(oversampling_ratio);
    std::sample(data.begin(), data.end(), local_samples.begin(), oversampling_ratio, std::mt19937{seed});
    auto global_samples = comm.allgather(send_buf(local_samples)).extract_recv_buffer();
    std::sort(global_samples.begin(), global_samples.end());
    for (size_t i = 0; i < comm.size() - 1; i++) {
        global_samples[i] = global_samples[oversampling_ratio * (i + 1)];
    }
    global_samples.resize(num_splitters);
    std::vector<std::vector<T>> buckets(global_samples.size() + 1);
    for (auto& element: data) {
        auto const bound = std::upper_bound(global_samples.begin(), global_samples.end(), element);
        buckets[static_cast<size_t>(bound - global_samples.begin())].push_back(element);
    }
    data.clear();
    std::vector<int> scounts;
    for (auto& bucket: buckets) {
        data.insert(data.end(), bucket.begin(), bucket.end());
        scounts.push_back(static_cast<int>(bucket.size()));
    }
    data = comm.alltoallv(send_buf(data), send_counts(scounts)).extract_recv_buffer();
    std::sort(data.begin(), data.end());
}

It is a lot more concise than the (verbose) plain MPI implementation, but also introduces no additional overhead to achieve this, as can be seen the following experiment. There we compare the sorting implementation in KaMPIng to other MPI bindings.

Platform 🖥️

• intensively tested with GCC and Clang and OpenMPI
• requires a C++17 ready compiler
• easy integration into other projects using modern CMake

Other MPI bindings

	MPI	Boost.MPI	RWTH MPI	MPL
STL support	❌	✔️1	✔️2	✔️1	✅
computation of defaults via additional communication	❌	❌	✅	❌	✅
custom reduce operations via lambdas	❌	✅	❌	✔️3	✅
containers can be resized automatically	❌	✔️4	✔️2	❌	✅
error handling	✅	✅	✅	❌	✅
actively maintained	✅	❌	✔️	✅	✅

LICENSE

KaMPIng is released under the GNU Lesser General Public License. See COPYING and COPYING.LESSER for details

Footnotes

1. only std::vector ↩ ↩²

2. only for send and receive buffers ↩ ↩²

3. not mapped to builtin operations ↩

4. partial ↩