Resource-based, Declarative task-Graphs for Parallel, Event-driven Scheduling
RedGrapes is a C++14 framework for declaratively creating and scheduling task-graphs, based on a high-level resource description.
Modern compute nodes concurrently perform computational tasks over various memory resource pools, cores, and accelerator devices. In order to achieve high scalability in such a compute node, communication and computation tasks need to be overlapped extensively.
Up until now, software developers that took up to this challenge had to juggle data and in-node execution dependencies manually, which is a tedious and error-prone process. Real-world workloads that depend on states at runtime and asynchronous communication models complicate the program flow even further.
For this reason, one should decouple aforementioned computational tasks from their execution model altogether. A typical approach involves task-graphs, which are directed acyclic graphs (DAGs), whose vertices are some sort of computation (or communication) and the edges denote the execution precedence order. The execution precedence arises from the order in which those tasks were declared by the programmer but also have to take into account the data dependencies between the tasks.
Consequently, RedGrapes provides you with a light-weight, application-level, task-based C++ programming framework. Herein, a task-graph is generated declaratively from access to resources and order of your code, just as in serial programming.
The program shall be divided into tasks. A task is can be a sub-step in a computational solver, the exchange of data between two memory resource pools, or anything else. Tasks are the smallest unit the RedGrapes scheduler operates with. Data dependencies are described via resources, which are accessed and potentially manipulated by tasks.
Each task has an annotation how the resources are accessed. Therein allowed access modes depend on the type of the resource. A simple example would be read/write, but also more complex operations are possible, e.g. on arrays or just a sub view. A resource can be associated with a specific access mode forming a resource access. These instances of a resource access can then be pairwise tested wheter they are conflicting and thereby creating a data-dependency (e.g. two writes to the same resource). So each task carries a list of these resource-accesses in its so-called task-properties. If two tasks have conflicting resource-accesses, the first created task is executed first. This is exactly the behaviour that one would also achieve when programming serially, without hints given via resources.
When tasks are created, their resource-access list is compared against the previous enqueued tasks and corresponding dependencies are created in the task-graph. The resulting task-graph is read by a scheduling algorithm that executes individual tasks, e.g. across parallel threads.
See examples for examples covering more features.
#include <cassert>
#include <redGrapes/manager.hpp>
#include <redGrapes/resource/ioresource.hpp>
#include <redGrapes/property/resource.hpp>
#include <redGrapes/property/label.hpp>
namespace rg = redGrapes;
using TaskProperties =
rg::TaskProperties<
rg::ResourceProperty,
rg::LabelProperty
>;
int main()
{
rg::Manager< TaskProperties, rg::ResourceEnqueuePolicy > mgr;
rg::IOResource< int > a;
mgr.emplace_task(
[]( auto a ){ *a = 123; },
TaskProperties::Builder().label("Task 1"),
a.write()
);
mgr.emplace_task(
[]( auto a ){ assert( *a == 123 ); },
TaskProperties::Builder().label("Task 2"),
a.read()
);
return 0;
}RedGrapes is documented using in-code doxygen comments and reStructured-text files (in docs/source), build with Sphinx.
This Project is free software, licensed under the Mozilla MPL 2.0 license.
RedGrapes is developed by members of the Computational Radiation Physics Group at HZDR. Its conceptual design is based on a whitepaper by A. Huebl, R. Widera, and A. Matthes (2017).
- Michael Sippel: library design & implementation
- Dr. Sergei Bastrakov: supervision
- Dr. Axel Huebl: whitepaper, supervision
- René Widera: whitepaper, supervision
- Alexander Matthes: whitepaper
RedGrapes requires a compiler supporting the C++14 standard. RedGrapes further depends on the following libraries: