Skip to content

Latest commit

 

History

History
187 lines (144 loc) · 8.83 KB

execution_model.rst

File metadata and controls

187 lines (144 loc) · 8.83 KB
Raw

Execution Model

Mint provides a mesh-aware sections/execution_model, based on the RAJA programming model abstraction layer. The execution model supports on-node fine-grain parallelism for mesh traversals. Thereby, enable the implementation of computational kernels that are born parallel and portable across different processor architectures.

Note

To utilize NVIDIA GPUs, using the RAJA CUDA backend, Axom needs to be compiled with CUDA support and linked to a CUDA-enabled RAJA library. Consult the Axom Quick Start Guide for more information.

The execution model consists of a set of templated functions that accept two arguments:

  1. A pointer to a mesh object corresponding to one of the supported MeshTypes.
  2. The kernel that defines the operations on the supplied mesh, which is usually specified by a C++11 Lambda Expression.

Note

Instead of a C++11 Lambda Expression a C++ functor may also be used to encapsulate a kernel. However, in our experience, using C++11 functors, usually requires more boiler plate code, which reduces readability and may potentially have a negative impact on performance.

The sections/execution_model provides NodeTraversalFunctions, CellTraversalFunctions and FaceTraversalFunctions to iterate and operate on the constituent Nodes, Cells and Faces of the mesh respectively. The general form of these functions is shown in figs/execModel.

General form of the constituent templated functions of the sections/execution_model

As shown in figs/execModel, the key elements of the functions that comprise the sections/execution_model are:

  • The Iteration Space: Indicated by the function suffix, used to specify the mesh entities to traverse and operate upon, e.g. the Nodes, Cells or Faces of the mesh.
  • The executionPolicy: Specified as as the first, required, template argument to the constituent functions of the sections/execution_model. The executionPolicy specifies where and how the kernel is executed.
  • The executionSignature: Specified by a second, optional, template argument to the constituent functions of the sections/execution_model. The executionSignature specifies the type of arguments supplied to a given kernel.
  • The Kernel: Supplied as an argument to the constituent functions of the sections/execution_model. It defines the body of operations performed on the supplied mesh.

See the sections/tutorial for code snippets that illustrate how to use the NodeTraversalFunctions, CellTraversalFunctions and FaceTraversalFunctions of the sections/execution_model.

Execution Policy

The ExecutionPolicy is specifed as the first template argument and is required by all of the constituent functions of the sections/execution_model. Axom defines a set of high-level execution spaces, summarized in the table below.

Execution Policy Requirements Description
SEQ_EXEC

None.

 Sequential execution on the CPU.
OMP_EXEC

RAJA + OpenMP

 Parallel execution on the CPU using OpenMP.
CUDA_EXEC< BLOCKSIZE >

RAJA + CUDA + Umpire (memory management)

 Parallel execution on CUDA-enabled GPUs.
CUDA_EXEC< BLOCKSIZE, ASYNC >

RAJA + CUDA + Umpire (memory management)

 Asynchronous parallel execution on CUDA-enabled GPUs.

Internally, the implementation uses the axom::execution_space traits object to map each execution space to corresponding RAJA execution policies and bind the default memory space for a given execution space. For example, the default memory space for the axom::CUDA_EXEC execution space is unified memory, which can be accessed from both the host (CPU ) and device (GPU).

Execution Signature

The executionSignature is specified as the second, optional template argument to the constituent functions of the sections/execution_model. The executionSignature indicates the list of arguments that are supplied to the user-specified kernel.

Note

If not specified, the default executionSignature is set to mint::xargs::index, which indicates that the supplied kernel takes a single argument that corresponds to the index of the corresponding iteration space, i.e, the loop index.

The list of currently available executionSignature options is based on commonly employed access patterns found in various mesh processing and numerical kernels. However, the sections/execution_model is designed such that it can be extended to accomodate additional access patterns.

  • mint::xargs::index
    • Default ExecutionSignature to all functions of the sections/execution_model
    • Indicates that the supplied kernel takes a single argument that corresponds to the index of the iteration space, i.e. the loop index.
  • mint::xargs::ij/mint::xargs::ijk
    • Applicable only with a StructuredMesh.
    • Used with NodeTraversalFunctions (mint::for_all_nodes()) and CellTraversalFunctions (mint::for_all_cells()).
    • Indicates that the supplied kernel takes the corresonding (i, j) or (i, j, k) indices, in 2D or 3D respectively, as additional arguments.
  • mint::xargs::x/mint::xargs::xy/mint::xargs::xyz
    • Used with NodeTraversalFunctions (mint::for_all_nodes()).
    • Indicates that the supplied kernel takes the corresponding nodal coordinates, x in 1D, (x, y) in 2D and (x, y, z) in 3D, in addition to the corresponding node index, nodeIdx.
  • mint::xargs::nodeids
    • Used with CellTraversalFunctions (mint::for_all_cells()) and FaceTraversalFunctions (mint::for_all_faces()).
    • Indicates that the specified kernel is supplied the constituent node IDs as an array argument to the kernel.
  • mint::xargs::coords
    • Used with CellTraversalFunctions (mint::for_all_cells()). and FaceTraversalFunctions (mint::for_all_faces())
    • Indicates that the specified kernel is supplied the constituent node IDs and corresponding coordinates as arguments to the kernel.
  • mint::xargs::faceids
    • Used with the CellTraversalFunctions (mint::for_all_cells()).
    • Indicates that the specified kernel is supplied an array consisting of the constituent cell face IDs as an additional argument.
  • mint::xargs::cellids
    • Used with the FaceTraversalFunctions (mint::for_all_faces()).
    • Indicates that the specified kernel is supplied the ID of the two abutting Cells to the given. By conventions, tor external boundary Faces, that are bound to a single cell, the second cell is set to  − 1.

Warning

Calling a traversal function with an unsupported executionSignature will result in a compile time error.