Home
Developed under the ExCALIBUR task parallelism cross-cutting research theme, Otter is a tool designed to facilitate data-driven parallelisation of serial code. Otter allows HPC developers to:
- Annotate, trace & visualise loop/task-based serial code as a directed graph;
- Recommend strategies for transforming serial code into effective task-based parallel code;
- Non-invasively trace & visualise loop/task-based OpenMP 5.x programs.
This wiki introduces the Otter toolset and explains how to use features above.
The Otter toolset includes:
- Otter-Task-Graph: an API and runtime library for annotating & tracing the task-graph of a (possibly parallel) target application.
- Otter-OMPT: an OMPT tool for non-invasive tracing of the loop/task-based structure of OpenMP 5.x programs.
- PyOtter: The visualisation & reporting tool for use with Otter trace data.
This section explains how to install Otter's pre-requisites, build the Otter-Serial & Otter-OMPT runtime libraries and obtain the PyOtter visualisation & reporting tool.
Otter has several pre-requisites which must be installed before Otter itself can be build & used:
- OTF2 v2.3 (required by all Otter components)
- The
igraphPython package (required by PyOtter)
- Graphviz (required by PyOtter)
A recent version of CMake is also required to build Otter itself. Otter-OMPT requires a compiler supporting OpenMP 5.0 and the OMPT interface in particular (LLVM-based compilers such as Intel's or Clang are recommended).
OTF2 v2.3 is available here. Full installation instructions for OTF2 are included with the OTF2 source. The default installation location of /opt/otf2 can be overridden with the --prefix option to ./configure. To download, build and install OTF2, run:
wget https://zenodo.org/record/4682684/files/otf2-2.3.tar.gz
tar -xzvf otf2-2.3.tar.gz && cd otf2-2.3
./configure
make
make install
The OTF2 Python package is available from pip:
pip install otf2
To check that the OTF2 package is available to Python:
$ python3 -c "import otf2;print(otf2.__file__)"
/opt/otf2/lib/python3.8/site-packages/otf2/__init__.py
The following command will install igraph for the user only:
pip install --user igraph
PyOtter uses graphviz software, in particular the dot command line tool, to create graph visualisations. Full installation instructions are available here but should be as simple as:
sudo apt install graphviz
To build Otter, first obtain the latest version of the Otter repository:
git clone -b dev https://github.com/Otter-Taskification/otter.git
Otter uses CMake to perform an out-of-source build:
mkdir otter/build && cd otter/build
cmake ../
By default, Otter checks for an OTF2 installation under /opt/otf2. If you installed OTF2 somewhere else, use -DOTF2_INSTALL_DIR=<install-path> to tell Otter where to look.
If you want to build the OMPT plugin, you should specify with -DCMAKE_C_COMPILER= a C compiler which supports OpenMP 5.0 and the OMPT interface (see pre-requisites).
Additional configuration options are documented here.
To build and install Otter:
cmake --build .
cmake --install . [--prefix /path/to/install]
The installation path can be overridden with the --prefix installation option.
The following artefacts will be installed:
-
include/otter/otter-task-graph-user.h(the main API for annotating application code) -
include/otter/otter-task-graph-stub.h(for developers of software which may use Otter but who don't want to require it as a dependency for their users) -
include/otter/otter-task-graph.h(required byotter-task-graph-user.h, not intended for direct use)
lib/libotter-task-graph.so-
lib/libotter-ompt.so(if requested)
-
etc/modulefiles/otter/otter(to providemodule load otteron machines which make use of modulefiles)
This section contains generic advice to help you set up your environment to use Otter.
-
It is strognly recommended that you install Otter before using it to annotate your application. This will ensure the necessary Otter headers and libraries are available when compiling your target application.
-
If you do a basic
cmake --install .from your build directory, this should install Otter to a standard location on your system, and the Otter headers and libraries should be visible to your compiler by default. -
If you do
cmake --install . --prefix=<installdir>then you will need to tell your compiler where you have installed Otter when using it in your target application. Use-I<installdir>/includeand-L<installdir>/libarguments to add the relevant include and library paths. This is taken care of for you if you use the modulefile provided with Otter (see below). -
To link against Otter, use
-lotter-task-graph.
Otter provides a modulefile at <installdir>/etc/modulefiles/otter/otter so that Otter can be used with module load otter. To use this modulefile, add it to your MODULEPATH e.g. with module use --append <installdir>/etc/modulefiles/otter The module file can be set to load Otter from a specific installation prefix using the OTTER environment variable i.e. export OTTER=<installdir> && module load otter will add the Otter installation under <installdir> to your environment. This should mean you don't need to pass -I and -L flags to your compiler (although you still need to pass -lotter-task-graph).
The latest version of PyOtter is obtained by running:
git clone -b dev https://github.com/Otter-Taskification/pyotter.git
pip install ./pyotter/
For Otter-Task-Graph, Otter-Serial or Otter-OMPT, please post here.
For PyOtter, please post here.
Otter is released under the BSD 3-clause license. See LICENCE for details.
Copyright (c) 2021, Adam Tuft All rights reserved.
Otter's development started as the subject of a final project and dissertation for the the Scientific Computing and Data Analysis MSc (MISCADA) at Durham University, UK. The current research is supported by EPSRC's Excalibur programme through its cross-cutting project EX20-9 Exposing Parallelism: Task Parallelism (Grant ESA 10 CDEL).