FloatGuard: Efficient Whole-Program Detection of Floating-Point Exceptions in AMD GPUs

Overview

FloatGuard is a tool that captures floating-point exceptions in AMD HIP kernels. FloatGuard leverages AMD GPU hardware registers to detect floating-point exceptions, overcoming the limitations of AMD’s built-in trapping mechanisms through a novel algorithm that combines assembly- and source-level instrumentation with debugger-guided execution.

The details of the tool and how it evaluates can be found in [1]. The workflow of FloatGuard is shown below:

FloatGuard consists of two modules implemented in Python: (1) a compiler wrapper which processes HIP program source files, compiles them into assembly, and injects code during link time that enables floating-point exception capture—it also accepts a list of already-captured floating-point exception code locations to prevent them from being triggered in subsequent iterations; and (2) a testing framework, which runs the instrumented HIP program with a custom ROCgdb debugger that detects and records floating- point exceptions as they occur. Both modules are run iteratively until all floating-point exceptions have been reported.

Download and Run FloatGuard

Prerequisites

1. Linux system. We have tested on Ubuntu 22.04 and TOSS 4.
2. Python (>= 3.9) installed.
3. AMD GPU with ROCm installed. Tested on ROCm 6.1.2 with AMD Clang 17.0.0. Follow https://rocm.docs.amd.com/projects/install-on-linux/en/latest/ for more information on how to install AMD ROCm. Specifically, make sure both rocm and rocm-llvm-dev packages are installed.

Download FloatGuard

Clone this GitHub repository with the following command.

git clone https://github.com/LLNL/FloatGuard [FloatGuard Directory]

Running Samples for make and CMake projects

We provide two sample projects, samples/div0 and samples/div0_cmake, for developers to try out and learn what is required for your own HIP code to work with FloatGuard. You can refer to samples/div0/Makefile and samples/div0_cmake/CMakeLists.txt for examples of the steps described in previous sections. To run these samples, simply enter the respective directories and run the following command:

[FloatGuard Directory]/gdb_script/time_measure.py

Log files containing details on the floating-point exceptions found are outputted as [code project directory name]_output.txt file in the FloatGuard code repo directory. Also result.csv updates the running time statistics and the number of exceptions found.

How to Use FloatGuard to capture floating-point exceptions

The following are instructions for setting up the AMD HIP code project of your own to use FloatGuard.

1. Replace the compiler in your code project with our wrapper script. For example, if your code project uses the HIPCC macro in a Makefile to indicate compiler command, you should replace it with this:

HIPCC = ${HOME}/FloatGuard/gdb_script/hipcc_wrapper.sh

2. If your code project has multiple source files, but you use a single compiler command to compile and link the sources, please separate the compile and link commands into two. For example:

${HIPCC} ${HIPFLAGS} -c main.cpp -o main.o
${HIPCC} ${HIPFLAGS} -c other.cpp -o other.o
${HIPCC} ${LINKFLAGS} main.o other.o -o main

3. Create a setup.ini file in the root directory of your code project. This file must contain three key-value pairs, as shown below:

[DEFAULT]
compile = # the command line to compile the executable
run =     # the command line to run the executable
clean =   # the command line to clean the executable

An example setup.ini can be found in samples/div0 directory:

[DEFAULT]
compile = make
run = ./div0
clean = make clean_base

4. Enter the root directory of your project, then call the time_measure.py script to run the FloatGuard tool.

[FloatGuard Directory]/gdb_script/time_measure.py

Log files containing details on the floating-point exceptions found are outputted as [code project directory name]_output.txt file in the FloatGuard code repo directory. Also result.csv updates the running time statistics and the number of exceptions found.

5. Run time_measure.py again but with a cleanup parameter -c to call the clean up command line set in setup.ini, and clean up temporary files generated by FloatGuard.

[FloatGuard Directory]/gdb_script/time_measure.py -c

References

[1] Dolores Miao, Ignacio Laguna, Cindy Rubio-González, FloatGuard: Efficient Whole-Program Detection of Floating-Point Exceptions in AMD GPUs, To Appear In Proceedings of the 34th International Symposium on High-Performance Parallel and Distributed Computing (HPDC '25).

License

FloatGuard is distributed under the terms of the MIT License.

See LICENSE and NOTICE for details.

LLNL-CODE-2007315