Developer's Manual

Directory Structure

The repository contains a number of sub-directories:

• common: Header-only library, containing a number of headers common to the compiler and the runtime component.
• compiler: Contains the compiler driver logic, implementing the compiler portion of the Python interface. The entry point is Driver.cpp, subclasses of MLIRToolchain.[h/cpp] (currently CPUToolchain and CUDAGPUToolchain) put together the actual compilation pipeline for the different targets.
• docs: Contains layout information for Doxygen and usually does not require any editing.
• execute: A minimal executable that allows to invoke the compiler and runtime directly from the command-line instead of going through the Python interface. This is mainly handy for development and debugging and not intended for end-users.
• mlir: Extensions of the MLIR framework, namely additional dialects (currently the SPN-specific HiSPN and LoSPN) as well as transformations and lowerings for these dialects.
• python-interface: The Pybind11-based Python interface of the compiler. Combines a minimal C++ wrapper around the compiler and runtime with some Python code for a more convenient interface. The CMakeLists.txt and setup.py.in (will be configured by CMake) contain the logic to package the distributable wheel.
• runtime: The runtime component to launch inference on compiled kernels. Loads and executes the kernels/shared objects generated by the compilers.
• xspn: Contains the serialization library for SPFlow-based SPNs implemented completely in Python, using Cap'n'Proto.

CMake options

SPNC defines and uses a number of CMake options, which might be relevant for developers:

Option	Explanation
CMAKE_PREFIX_PATH	Used to specify the location of SPNC's dependencies, see the Installation manual for details.
LLVM_ENABLE_LLD	Use the faster lld for linking. Should match the configuration of LLVM.
LLVM_ENABLE_ASSERTIONS	Enable assertions in LLVM headers and code. Should match the configuration of LLVM.
BUILD_SHARED_LIBS	Set to OFF to build a statically linked library, use ON for faster shared linking.
CMAKE_POSITION_INDEPENDENT_CODE	Should be set to ON when compiling with BUILD_SHARED_LIBS=OFF for compatibility with Pybind11.
SPNC_BUILD_DOC	Set to ON to generate Doxygen documentation as part of the compilation process.
CPU_PARALLEL_RUNTIME	Use OpenMP for parallelized processing of batches in the runtime.
CUDA_GPU_SUPPORT	Enable support for CUDA GPUs. Requires a working installation of the CUDA toolkit and driver.
CUDA_TOOLKIT_ROOT_DIR	Use this option to specify the location of the CUDA toolkit if it is not automatically detected by CMake (usually the case, if nvcc is on the PATH).
CUDA_RUNTIME_LIBRARY	Use this option to specify the location of the CUDA runtime library (libcuda.so) if it is not automatically found by CMake.
SPNC_ENABLE_CLANG_TIDY	Set to ON to run clang-tidy as part of every build.
SPNC_RUN_CLANG_TIDY_LOCATION	Use this option to specify the location of the clang-tidy wrapper script (run-clang-tidy(.py)) if it is not automatically found by CMake.
CMAKE_EXPORT_COMPILE_COMMANDS	Set to ON to either use clang-tidy as part of the build process or to generate the wrapper script.
SPNC_SLP_DEBUG	Set to ON to enable additional debug output for the SLP vectorizer. Defaults to OFF