This document explains how you can build this library from source, including some examples of build environment. In this repository there are two builds:
- A
power-grid-model
pip Python package with C++ extension as the calculation core. - A CMake C++ project to build native C++ unit tests and/or performance benchmark.
- In the future, the following two libraries might be provided:
- A C++ header-only library of the calculation core
- A C/C++ dynamic shared library (
.dll
or.so
) with pure C ABI interface.
To build the library from source, you need to first prepare the compiler toolchains and the build dependencies. In this section a list of general requirements are given. After this section there are examples of setup in Linux (Ubuntu 20.04) , Windows 10, and macOS (Big Sur).
This library is written and tested on x86_64
and aarch64
architecture. Building the library in x86_32
might be working, but is
not tested. The MKL library is only
available for x86_64
. So only Eigen sparse solver is available for aarch64
.
The source code is written with the mindset of ISO standard C++ only, i.e. avoid compiler-extension or platform-specific features as much as possible. In this way the effort to port the library to other platform/architecture might be minimum.
You need a C++ compiler with C++17 support. Below is a list of tested compilers:
Linux
- gcc >= 9.3
- clang >= 9.0
You can define the environment variable CXX
to for example clang++
to specify the C++ compiler.
Windows
- MSVC >= 14.2 (Visual Studio 2019, IDE or build tools)
macOS
- clang >= 12.0
This repository uses CMake and Ninja as C++ build system.
The table below shows the C++ build dependencies
Library name | Requirements to build Python package | Requirements to build CMake project | Remark | License |
---|---|---|---|---|
boost | Define environment variable BOOST_INCLUDE to the include folder of boost * |
CMake needs to be able find boost |
header-only | Boost Software License - Version 1.0 |
eigen3 | Define environment variable EIGEN_INCLUDE to the include folder of eigen3 * |
CMake needs to be able find eigen3 |
header-only | Mozilla Public License, version 2.0 |
Catch2 | None | CMake needs to be able find Catch2 |
header-only | Boost Software License 1.0 |
nlohmann-json | None | CMake needs to be able find nlohmann_json |
header-only | MIT |
MKL | Add path to MKL runtime library (libmkl_rt.so or mkl_rt.dll ) into PATH (Windows) or LD_LIBRARY_PATH (Linux) |
|
Optional for use of MKL PARDISO sparse solver. | Intel Simplified Software License (proprietary license) |
* The environment variables should point to the root include folder of the library, not a subfolder. For example in the
path BOOST_INCLUDE
there should be a folder called boost
which has all the boost
header files.
The table below shows the Python dependencies
Library name | Remark | License |
---|---|---|
cython | build dependency | Apache License 2.0 |
numpy | build/runtime dependency | BSD-3 |
wheel | build dependency | MIT |
pytest | Development dependency | MIT |
pytest-cov | Development dependency | MIT |
You can install the development dependencies using pip
.
pip install -r dev-requirements.txt
Once you have prepared the build dependencies, you can install the library from source in develop mode. Go to the root folder of the repository.
pip install -e .
If you have the development dependencies installed in your Python environment, you can run the tests.
pytest
There is a root cmake file in the root folder of the repo CMakeLists.txt
. It specifies
dependencies and the build options for the project. The core algorithm is implemented in the header-only
library power_grid_model
. There are two sub-project defined in the root cmake file:
tests/cpp_unit_tests/CMakeLists.txt
: the unit test project usingCatch2
framework.tests/benchmark_cpp/CMakeLists.txt
: the C++ benchmark project for performance measure.
In principle, you can use any C++ IDE with cmake and ninja support to develop the C++ project. When you
use cmake build
for the root cmake file, the following additional options are available besides the standard cmake
option. If no option is defined, the cmake project will build the unit tests with Eigen sparse solver.
Option | Description |
---|---|
POWER_GRID_MODEL_SPARSE_SOLVER |
Specify which sparse solver to use during the build. There are three options: EIGEN : use built-in SparseLU solver of eigen3 . MKL : use MKL PARDISO solver and link MKL library at link-time. MKL_RUNTIME : build the project using both Eigen sparse solver and MKL PARDISO solver. The MKL library mkl_rt is loaded at runtime. If the MKL library cannot be found, it falls back to the built-in Eigen sparse solver. This is the build option for Python package. |
POWER_GRID_MODEL_BUILD_BENCHMARK |
When set to 1 , build the both sub-projects: unit test and benchmarks. Otherwise only build the unit test. |
In this section an example is given for setup in Ubuntu 20.04. You can use this example in Windows Subsystem for Linux ( WSL), or in a physical/virtual machine.
Append the following lines into the file ${HOME}/.bashrc
.
export CXX=clang++-11 # or clang++-10, or g++-9, or g++-10
export CC=clang-11 # or clang-10, or gcc-9, or gcc-10
export VCPKG_FEATURE_FLAGS=-binarycaching
export VCPKG_ROOT=${HOME}/vcpkg
export EIGEN_INCLUDE=${VCPKG_ROOT}/installed/x64-linux/include/eigen3
export BOOST_INCLUDE=${VCPKG_ROOT}/installed/x64-linux/include
export MKL_THREADING_LAYER=SEQUENTIAL
export MKL_INTERFACE_LAYER=LP64
export LD_LIBRARY_PATH=${HOME}/.local/lib:${LD_LIBRARY_PATH}
export MKL_LIB=${HOME}/.local/lib
export MKL_INCLUDE=${HOME}/.local/include
Install the following packages from Ubuntu. In the .bashrc
you can set environment variable CXX
to select a C++
compiler.
sudo apt update && sudo apt -y upgrade
sudo apt install -y curl zip unzip tar git build-essential lcov gcc g++ gcc-10 g++-10 clang clang-11 make cmake gdb ninja-build pkg-config python3.9 python3.9-dev python3.9-venv python3-pip
The recommended way to get C++ package is via vcpkg.
cd ${HOME}
git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg install eigen3 catch2 boost nlohmann-json
The installation of boost
will take a long time, be patient
The easiest way to install mkl
is through pip
. Since you can have multiple Python environment which may depend on a
single mkl
library. It is better to install mkl
without virtual environment. Make sure your current shell is NOT
in a virtual environment (no parentheses before the prompt). Without sudo
, it will install the mkl
into the
folder ${HOME}/.local
.
python3.9 -m pip install mkl mkl-include mkl-devel
It is recommended to create a virtual environment. Clone repository, create and activate virtual environment, and install the build dependency. go to a root folder you prefer to save the repositories.
git clone https://github.com/alliander-opensource/power-grid-model.git # or git@github.com:alliander-opensource/power-grid-model.git
cd power-grid-model
python3.9 -m venv .venv
source ./.venv/bin/activate
pip install -r dev-requirements.txt
Install from source in develop mode, and run pytest
.
pip install -e .
pytest
There is a convenient shell script to build the cmake project:
build.sh
. You can study the file and write your own build script. Six configurations are pre-defined
for two input arguments, which will be passed into cmake
. It includes debug or release build, as well as the option to
use MKL at link-time, or at runtime, or not at all.
- Option 1
- Debug
- Release
- Option 2
- EIGEN
- MKL
- MKL_RUNTIME
As an example, go to the root folder of repo. Use the following command to build the project with MKL at link-time and benchmark:
./build.sh Release MKL
One can run the unit tests and benchmark by:
./cpp_build_Relase_MKL/tests/cpp_unit_tests/power_grid_model_unit_tests
./cpp_build_Relase_MKL/tests/benchmark_cpp/power_grid_model_benchmark_cpp
If you want to generate a C++ test coverage report you should do a few extra steps.
- Install the following extra packages from Ubuntu.
sudo apt install lcov gcovr
- Change your compiler in
${HOME}/.bashrc
export CXX=g++-10 # or clang++-10, or g++-9, or g++-10
export CC=gcc-10 # or clang-10, or gcc-9, or gcc-10
Define the following environment variables in user wide.
Name | Value |
---|---|
PreferredToolArchitecture | x64 |
VCPKG_ROOT | <ROOT_FOLDER_OF_VCPKG> |
VCPKG_DEFAULT_TRIPLET | x64-windows |
VCPKG_FEATURE_FLAGS | -binarycaching |
EIGEN_INCLUDE | <ROOT_FOLDER_OF_VCPKG>\installed\x64-windows\include\eigen3 |
BOOST_INCLUDE | <ROOT_FOLDER_OF_VCPKG>\installed\x64-windows\include |
MKL_LIB | <ROOT_FOLDER_OF_MINICONDA>\envs\mkl-base\Library\lib |
MKL_INCLUDE | <ROOT_FOLDER_OF_MINICONDA>\envs\mkl-base\Library\include |
MKL_THREADING_LAYER | SEQUENTIAL |
MKL_INTERFACE_LAYER | LP64 |
Further, append <ROOT_FOLDER_OF_MINICONDA>\envs\mkl-base\Library\bin
to environment variable PATH
in user wide.
You need to install the MSVC compiler. You can either install the whole Visual Studio IDE or just the build tools.
- Visual Studio Build Tools (free)
- Select C++ build tools
- Full Visual Studio (All three versions are suitable. Check the license!)
- Select Desktop Development with C++
Other toolchains:
The recommended way to get C++ package is via vcpkg. Open a PowerShell console, go to the parent folder of your desired <ROOT_FOLDER_OF_VCPKG>.
git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.bat
./vcpkg install eigen3 catch2 boost nlohmann-json
You can install mkl
from conda
. Since you might want to use mkl
in different projects, it is recommended to (only)
install mkl
in a separate conda environment. In this example the environment is called mkl-base
. Open a Miniconda
PowerShell Prompt.
conda create -n mkl-base python=3.9
conda activate mkl-base
conda install mkl mkl-include mkl-devel
It is recommended to create a conda
environment. Clone repository, create and activate conda
environment, and
install the build dependency. go to a root folder you prefer to save the repositories, open a Git Bash Console.
git clone https://github.com/alliander-opensource/power-grid-model.git
Then open a Miniconda PowerShell Prompt, go to the repository folder.
conda create -n power-grid-env python=3.9
conda activate power-grid-env
pip install -r dev-requirements.txt
Install from source in develop mode, and run pytest
.
pip install -e .
pytest
If you have installed Visual Studio 2019/2022 (not the build tools), you can open the repo folder as a cmake project.
The IDE should be able to automatically detect the Visual Studio cmake configuration file
CMakeSettings.json
. Six configurations are pre-defined. It includes debug or release build,
as well as the option to use MKL at link-time, or at runtime, or not at all.
x64-Debug
x64-Debug-MKL
x64-Debug-MKL-at-runtime
x64-Release
x64-Release-MKL
x64-Release-MKL-at-runtime
In this section an example is given for setup in macOS Big Sur and Python 3.10. It is likely that other versions (3.8+) of Python are also supported, though you'll need to change some paths accordingly.
Append the following lines into the file ${HOME}/.bashrc
.
export CXX=clang++
export CC=clang
export VCPKG_FEATURE_FLAGS=-binarycaching
export VCPKG_ROOT=${HOME}/vcpkg
export EIGEN_INCLUDE=${VCPKG_ROOT}/installed/x64-osx/include/eigen3 # use arm64-osx in m1 Mac
export BOOST_INCLUDE=${VCPKG_ROOT}/installed/x64-osx/include # use arm64-osx in m1 Mac
# Skip the following for Mac M1 (due to MKL)
export MKL_THREADING_LAYER=SEQUENTIAL
export MKL_INTERFACE_LAYER=LP64
export LD_LIBRARY_PATH=/Library/Frameworks/Python.framework/Versions/3.9/lib:${LD_LIBRARY_PATH}
export MKL_LIB=/Library/Frameworks/Python.framework/Versions/3.9/lib
export MKL_INCLUDE=/Library/Frameworks/Python.framework/Versions/3.9/include
Install the following packages with Homebrew.
brew install ninja cmake pkg-config
The recommended way to get C++ package is via vcpkg.
cd ${HOME}
git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg install eigen3 catch2 boost nlohmann-json
The installation of boost
will take a long time, be patient
Skip this step for Mac M1.
The easiest way to install mkl
is through pip
. Since you can have multiple Python environment which may depend on a
single mkl
library. It is better to install mkl
without virtual environment. Make sure your current shell is NOT
in a virtual environment (no parentheses before the prompt). Without sudo
, it will install the mkl
into the
folder ${HOME}/.local
.
python3.9 -m pip install mkl mkl-include mkl-devel
It is recommended to create a virtual environment. Clone repository, create and activate virtual environment, and install the build dependency. go to a root folder you prefer to save the repositories.
git clone https://github.com/alliander-opensource/power-grid-model.git # or git@github.com:alliander-opensource/power-grid-model.git
cd power-grid-model
python3.9 -m venv .venv
source ./.venv/bin/activate
pip install -r dev-requirements.txt
Install from source in develop mode, and run pytest
.
pip install -e .
pytest
There is a convenient shell script to build the cmake project:
build.sh
. You can study the file and write your own build script. Six configurations are pre-defined
for two input arguments, which will be passed into cmake
. It includes debug or release build, as well as the option to
use MKL at link-time, or at runtime, or not at all.
- Option 1
- Debug
- Release
- Option 2
- EIGEN
- MKL
- MKL_RUNTIME
As an example, go to the root folder of repo. Use the following command to build the project with MKL at link-time and benchmark:
./build.sh Release MKL
One can run the unit tests and benchmark by:
./cpp_build_Release_MKL/tests/cpp_unit_tests/power_grid_model_unit_tests
./cpp_build_Release_MKL/tests/benchmark_cpp/power_grid_model_benchmark_cpp