Skip to content

Installing SMS++

Jump to bottom
Donato Meoli edited this page Feb 15, 2024 · 119 revisions

This guide contains detailed instructions that will help you to install the required tools and libraries needed to build SMS++.

[[TOC]]

Required tools

Building SMS++ requires the following tools:

  • A C++17-compliant C++ compiler. At the moment, SMS++ works with either GCC or Clang;
  • CMake, suggested for SMS++ but actually required for building some requirements;
  • Git and make.

In most cases, tools and requirements can be installed using common package managers like apt for Debian-based distributions, vcpkg for Windows, or homebrew for macOS.

macOS

Building stuff on macOS requires Apple's Command Line Tools. You can obtain them by either installing XCode and launching it once, or by running in the terminal:

xcode-select --install

CLT include the Clang C/C++ compiler, Git and make.

We suggest using homebrew for the other tools and requirements. Install it with:

/bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

Use it to install CMake:

brew install cmake

Debian/Ubuntu

The required tools can be installed using apt:

sudo apt install build-essential cmake git

build-essential includes GCC and make. If you prefer Clang, run:

sudo apt install clang
sudo update-alternatives --set cc /usr/bin/clang
sudo update-alternatives --set c++ /usr/bin/clang++

Other UNIX/Linux systems

We do not actively support other UNIX and Linux systems, but you can probably adapt the following instructions to your OS with little effort.

Windows

There are two different ways to build SMS++ under Windows.

The first way is using the WSL, so in this case please follow to the Debian/Ubuntu part of this guide.

The second one is a native way, does not use any kind of virtualization, and needs:

  • the MSVC toolchain, please check on/off the last version of the MS build tools for the command line and install it, notice that you will need admin rights.

  • the vcpkg package manager for Windows, better if installed under C:\.

The subsequent commands have to be executed in the directory where the vcpkg has been installed to install some libs already present in the UNIX based systems:

vcpkg install zlib bzip2 pthreads getopt --triplet x64-windows

MSVC includes both GCC and Clang C/C++ compilers, and also a porting of the UNIX make tool. MSVC also includes CMake, to install it check to on C++ CMake for Windows under the Desktop Development with C++ tools list. Otherwise, manually install following the instructions here.

To install Git follows the instructions here.

Getting started

Before installing the required libraries you should assess which SMS++ modules you really need: different modules have different requirements, and by excluding some modules you might avoid installing unnecessary software.

Depending on how many modules you need, you can proceed in two ways:

  • Use the SMS++ project (aka umbrella) and comment out the modules you don't need from the CMakeLists.txt file. This is the best solution if you want to install everything except maybe a few modules.
  • Fetch, build and install the modules individually, starting from the SMS++ core library. This is the best approach if you need only the library and few other modules.

Once you decided which modules install and how, you will need to install the required software.

Requirements

Note: requirements are also specified in each module's README file. Moreover, the CMakeLists.txt file of the umbrella sums them up briefly.

The SMS++ core library requires the following libraries (here is why they are needed):

So, you will need at least those. The other submodules add the following (nested) requirements:

  • BundleSolver

    • NDOSolver/FiOracle: it's included as a subproject of BundleSolver, so it will be compiled with the umbrella by default. If you are installing BundleSolver individually, you will need to manually install this first. This library requires these libraries:
      • CoinUtils, required by Clp and Osi;
      • Clp and Osi, optional for NDOSolver/FiOracle but actually needed by BundleSolver;
      • CPLEX and/or GUROBI, optional requirement for Osi, but actually needed by BundleSolver.

  • MCFBlock

    • MCFClass: it's included as a subproject of MCFBlock, so it will be compiled with the umbrella by default. If you are installing MCFBlock individually, you will need to manually install this first. Optionally, MCFClass requires CPLEX.

  • MILPSolver

    • CPLEX (optional, minimum version 12.8.0)
    • GUROBI (optional minimum versio 10.0.0)
    • SCIP (optional, minimum version 7.0.0)
    • HiGHS (optional, minimum version 1.5.3)

    Both requirements are optional, but you will need at least one of them to obtain a useful solver.

  • SDDPBlock

Below you will find the steps for each requirement.

Boost

On macOS, homebrew's repository includes an updated and working version:

brew install boost

On Debian/Ubuntu, if the repository has a supported version (at least 1.72) you can install it with:

# For the core library:
sudo apt install libboost-dev

# For StOpt (SDDPBlock) also do:
sudo apt install libboost-system-dev libboost-timer-dev libboost-mpi-dev

On Windows, you can install it with:

# For the core library:
vcpkg install boost --triplet x64-windows

# For StOpt (SDDPBlock) also do:
vcpkg install boost-mpi --triplet x64-windows

Note: In case of failure of the second one command, execute vcpkg\downloads\msmpisetup-<x.y.z>.exe as suggested by the output error message.

Otherwise, you will need to build it from the source. Download the source from here and extract it, then navigate the extracted directory and run:

# If you only need SMS++ core library:
./bootstrap.sh --with-libraries=headers
sudo ./b2 install

# If you also need Boost.system and Boost.timer for SDDPBlock:
./bootstrap.sh --with-libraries=headers,system,timer
sudo ./b2 install

You can find more details on how to obtain and build Boost here.

NetCDF-C++

NetCDF-C++ is actually a wrapper library for NetCDF-C, so you will need both.

On macOS, homebrew's repository includes a single, updated package that includes both NetCDF-C/C++:

brew install netcdf

On Windows you can install them with:

vcpkg install netcdf-cxx4 --triplet x64-windows

On Debian/Ubuntu you can install them with:

sudo apt install libnetcdf-c++4-dev

In some Debian and Ubuntu versions, we found that the packages provided in the repositories were missing the configuration file needed by CMake to find the library while building SMS++. If CMake can't find NetCDF-C/C++ while configuring SMS++, try follow these instructions to build them manually.

  1. First install NetCDF's requirements:

    # On Debian/Ubuntu
sudo apt install libhdf5-dev libcurl4-openssl-dev

# On macOS
brew install hdf5

  2. Download the latest NetCDF-C release from here and extract it.

  3. In the extracted directory, configure and build the library with CMake:

    mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DBUILD_UTILITIES=OFF -DENABLE_TESTS=OFF
cmake --build .
cmake --install .

  4. Download the latest NetCDF-C++ release from here and extract it.

  5. In the extracted directory, configure and build the library with CMake:

    export CPATH="/usr/include/hdf5/serial/"
mkdir build && cd build
cmake .. -DCMAKE_BUILD_TYPE=Release -DNCXX_ENABLE_TESTS=OFF
cmake --build .
cmake --install .

You can find more details on how to obtain and build NetCDF-C/C++ here.

Eigen

On macOS, homebrew's repository includes an updated and working version:

brew install eigen

On Debian/Ubuntu, the repository should contain an updated and working version:

sudo apt install libeigen3-dev

On Windows, Microsoft's vcpkg repository should contain an updated and working version:

vcpkg install eigen3 --triplet x64-windows

Otherwise, you will need to install it from the source.

  1. Download the latest Eigen release from here and extract it.

  2. In the extracted directory, configure and install the library with CMake:

    mkdir build && cd build
cmake ..
cmake --build .
cmake --install .

You can find more details on how to obtain and install Eigen here.

CPLEX

Install CPLEX using the installer provided by IBM for your OS. It's recommended to install it in the default directory, e.g., /Applications/CPLEX_Studio<ver> for macOS, /opt/ibm/ILOG/CPLEX_Studio<ver> on Linux, or C:\Program Files\IBM\ILOG\CPLEX_Studio<ver> for Windows.

For Windows, it's highly recommended to choose a space-free path, i.e., C:\IBM\ILOG\CPLEX_Studio<ver>, to avoid problems during the resource paths linking (e.g., lib and include files) during the Coin-Or Osi installation phase.

GUROBI

Install GUROBI using the installer you can find here. It's recommended to install it in the default directory, e.g., /Library/scip for macOS, /opt/scip for Linux, or C:\Program Files\SCIPOptSuite <ver> for Windows.

Note: Under Linux, GUROBI might be not found at runtime, in that case, run:

echo “/opt/gurobi<ver>/linux64/lib” > /etc/ld.so.conf.d/gurobi.conf
sudo ldconfig

SCIP

It's recommended to install the SCIP Optimization Suite using the installers you can find here. It's recommended to install it in the default directory, e.g., /Library/gurobi<ver> for macOS, /opt/gurobi<ver> for Linux, or C:\gurobi<ver> for Windows.

Before installing the suite, you will need to install the requirements:

# On Debian/Ubuntu - see note below
sudo apt install libgfortran-7-dev libtbb-dev

# On macOS - see note below
brew install gcc tbb

Note: The actual version of the GCC/Fortran library you need depends on the version of SCIP you want to install. The download page contains information about that.

Under Linux, SCIP might be not found at runtime, in that case, run:

echo “/opt/scip/lib” > /etc/ld.so.conf.d/gurobi.conf
sudo ldconfig

Once downloaded the installer for your OS, launch it:

chmod u+x SCIPOptSuite-<ver>-<OS>.sh
./SCIPOptSuite-<ver>-<OS>.sh --prefix=/opt/scip --exclude-subdir --skip-license

Note: By using the --skip-license flag you implicitly accept the license.

HiGHS

To install HiGHS you will need to build sources. It's recommended to install it in the default directory, e.g., /Library/HiGHS for macOS, /opt/HiGHS for Linux, or C:\HiGHS for Windows.

So, after placing in one of these folders, download the sources:

git clone https://github.com/ERGO-Code/HiGHS.git

then configure them:

cd HiGHS
mkdir build && cd build
# On Debian/Ubuntu
cmake .. -DFAST_BUILD=ON -DCMAKE_INSTALL_PREFIX=/opt/HiGHS
# On Windows
cmake .. -G "MSYS Makefiles" -DFAST_BUILD=ON -DCMAKE_INSTALL_PREFIX=C:/HiGHS

finally, build, run tests, and optionally install:

cmake --build .
cmake --install .

Note: Under Linux, HiGHS might be not found at runtime, in that case, run:

echo “/opt/HiGHS/lib” > /etc/ld.so.conf.d/highs.conf
sudo ldconfig

COIN-OR CoinUtils

On macOS, homebrew's repository includes an updated version of CoinUtils. This package has the following downside: it depends on OpenBLAS instead of using macOS's own BLAS implementation, so you will install an otherwise useless package.

If you are fine with that, you can do:

brew install coinutils

On Debian/Ubuntu, so you can do:

sudo apt install coinor-libcoinutils-dev

On Windows, in the vcpkg installation folder, you can do:

vcpkg install coinutils pybind11 --triplet x64-windows

Otherwise, you have two choices for installing it from the source:

  • Use coinbrew, COIN-OR's own installation helper
  • Fetch and build the libraries manually

The complexity is more or less the same.

Coinbrew

coinbrew in an helper script that will fetch, build and install COIN-OR packages. By default, it would also build and install some requirements such as BLAS and LAPACK, but it's recommended to install them first with your package manager to save some time:

  1. On Debian/Ubuntu, install the requirements with apt:

    sudo apt install libbz2-dev liblapack-dev libopenblas-dev

    On Windows, install the requirements with vcpkg:

    vcpkg install blas lapack --triplet x64-windows

    macOS already has those requirements installed.

  2. Get the coinbrew script with:

    curl -O https://raw.githubusercontent.com/coin-or/coinbrew/master/coinbrew
chmod u+x coinbrew

  3. Run coinbrew with the following parameters:

    ./coinbrew build CoinUtils --latest-release \
    --skip-dependencies \
    --prefix=/opt/coin-or \
    --tests=none

See the coinbrew page for further details.

Manual install

In case you need to build the libraries manually, follow these instructions.

  1. On Debian/Ubuntu, install the requirements with apt:

    sudo apt install libbz2-dev liblapack-dev libopenblas-dev

    On Windows, install the requirements with vcpkg:

    vcpkg install blas lapack --triplet x64-windows

    macOS already has those requirements installed.

  2. Download the latest CoinUtils release from here and extract it.

  3. In the extracted directory, configure and build the library with CMake:

    ./configure --prefix=/opt/coin-or
cmake --build .
cmake --install .

COIN-OR Osi/Clp

The precompiled packages of these libraries have the following downside: Osi packages do not come with CPLEX support, which is required by BundleSolver; and since Clp packages depend on Osi ones, they cannot be used, either.

For installing Osi and Clp with CPLEX support you have two choices:

  • Use coinbrew, COIN-OR's own installation helper
  • Fetch and build the libraries manually

The complexity is more or less the same.

Coinbrew

See above how to get coinbrew and install CoinUtils. Then follow these steps.

  1. Fetch and install Osi:

    # Set here your main CPLEX Studio directory
CPLEX_DIR=<cplex-studio-dir>
CPLEX_LIB_DIR=`ls -bd1 $CPLEX_DIR/cplex/lib/*/static_pic | tail -n1`

#Set here your main GUROBI directory
GUROBI_DIR=<gurobi-dir>
GUROBI_LIB_DIR=`ls -bd1 $GUROBI_DIR/*/lib | tail -n1`
GUROBI_INCLUDE_DIR=`ls -bd1 $GUROBI_DIR/*/include | tail -n1`

./coinbrew build Osi --latest-release \
    --skip-dependencies \
    --prefix=/opt/coin-or \
    --tests=none \
    --with-cplex \
    --with-cplex-lib="-L$CPLEX_LIB_DIR -lcplex -lpthread -lm -ldl" \
    --with-cplex-incdir="$CPLEX_DIR/cplex/include/ilcplex" \
    --with-gurobi \
    --with-gurobi-lib="-L$GUROBI_LIB_DIR -lgurobi100" \
    --with-gurobi-incdir="$GUROBI_INCLUDE_DIR"

    Note: On macOS the configuration script may not be able to properly check the CPLEX library via the CPXgetstat() symbol. If you think that your --with-cplex-lib flag is correct, you can suppress the check with the --disable-cplex-libcheck flag.

  2. Fetch and install Clp:

    ./coinbrew build Clp --latest-release \
    --skip-dependencies \
    --prefix=/opt/coin-or \
    --tests=none

See the coinbrew page for further details.

Manual install

On Windows, in order to be able to install the package coin-or-osi with the interfaces to CPLEX and/or GUROBI, we will need to modify its vcpkg configuration file, so go to vcpkg\ports\coin-or-osi\portfile.cmake and substitute:

--without-cplex
--without-gurobi

with something like this:

--with-cplex
--with-cplex-lib=C:\/IBM\/ILOG\/CPLEX_Studio<ver>\/cplex\/lib\/x64_windows_msvc<ver>\/stat_mda\/cplex<ver>.lib
--with-cplex-incdir=C:\/IBM\/ILOG\/CPLEX_Studio<ver>\/cplex\/include\/ilcplex
--with-cplex-cflags=-IC:\/IBM\/ILOG\/CPLEX_Studio<ver>\/cplex\/include\/ilcplex
--with-cplex-lflags=C:\/IBM\/ILOG\/CPLEX_Studio<ver>\/cplex\/lib\/x64_windows_msvc<ver>\/stat_mda\/cplex<ver>.lib

and this:

--with-gurobi
--with-gurobi-lib=C:\/gurobi<ver>\/win64\/lib\/gurobi<ver>.lib
--with-gurobi-incdir=C:\/gurobi<ver>\/win64\/include
--with-gurobi-cflags=-IC:\/gurobi<ver>\/win64\/include
--with-gurobi-lflags=C:\/gurobi<ver>\/win64\/lib\/gurobi<ver>.lib

respectively, with your paths to the same resources.

Note: It may be needed to not have any white spaces in the paths for this to work properly. Also notice that the \/ for Windows path specification is not optional.

Then, from the vcpkg installation folder, you can finally install the packages:

vcpkg install coin-or-osi coin-or-clp glpk pybind11 --triplet x64-windows

In case you need to build the libraries manually, follow these instructions.

  1. Download the latest Osi release from here and extract it.

  2. In the extracted directory, configure and build the library with CMake:

    # Set here your main CPLEX Studio directory
CPLEX_DIR=<cplex-studio-dir>
CPLEX_LIB_DIR=`ls -bd1 $CPLEX_DIR/cplex/lib/*/static_pic | tail -n1`

# Set here your main GUROBI directory
GUROBI_DIR=<gurobi-dir>
GUROBI_LIB_DIR=`ls -bd1 $GUROBI_DIR/*/lib | tail -n1`
GUROBI_INCLUDE_DIR=`ls -bd1 $GUROBI_DIR/*/include | tail -n1`

./configure --prefix=/opt/coin-or \
    --with-cplex \
    --with-cplex-lib="-L$CPLEX_LIB_DIR -lcplex -lpthread -lm -ldl" \
    --with-cplex-incdir="$CPLEX_DIR/cplex/include/ilcplex" \
    --with-gurobi \
    --with-gurobi-lib="-L$GUROBI_LIB_DIR -lgurobi100" \
    --with-gurobi-incdir="$GUROBI_INCLUDE_DIR"
cmake --build .
cmake --install .

    Note: On macOS the configuration script may not be able to properly check the CPLEX library via the CPXgetstat() symbol. If you think that your --with-cplex-lib flag is correct, you can suppress the check with the --disable-cplex-libcheck flag.

  3. Download the latest Clp release from here and extract it.

  4. In the extracted directory, configure and build the library with CMake:

    ./configure --prefix=/opt/coin-or
cmake --build .
cmake --install .

StOpt

On the most recent Debian/Ubuntu distributions, the Stochastic Control library is available with apt:

sudo apt install libstopt5 libstopt-dev

Otherwise, you will need to build and install from the source:

  1. Install Eigen, Boost.system and Boost.timer (see above).

  2. On Debian/Ubuntu, install the requirements with apt:

    sudo apt install zlib1g-dev libbz2-dev

    macOS already has those requirements installed.

  3. Download the latest StOpt source code:

    git clone https://gitlab.com/stochastic-control/StOpt

  4. Then configure and build the library with CMake:

    cd StOpt
mkdir build && cd build
cmake .. -DBUILD_PYTHON=OFF -DBUILD_TEST=OFF
cmake --build .
cmake --install . --prefix /opt/stopt

On Windows, to install the StOpt package we will need to install before its vcpkg registry since it is not available on the public Microsoft vcpkg repository, so download:

git clone https://gitlab.com/stochastic-control/vcpkg-registry

and then, back to the vcpkg installation folder, and run:

vcpkg install stopt --overlay-ports=C:\vcpkg\vcpkg-registry\ports\stopt --triplet x64-windows

making attention to substitute the right vcpkg path with yours.

SMS++

You can proceed in two ways, depending on how many modules you need:

  • Use the umbrella to fetch multiple modules at once;
  • Fetch, build and install the modules individually.

If you prefer, you can use the instructions in the README files of the individual projects.

Using the umbrella

  1. Fetch the SMS++ project and all its modules at once, with the command:

    git clone -b develop --recurse-submodules https://gitlab.com/smspp/smspp-project.git # see note below
cd smspp-project

    Alternatively, you can save some time by fetching only the modules you need:

    git clone -b develop https://gitlab.com/smspp/smspp-project.git # see note below
cd smspp-project
git submodule update --init SMS++ UCBlock tools # ...

    Note: It is highly recommended to clone the develop branch directly, i.e., -b develop, if any, without passing from the master one, since the development of the SMS++ library is rapid and the master branch quickly becomes stale.

  1. Optional: edit the CMakeLists.txt file and comment the lines referring to the modules you don't need.

  2. Use CMake to configure and build the project:

    # On Debian/Ubuntu or macOS
mkdir build && cd build
cmake ..

# On Windows
mkdir build && cd build
cmake .. --DCMAKE_BUILD_TYPE=Debug -DCMAKE_TOOLCHAIN_FILE=C:\vcpkg\scripts\buildsystems\vcpkg.cmake # see note below

    making attention to substitute the right vcpkg path with yours.

    You can specify some configuration options in the cmake command (see here).

    Note: The Release build seems to be broken at the moment on Windows due to a bug with the vcpkg version of netcdf package, so build with the --DCMAKE_BUILD_TYPE=Debug option.

    Consider to use the CMake flag -Wno-dev to suppress warnings that are intended for the developers of the CMake project and not the users.

  3. Build and install the project with:

    cmake --build .
cmake --install . # optional

Alternatively, you can build the project without using CMake. See the Build and install with makefiles section in the README file of the project.

Build the individual modules

Repeat the following steps for each module you want to build (as an example, the steps apply to the SMS++ core library).

  1. Fetch the module with the command:

    git clone -b develop https://gitlab.com/smspp/smspp.git # see note below
cd smspp

    Note: It is highly recommended to clone the develop branch directly, i.e., -b develop, if any, without passing from the master one, since the development of the SMS++ library is rapid and the master branch quickly becomes stale.

  1. Use CMake to configure the module:

    # On Debian/Ubuntu or macOS
mkdir build && cd build
cmake ..

# On Windows
mkdir build && cd build
cmake .. --DCMAKE_BUILD_TYPE=Debug -DCMAKE_TOOLCHAIN_FILE=C:\vcpkg\scripts\buildsystems\vcpkg.cmake # see note below

    making attention to substitute the right vcpkg path with yours.

    You can specify some configuration options in the cmake command (see here).

    Note: The Release build seems to be broken at the moment on Windows due to a bug with the vcpkg version of netcdf package, so build with the --DCMAKE_BUILD_TYPE=Debug option.

    Consider to use the CMake flag -Wno-dev to suppress warnings that are intended for the developers of the CMake project and not the users.

  2. Build and install the project with:

    cmake --build .
cmake --install . # see note below

    Note: Installation is required for other modules to find the library. As an alternative, you can use CMake's User Package Registry, see the relevant section in the customization guide.

The test environment

You can find a good example of the above steps for the installation of SMS++ and its requirements in our Test Environment.

You can either use the environment as-is or use its scripts as a reference for installing SMS++ in your machine. In particular, smsbuild script allows to build SMS++ in the two methods described above (with or without the umbrella) and provides several customization options.

Issues and troubleshooting

If you need support, you want to submit bugs or propose a new feature for an individual module, see the Getting help section of the README for that module.

If you need support on the project installation that is not covered by the troubleshooting page, or want to propose a new module, you can open a new issue.

Clone this wiki locally