INSTALL.md

Installing Shogun

For certain systems, we offer pre-built packages of Shogun. This is the easiest way to start using it. For other cases, we describe how to build Shogun from source code.

Quicklinks

• Ready-to-install packages
  • Anaconda
  • Ubuntu
  • Debian
  • Fedora
  • MacOS
  • Windows
• Docker images
• Integration with interface languages
  • Python
• Compiling manually
  • Requirements
  • Basics
  • Interfaces
  • Examples
  • Problems
  • CMake tips
  • Customized Python
  • Windows

Ready-to-install packages

Anaconda packages

The base shogun library and its Python interface are available through the conda package manager, via conda-forge. To install both:

conda install -c conda-forge shogun

or to get just the library:

conda install -c conda-forge shogun-cpp

These packages include most of the optional dependencies and are currently available for Linux, MacOS and Windows.

Ubuntu ppa

We are working on integrating Shogun with Debian/Ubuntu. In the meantime, we offer a prepackaged ppa. These currently do contain the C++ library and Python bindings. Add this to your system as

sudo add-apt-repository ppa:shogun-toolbox/stable
sudo apt-get update

Then, install as

sudo apt-get install libshogun18

The Python (2) bindings can be installed as

sudo apt-get install python-shogun

In addition to the latest stable release, we offer nightly builds of our development branch.

Debian

Latest packages for Debian jessie and stretch are available in our own repository at http://apt.shogun.ml. We provide both the stable and nightly packages, currently only for amd64 architecture. In order to add the stable packages to your system, simply run the following commands

sudo apt-key adv --keyserver keyserver.ubuntu.com --recv-keys 3DD2174ACAB30365
echo "deb http://apt.shogun.ml/ stretch main" | sudo tee /etc/apt/sources.list.d/shogun-toolbox.list  > /dev/null
sudo apt-get update

After this just simply install the shogun library

sudo apt-get install libshogun18

The nightly packages are available in the nightly component, i.e. deb http://apt.shogun.ml/ stretch nightly

Fedora

Shogun is part of Fedora 25. Install as

sudo dnf install shogun

MacOS

Shogun is part of Homebrew. Install the latest stable version as

brew install shogun

Windows

Shogun natively compiles under Windows using MSVC, see the AppVeyor CI build and the Windows section. We currently only support binary packages via conda. If you are interested in packaging, documenting, or contributing otherwise, please contact us.

Docker images

You can set up Shogun using our Docker images as:

sudo docker pull shogun/shogun:master
sudo docker run -it shogun/shogun:master bash

The docker image follows both the master and the develop branch of the repository, just specify the desired branch name as tag for the image. For example in order to use the develop version of shogun simply pull the shogun/shogun:develop docker image.

There's an SDK docker image for shogun development as well, which we use to run our Travis CI jobs.

Sometimes mounting a local folder into the docker image is useful. You can do this via passing an additional option

-v /your/local/folder:/same/folder/in/docker

See the Docker documentation for further details.

Building shogun - Using vcpkg

You can download and install shogun using the vcpkg dependency manager:

git clone https://github.com/Microsoft/vcpkg.git
cd vcpkg
./bootstrap-vcpkg.sh
./vcpkg integrate install
./vcpkg install shogun

The shogun port in vcpkg is kept up to date by Microsoft team members and community contributors. If the version is out of date, please create an issue or pull request on the vcpkg repository.

Integration with interface language build systems

Shogun is can be automatically built from source from the following langauges.

Python pypi

You can install from pypi. There is limited control over options and it might take a long time as everything is done from scratch.

pip install shogun-ml

We do not recommend this option and suggest to rather compile by hand as described below.

Compiling manually

In case none of the binary packages listed on our website work for your system, or you want to modify Shogun, you will need to build it from source.

Requirements

The standard GNU/Linux tools and Python are minimal requirements to compile Shogun.

By default, building the meta examples is enabled, requiring python-ply and ctags. Disable using -DBUILD_META_EXAMPLES=Off if those requirements are a problem, also see generating meta examples.

To compile the interfaces, in addition to swig itself, you will need language specific development packages installed, see interfaces below.

There is a larger number of optional requirements. The output of cmake output lists optional dependencies that were found and not found. If a particular Shogun class is unavailable, this is likely due to an unmet dependency. See our docker configuration file for an example configuration used in our test builds.

You need at least 1GB free disk space. If you compile any interface, roughly 4 GB RAM are need (we are working on reducing this). CCache will massively speed up the compilation process and is enabled by default if installed.

Basics

Shogun uses CMake for its build. The general workflow is now explained. For further details on testing etc, see DEVELOPING.md.

Download the latest stable release source code, or (as demonstrated here) clone the latest develop code. Potentially update submodules

git clone https://github.com/shogun-toolbox/shogun.git
cd shogun
git submodule update --init

Create the build directory in the source tree root

mkdir build

Configure cmake, from the build directory, passing the Shogun source root as argument. It is recommended to use any of CMake GUIs (e.g. replace cmake .. with ccmake ..), in particular if you feel unsure about possible parameters and configurations. Note that all cmake options read as -DOPTION=VALUE.

cd build
cmake [options] ..

Compile

make

Install (prepend sudo if installing system wide), and your are done.

make install

Sometimes you might need to clean up your build (e.g. in case of some major changes). First, try

make clean

If that does not help, try removing the build directory and starting from scratch afterwards

rm -rf build

If you prefer to not run the sudo make install command system wide, you can either install Shogun to a custom location (-DCMAKE_INSTALL_PREFIX=/custom/path, defaults to /usr/local), or even skip make install at all. In both cases, it is necessary to set a number of system libraries for using Shogun, see INTERFACES.md.

Interfaces

The native C++ interface is always included. The cmake options for building interfaces are -DINTERFACE_PYTHON=ON -DINTERFACE_R .. etc. For example, replace the cmake step above by

cmake -DINTERFACE_PYTHON=ON [potentially more options] ..

The required packages (here debian/Ubuntu package names) for each interface are

• Python
  • python-dev python-numpy
  • For dealing with customized Python environments, see here
• Octave
  • octave liboctave-dev
• R
  • r-base-core
• Java
  • oracle-java8-installer, non-standard, e.g. https://launchpad.net/~webupd8team/+archive/ubuntu/java
  • jblas, a standard third party library, https://mikiobraun.github.io/jblas/
• Ruby
  • ruby ruby-dev, and narray a non-standard third party library, http://masa16.github.io/narray/, install with gem install narray
• Lua
  • lua5.1 liblua5.1-0-dev
• C-Sharp
  • mono-devel mono-gmcs cli-common-dev

To use the interfaces, in particular if not installing to the default system-wide location, see INTERFACES.md. See examples below for how to create the examples from the website locally.

Generating examples

All Shogun examples at our website are automatically generated code. You can generate them (plus additional ones) locally (needs cmake switch -DBUILD_META_EXAMPLES=ON)

make meta_examples

This requires PLY for Python, package python-ply, and ctags, package ctags. Both source code and potential executables (C++, Java, C-Sharp) are created in build/examples/meta/ when running make.

See INTERFACES.md to run the generated examples and see EXAMPLES.md for more details on their mechanics. See DEVELOPING.md for how the examples are used as tests.

Problems? Got stuck? Found a bug? Help?

In case you have a problem building Shogun, please open an issue on github with your system details, exact commands used, and logs posted as a gist.

CMake tips

CMake is a beast. Make sure to read the docs and CMake_Useful_Variables. Make sure to understand the concept of out of source builds. Here are some tips on common options that are useful

Specify a different swig executable:

cmake -DSWIG_EXECUTABLE=/usr/bin/swig_custom

To specify a different compiler, see CMake FAQ, "How do I use a different compiler?". You might have to delete the build directory or clear the cmake cache otherwise for this to work.

CC=/path/to/gcc CXX=/path/to/g++ cmake ..

In case header files or libraries are not at standard locations one needs to manually adjust the libray and include paths, -DCMAKE_INCLUDE_PATH=/my/include/path and -DCMAKE_LIBRARY_PATH=/my/library/path.

Customized Python environments

Often, there are multiple Python versions installed on the system. There are various reasons for this, i.e. Linux without root access, MacOS + homebrew, using Anaconda or virtualenv. If Shogun is executed using a different Python version that the one it was built against, one will observe crashes when importing Shogun. If this is your setup, you need to make sure that Shogun is both built and executed against the Python environment of your choice. For that, you need to do something similar to

cmake -DPYTHON_INCLUDE_DIR=/path/to/python/include/dir -DPYTHON_LIBRARY=path/to/python/libpythonVERSION.{so|dynlib} -DPYTHON_EXECUTABLE=/path/to/python/executable -DPYTHON_PACKAGES_PATH=/path/to/python/dist-packages ..

For example, for brew installed Python under MacOS, use something like:

cmake -DPYTHON_INCLUDE_DIR=/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Headers -DPYTHON_LIBRARY=/usr/local/Cellar/python/2.7.5/Frameworks/Python.framework/Versions/2.7/lib/libpython2.7.dylib  -DINTERFACE_PYTHON=ON ..

Under Linux, where you want to use Python 3, which is not the system's default:

cmake -DPYTHON_INCLUDE_DIR=/usr/include/python3.3 -DPYTHON_EXECUTABLE:FILEPATH=/usr/bin/python3 -DPYTHON_PACKAGES_PATH=/usr/local/lib/python3.3/dist-packages -DINTERFACE_PYTHON=ON ..

On a Linux cluster without root access, using Anaconda (note you will need to activate your environment everytime you want to run Shogun):

source path/to/anaconda/bin/activate
cmake -DCMAKE_INSTALL_PREFIX=path/to/shogun/install/dir -DPYTHON_INCLUDE_DIR=path/to/anaconda/include/python2.7/ -DPYTHON_LIBRARY=path/to/anaconda/lib/libpython2.7.so  -DPYTHON_EXECUTABLE=path/to/anaconda/bin/python -DINTERFACE_PYTHON=ON ..

Windows build

Please see our Azure Pipelines build. It is recommended to use "Visual Studio 16 2019" or "MSBuild". You will need to adjust all path names to the Windows style, e.g.

git clone https://github.com/shogun-toolbox/shogun.git C:\projects\shogun
git submodule -q update --init
cd C:\projects\shogun
md build && cd build

You need to specify a different generator in cmake (to match your IDE), e.g.

cmake -G"Visual Studio 16 2019 Win64" -DCMAKE_BUILD_TYPE=Release -DBUILD_META_EXAMPLES=OFF -DENABLE_TESTING=ON ..  

The above cmake has the following arguments:

1. Visual Studio 16 2019 Win64 specifies the target platform to be x64.
2. -DCMAKE_BUILD_TYPE=Release specifies a build type and asks compiler to perform optimization and omit debug information.
3. -DBUILD_META_EXAMPLES=OFF specifies to not generate meta examples.
4. -DENABLE_TESTING=ON Enable testing while cmake.

Compiling works as

msbuild "C:\projects\shogun\build\shogun.sln" /verbosity:minimal /t:Clean /p:Configuration=Release /p:Platform=x64

Note: If you use /m in msbuild command without specifying the number, it may occur out of memory errors.