BSPONMPI Template C++ Project
This is a template project which uses SCons to automatically locate bsponmpi and its dependencies. Also, it can test for a few more libraries I found useful.
At minimum, you will need:
- SCons for making/compiling the project
For writing parallel code, you need:
- The Boost libraries.
- Threading Building Blocks
- Optionally: a version of MPI (e.g. OpenMPI or Microsoft HPC Pack) on Windows)
Additionally, you can check for these:
- A version of BLAS e.g. OpenBLAS
- Agner Fog's Vectorclass library
- MATLAB : MATLAB, The MathWorks, Inc., Natick, Massachusetts, United States. (for building MEX files/writing MAT files).
You can then compile the project by running
... in the project's checkout folder (make clean using
The locations of all dependencies can be specified in an options file or on the command line. A straightforward way to get the options file is to copy
opts.py from bsponmpi.
A summary of all options is shown when running
$ scons --help
All source code is contained in
src. An autoconfig header is created in there, too. Also, the
include directory can contain public API files (make sure these don't depend on the autoconfig header).
The only files you will hopefully need to modify are
src/SConscript, and (potentially) an options file (see section below).
src/SConscript, you can add files to your project. By default, we compile
Import("root") root.Program('#bin/main', 'main.cpp')
Everything that is compiled using
root will have access to all libraries which were detected during configuration.
tests/SConscript will attempt to build and execute unit tests. All C++ files which have a name starting with
test_ will be compiled and run. If their return value is zero, the test is assumed to have run successfully.
An example is shown in
tests/test_main.cpp. The compiled test executables go into
When running the default version of this project, this should be the output:
$ scons runtests=1 mode=release scons: Reading SConscript files ... To use specific options for this platform, use options file "opts_Darwin_i386.py" To use specific options for this system, use options file "opts_naphta.cov.warwick.ac.uk_Darwin_i386.py" Using options from opts.py Performing autoconfiguration for Darwin/i386 Checking for bsponmpi... found NOMPIMT (cached) yes Checking for Boost version >= 1.40... yes Checking for TBB version >= 3.0... yes Checking for Agner Fog's libraries veclib(y) asmlib(y) (cached) yes Checking for CBLAS... (cached) no No version of BLAS was found. Have a look at xscons/blas.py scons: done reading SConscript files. scons: Building targets ... g++ -o src/main.o -c -g -fast -Iinclude -Isrc -I/Users/peterkrusche/Documents/Code/workspace/bsponmpi/include src/main.cpp g++ -o bin/main -g src/main.o -Llib -L/Users/peterkrusche/Documents/Code/workspace/bsponmpi/lib -ltbb -ltbbmalloc -lboost_program_options -lbsponmpinompi_mt g++ -o tests/test_main.o -c -g -fast -Iinclude -Isrc -I/Users/peterkrusche/Documents/Code/workspace/bsponmpi/include tests/test_main.cpp g++ -o bin/unit_tests/main -g tests/test_main.o -Llib -L/Users/peterkrusche/Documents/Code/workspace/bsponmpi/lib -ltbb -ltbbmalloc -lboost_program_options -lbsponmpinompi_mt builder_unit_test(["bin/unit_tests/main.passed"], ["bin/unit_tests/main"]) Great success. scons: done building targets.
Options can be specified by means of SCons build variables. These can be given on the command line, e.g. like this:
$ scons mode=release
This will compile release binaries (with optimisation enabled, ...).
Another way is to create an options file. You can create a file called
opts.py which looks like this:
mode = 'release'
If you have configuration files for different platforms/hosts which you want to store in a repository without overwriting, you can use a platform-specific, or a host-specific options file. SCons will tell you the names of these files when it is run:
$ scons scons: Reading SConscript files ... To use specific options for this platform, use options file "opts_Darwin_i386.py" To use specific options for this system, use options file "opts_naphta.cov.warwick.ac.uk_Darwin_i386.py" ...
Here is a list of variables you can specify.
- mode Build mode: set to debug or release', 'debug',
- profile Enable profiling (Linux). Also enables debug information.
- runtests Run unit tests.
- debuginfo Switch to include debug information also in release version, defaults to 1
- toolset Specify compiler and linker tools: msvc|gnu|intel -- see
- additional_lflags and additional_cflags these allow you to add additional compiler and linker flags
- mpiexec MPI exec command for testing
- mpiexec_params MPI exec parameters for testing -- you can specify the number of processors to test on, e.g. using
Getting Boost on Linux or Mac OS X is relatively easy, use your favourite package manager, e.g.
sudo apt-get install libboost-all-dev
...or MacPorts on MacOS/X)
On Windows (or if you want the most recent version of Boost), you will need to compile Boost, which may take some time. Once you have compiled Boost, you will need to install it somewhere (see their documentation). On Windows, the default folder is in
C:\\Boost. The build variable
boostdir needs to point to the include directory. On Windows, Boost will install different versions in subfolders of the include folder. In
xscons/boost.py, this is handled by trying to choose the most recent version of Boost.
These options are available in
- boostdir : the include directory for boost.
- boost_minversion : minimum version of Boost required (default: 1.40)
- boostlibs Boost libraries to link with, separated by comma/whitespace/colon. Default is
boost_program_options, because bsponmpi needs this (when setting this option, don't forget to include it if you want to use bsponmpi
The easiest way to use TBB is to download the binary distribution for your platform, and point to it in the options file using the
tbbdir variable e.g. like this:
tbbdir = "Z:\\Peter\\workspace\\tbb40_297oss"
tbb.py will try to automatically figure out which version of the library to link to.
xscons/bsponmpi.py will try to automatically determine in which mode
bsponmpi was compiled (this is saved in BSPonMPI's
bsp_config.h) and link with the right library and MPI if necessary.
Using the variable
bsponmpidir, you can set the path to bsponmpi (default:
On Windows, you need to specify the location of Microsoft Compute Cluster Pack using
MPIDIR (default: 'C:\Program Files\Microsoft Compute Cluster Pack').
On other platforms, the easiest way is to use
mpicc and friends. You can specify these individually using
Using the right version of BLAS will dramatically improve performance (rule of thumb: if the cblas_ddot flop rate is below 1 GFLOP/s for vectors which aren't too short, you're doing something wrong). The config variable
cblas can specify which version of BLAS we should link to. Possible options are :
none|blas|atlas|openblas|accelerate. If you require additional library/include paths, you can use the
additional_cflags variables to add these.
This is a useful library for implementing vector operations using MMX/SSE/AVX (also, there is a library with a faster version of
You can use the configuration variables
asmlibdir to point to the locations of these libraries (default locations are in
In site_scons/site_tools/mex.py, there is now a simple builder for MEX files. If MATLAB is on your path, it should work out of the box, otherwise, you can set the MATLAB_PATH build variable to the bin directory of your installation e.g. like this:
$ scons MATLAB_PATH=/Applications/MATLAB_R2012b.app/bin
if root['MEX_EXT'] != '': root.MEX('#bin/mex_test', 'mex_test.cpp')
Using boost in MEX files can make trouble. MATLAB ships with a dynamic library version of Boost which may be incompatible with the one you're using (e.g. it appears that the one that comes with R2012b doesn't know about the Bzip2 part in boost::iostreams). To fix such runtime link errors, I found the easiest way is to link boost statically (build boost using
b2 link=static). Other possible (and more tricky) solutions are:
- This one (which I wasn't able to reproduce on MacOS X, but it probably works on Linux): http://stackoverflow.com/questions/13934107/using-boost-in-matlab-mex-library-different-from-matlabs-version
- Fix rpath manually in the shared object/Mex file after building.