INSTALL

# Software Stack:

* C++ compiler with C++11 standard support. 
  Supported compilers include:
    * GCC 4.7 or later (Linux only)
    * Clang 3.3 or later
    * Apple Clang 5.0 or later (OS X only)  
    * Intel 14 or later
  Note, GCC is NOT supported on OS X. There are compatibility issues that arise
  when mixing libraries compiled with clang (or system libraries) with those
  compiled with GCC.
* Cmake 2.8.8 or later
* Git 1.8 or later (required to download TiledArray and MADNESS from GitHub)
* Eigen 3.x or later
  * Mercurial (required to download Eigen from Bitbucket)
* MADNESS
  Only the MADworld and MADtensor library of MADNESS are required by TiledArray.
  Requirements for MADNESS include:
    * autoconf 2.59 or later
    * automake 1.11 or later
    * libtool 2.x or later (??? not sure on earliest working version)
    * Intel TBB  4.x or later (4.3 Update 5 or later on BlueGene/Q)
    * MPI-2 or MPI-3
    * LAPACK and BLAS
    * Elemental (optional)

Optional:
* Doxygen (required to generating documentation)
* Boost 1.33.0 or later (required for unit tests)

Dependencies can be installed via the package manager of your choice, e.g.
Homebrew on OS X or apt-get on Debian Linux distributions. See notes below for
more details on installing MADNESS.

# Configure TiledArray

TiledArray is configured and built with CMake, which is similar in function to
the typical configure script. When configuring with CMake, you specify a set
of cache variables where each variable argument is prepended with the '-D'
option. Typically, you will need to specify the install path for TiledArray,
build type, MPI Compiler wrappers, and BLAS and LAPACK libraries. Though other
options my be necessary. Below are several common configure examples. Most of
the dependencies can be installed with a package manager (except for MADNESS),
which is the preferred method. Since configuring
and building MADNESS can be difficult on some systems, we recommend letting the
TiledArray configure script download and build MADNESS for you.

* Basic configuration. This will search for dependencies on your system. If the 
  required dependencies are not found on your system, they will be downloaded 
  and installed during the build process (this includes Eigen, Boost, Elemental,
  and MADNESS, but not MPI or TBB). The CMAKE_PREFIX_PATH cache variables
  is a semicolon separated list of search paths. 

$ cmake -D CMAKE_INSTALL_PREFIX=/path/to/install/tiledarray \
        -D CMAKE_BUILD_TYPE=Release \
        -D CMAKE_PREFIX_PATH=/path/to/dependency;/path/to/another/dependency \
        /path/to/tiledarray/source
        
* Basic configuration with unit tests.

$ cmake -D CMAKE_INSTALL_PREFIX=/path/to/install/tiledarray \
        -D CMAKE_BUILD_TYPE=Debug \
        -D TA_BUILD_UNITTEST=ON \
        -D BOOST_ROOT=/path/to/boost \
        -D CMAKE_PREFIX_PATH=/path/to/dependency;/path/to/another/dependency \
        /path/to/tiledarray/source
        
* Specify dependencies, w/o MADNESS installed on the system

$ cmake -D CMAKE_INSTALL_PREFIX=/path/to/install/tiledarray \
        -D CMAKE_BUILD_TYPE=Release \
        -D EIGEN3_INCLUDE_DIR=/path/to/eigen/include/eigen3 \
        -D LAPACK_LIBRARIES="-L/path/to/lapack/lib -llapack" \
        -D BLAS_LIBRARIES="-L/path/to/blas/lib -lblas" \
        -D TBB_ROOT_DIR=/path/to/tbb \
        -D CMAKE_C_COMPILER=gcc \
        -D CMAKE_CXX_COMPILER=g++ \
        -D MPI_C_COMPILER=mpicc \
        -D MPI_CXX_COMPILER=mpicxx \
        /path/to/tiledarray/source

* Specify dependencies, w/ MADNESS installed on the system

$ cmake -D CMAKE_INSTALL_PREFIX=/path/to/install/tiledarray \
        -D CMAKE_BUILD_TYPE=Release \
        -D EIGEN3_INCLUDE_DIR=/path/to/eigen/include/eigen3 \
        -D LAPACK_LIBRARIES="-L/path/to/lapack/lib -llapack" \
        -D BLAS_LIBRARIES="-L/path/to/blas/lib -lblas" \
        -D MADNESS_ROOT_DIR=/path/to/madness \
        -D CMAKE_C_COMPILER=gcc \
        -D CMAKE_CXX_COMPILER=g++ \
        -D MPI_C_COMPILER=mpicc \
        -D MPI_CXX_COMPILER=mpicxx \
        /path/to/tiledarray/source

Additional configure options are given below. You may also configure TiledArray
with the accompanying "configure" script, which is a wrapper for CMake.

# Common complier flags

* CMAKE_C_COMPILER -- The C compiler
* CMAKE_CXX_COMPILER -- The C++ compiler
* CMAKE_C_FLAGS -- The C compile flags (includes CPPFLAGS and CFLAGS)
* CMAKE_CXX_FLAGS -- The C++ compile flags (includes CPPFLAGS and CXXFLAGS)
* CMAKE_EXE_LINKER_FLAGS -- The executable compile flags
* CMAKE_BUILD_TYPE -- Optimization/debug build type options include empty, 
  Debug, Release, RelWithDebInfo and MinSizeRel.
  
It is typically not necessary to specify optimization or debug flags as the
default values provided by CMake are usually correct.

# MPI

You may choose from MPICH, MVAPICH, OpenMPI, Intel MPI, or your vendor provided 
MPI implementation. Specify the C and C++ MPI compiler wrappers with the 
following CMake cache variables:

* MPI_C_COMPILER -- The MPI C compiler wrapper
* MPI_CXX_COMPILER -- The MPI C++ compiler wrapper

You can build TiledArray without MPI support by setting ENABLE_MPI to OFF.
Though we strongly recommend compiling with MPI even if you do not intend
to use TiledArray in a distributed memory environment. Note, if you
build MADNESS yourself, you must also configure MADNESS with --enable-stub-mpi
to enable this option.

# BLAS and LAPACK

TiledArray requires a serial BLAS implementation, either by linking with a
serial version of the BLAS library or by setting the number of threads to one
(1) with an environment variable. This is necessary because TiledArray handles
parallelism with tasks, and attempting to run threaded BLAS function inside
tasks will over subscribe the hardware cores. By default, TiledArray configure
will search your system for compatible BLAS and LAPACK libraries. However, it is
recommended you specify the BLAS and LAPACK libraries if there is a serial
version available since CMake will prefer the threaded version of the library. 
You may specify the necessary libraries and/or link flags with the following
CMake cache variables:

* LAPACK_LIBRARIES -- LAPACK libraries (including dependencies)
* BLAS_LIBRARIES -- BLAS libraries (including dependencies)
* LAPACK_LINKER_FLAGS -- Linker flags required to link LAPACK
* BLAS_LINKER_FLAGS -- Linker flags required to link BLAS

You may also need to specify the integer size for your BLAS library with:

* INTEGER4 -- Specifies the integer size of BLAS function calls [Default=TRUE]
      TRUE = Fortran integer*4, FALSE = Fortran integer*8

You should use the default value unless you know it is necessary for your BLAS
implementation.

Common optimized libraries OpenBLAS/GotoBLAS, BLIS, MKL (on Intel platforms),
Atlas, Accelerate (on OS X), ESSL (on BlueGene platforms), or ACML (on AMD 
platforms). You can also use the Netlib reference implementation if nothing else
is available, but this will be very slow.

Example flags:

* Accelerate on OS X

  -D BLAS_LINKER_FLAGS="-framework Accelerate"

* OpenBLAS with Netlib LAPACK

  -D LAPACK_LIBRARIES="-L/path/to/lapack/lib -llapack"
  -D BLAS_LIBRARIES="-L/path/to/openblas/lib -lopenblas -lpthread"

* Netlib

  -D LAPACK_LIBRARIES="-L/path/to/lapack/lib -llapack"
  -D BLAS_LIBRARIES="-L/path/to/blas/lib -lblas"

* MKL on Linux

  -D BLAS_LIBRARIES="-L${MKLROOT}/lib/intel64 -Wl,--start-group -lmkl_intel_lp64 -lmkl_core -lmkl_sequential -Wl,--end-group -lpthread -lm”
  
* MKL on OS X

  -D BLAS_LIBRARIES="-L${MKLROOT}/lib -lmkl_intel_lp64 -lmkl_core -lmkl_sequential -lpthread -lm"

For additional information on linking different versions of MKL, see the MKL
Link Advisor page.

    https://software.intel.com/en-us/articles/intel-mkl-link-line-advisor

# Eigen 3

You can specify the install location of Eigen 3 with the following CMake cache
variable:

* EIGEN3_INCLUDE_DIR -- The path to the Eigen 3 include directory

If Eigen is not found by the configure script, it will be downloaded from the
hg repository on Bitbucket.

# MADNESS

TiledArray uses a non-release version of MADNESS. Therefore, you should NOT
expect the most recent release of MADNESS to work with TiledArray. To ensure you
are using the correct version of MADNESS, we recommend allowing CMake to
automatically download, configure, and build MADNESS (this is the default
behavior). When CMake is building MADNESS, the configure script will checkout
the correct revision of MADNESS.

The following CMake options may be used to modify build behavior or find
MADNESS:

* ENABLE_MPI -- Enable MPI [Default=ON]
* ENABLE_ELEMENTAL -- Enable use of MADNESS provided Elemental [Default=OFF]
* ENABLE_TBB -- Enable the use of TBB when building MADNESS [Default=ON]
* TBB_ROOT_DIR -- The install directory for TBB
* TBB_INCLUDE_DIR -- The include directory for TBB header files
* TBB_LIBRARY -- The library directory for TBB shared libraries
* BUILD_SHARED_LIBS -- Enable shared MADNESS libraries [Default=ON if supported 
      by the platform]
* MADNESS_SOURCE_DIR -- Path to the MADNESS source directory
* MADNESS_BINARY_DIR -- Path to the MADNESS build directory
* MADNESS_URL -- Path to the MADNESS repository [Default=MADNESS git repository]
* MADNESS_TAG -- Revision hash or tag to use when building MADNESS (expert only)
* MADNESS_CMAKE_EXTRA_ARGS -- Extra flags passed to MADNESS cmake command

If you wish to install MADNESS yourself, we recommend downloading the latest
version from the MADNESS git repository. You should not expect the latest 
release version to work correctly with TiledArray. You can specify the install
directory with:

* MADNESS_ROOT_DIR -- MADNESS install directory
* CMAKE_INSTALL_PREFIX -- Semicolon separated list of directory CMake will use
      to search for software dependencies.

# Advanced configure options:

The following CMake cache variables are tuning parameters. You should only
modify these values if you know the values for your patricular system.

* VECTOR_ALIGNMENT -- The alignment of memory for Tensor in bytes [Default=16]
* CACHE_LINE_SIZE -- The cache line size in bytes [Default=64]

VECTOR_ALIGNMENT controls the alignment of Tensor data, and CACHE_LINE_SIZE
controls the size of automatic loop unrolling for tensor operations. TiledArray
does not currently use explicit vector instructions (i.e. intrensics), but
the code is written in such a way that compilers can more easily autovectorize 
the operations when supported. In a future version, explicit vectorization
support may be added.

# Build TiledArray

    $ make -j
    ... many lines omitted ...
    $ make check
    ... many lines omitted ...
    $ make install