Albany: Building Trilinos

ambrad edited this page Jan 6, 2016 · 6 revisions

Most of the effort of building Albany is in configuring and building Trilinos. An instructional Trilinos configuration script is available at

    Albany/doc/do-cmake-trilinos-instructional

The current version follows. At the end of the script is a discussion of various options to speed up Albany build time.

# Early Nov 2014. Instructional Trilinos configuration script for Tpetra-enabled
#                 Albany.
# 18 Feb 2015.    Now updated for Kokkos-enabled Albany.

# Modify these paths for your system.
TRILINSTALLDIR=../install-basic
BOOSTDIR=/usr/local
NETCDFDIR=/usr/local/parallel
HDF5DIR=/usr/local/parallel
SUPERLUDIR=/usr/local/SuperLU_4.3

# Remove the CMake cache. For an extra clean start in an already-used build
# directory, rm -rf CMake* to get rid of all CMake-generated files.
rm -f CMakeCache.txt;

# The CMake command. I divide it up into blocks using \.
#   Block 1 is basic build info.
#   Block 2 has all the packages for the old Epetra-only Albany.
#   Block 3 has the packages (STKIo and STKMesh) and TPL (Boost) needed for the
# new STK. STKClassic cannot simultaneously be built, as it conflicts with the
# new STK.
#   Block 4 has I/O util packages and TPLs.
#   Blocks 1-4 are for the basic old Epetra-only Albany build. If you already
# have a Trilinos configuration script for that build, you can use it instead of
# blocks 1-4.
#   Block 5 has the packages required by the Tpetra-enabled Albany.
#   Blocks 6-7 are new for the Kokkos-enabled Albany, though the flags have some
# overlap with the old ones.
#   Block 6 provides the types and the explicit template instantiation we need.
#   Block 7 handles Kokkos flags to set up the Serial node, which matches
# pre-Kokkos behavior.
cmake \
 -D Trilinos_DISABLE_ENABLED_FORWARD_DEP_PACKAGES=ON \
 -D CMAKE_INSTALL_PREFIX:PATH=${TRILINSTALLDIR} \
 -D CMAKE_BUILD_TYPE:STRING=RELEASE \
 -D BUILD_SHARED_LIBS:BOOL=ON \
 -D TPL_ENABLE_MPI:BOOL=ON \
 -D CMAKE_VERBOSE_MAKEFILE:BOOL=OFF \
 -D Trilinos_ENABLE_ALL_PACKAGES:BOOL=OFF \
 -D Trilinos_WARNINGS_AS_ERRORS_FLAGS:STRING="" \
 -D Teuchos_ENABLE_LONG_LONG_INT:BOOL=ON \
\
 -D Trilinos_ENABLE_Teuchos:BOOL=ON \
 -D Trilinos_ENABLE_Shards:BOOL=ON \
 -D Trilinos_ENABLE_Sacado:BOOL=ON \
 -D Trilinos_ENABLE_Epetra:BOOL=ON \
 -D Trilinos_ENABLE_EpetraExt:BOOL=ON \
 -D Trilinos_ENABLE_Ifpack:BOOL=ON \
 -D Trilinos_ENABLE_AztecOO:BOOL=ON \
 -D Trilinos_ENABLE_Amesos:BOOL=ON \
 -D Trilinos_ENABLE_Anasazi:BOOL=ON \
 -D Trilinos_ENABLE_Belos:BOOL=ON \
 -D Trilinos_ENABLE_ML:BOOL=ON \
 -D Trilinos_ENABLE_Phalanx:BOOL=ON \
 -D Trilinos_ENABLE_Intrepid:BOOL=ON \
 -D Trilinos_ENABLE_Intrepid2:BOOL=ON \
 -D Trilinos_ENABLE_NOX:BOOL=ON \
 -D Trilinos_ENABLE_Stratimikos:BOOL=ON \
 -D Trilinos_ENABLE_Thyra:BOOL=ON \
 -D Trilinos_ENABLE_Rythmos:BOOL=ON \
 -D Trilinos_ENABLE_MOOCHO:BOOL=ON \
 -D Trilinos_ENABLE_Stokhos:BOOL=ON \
 -D Trilinos_ENABLE_Piro:BOOL=ON \
 -D Trilinos_ENABLE_Teko:BOOL=ON \
\
 -D Trilinos_ENABLE_STKIO:BOOL=ON \
 -D Trilinos_ENABLE_STKMesh:BOOL=ON \
 -D TPL_ENABLE_Boost:BOOL=ON \
 -D Boost_INCLUDE_DIRS:FILEPATH="$BOOSTDIR/include" \
 -D Boost_LIBRARY_DIRS:FILEPATH="$BOOSTDIR/lib" \
 -D TPL_ENABLE_BoostLib:BOOL=ON \
 -D BoostLib_INCLUDE_DIRS:FILEPATH="$BOOSTDIR/include" \
 -D BoostLib_LIBRARY_DIRS:FILEPATH="$BOOSTDIR/lib" \
\
 -D Trilinos_ENABLE_SEACASIoss:BOOL=ON \
 -D Trilinos_ENABLE_SEACASExodus:BOOL=ON \
 -D TPL_ENABLE_Netcdf:BOOL=ON \
 -D Netcdf_INCLUDE_DIRS:PATH="$NETCDFDIR/include" \
 -D Netcdf_LIBRARY_DIRS:PATH="$NETCDFDIR/lib" \
 -D TPL_ENABLE_HDF5:BOOL=ON \
 -D HDF5_INCLUDE_DIRS:PATH="$HDF5DIR/include" \
 -D HDF5_LIBRARY_DIRS:PATH="$HDF5DIR/lib" \
\
 -D Trilinos_ENABLE_Tpetra:BOOL=ON \
 -D Trilinos_ENABLE_Kokkos:BOOL=ON \
 -D Trilinos_ENABLE_Ifpack2:BOOL=ON \
 -D Trilinos_ENABLE_Amesos2:BOOL=ON \
 -D Trilinos_ENABLE_Zoltan2:BOOL=ON \
 -D Trilinos_ENABLE_MueLu:BOOL=ON \
 -D Amesos2_ENABLE_KLU2:BOOL=ON \
\
 -D Trilinos_ENABLE_EXPLICIT_INSTANTIATION:BOOL=ON \
 -D Tpetra_INST_INT_LONG_LONG:BOOL=ON \
 -D Tpetra_INST_INT_INT:BOOL=ON \
 -D Tpetra_INST_DOUBLE:BOOL=ON \
 -D Tpetra_INST_FLOAT:BOOL=OFF \
 -D Tpetra_INST_COMPLEX_FLOAT:BOOL=OFF \
 -D Tpetra_INST_COMPLEX_DOUBLE:BOOL=OFF \
 -D Tpetra_INST_INT_LONG:BOOL=OFF \
 -D Tpetra_INST_INT_UNSIGNED:BOOL=OFF \
\
 -D Trilinos_ENABLE_Kokkos:BOOL=ON \
 -D Trilinos_ENABLE_KokkosCore:BOOL=ON \
 -D Phalanx_KOKKOS_DEVICE_TYPE:STRING="SERIAL" \
 -D Phalanx_INDEX_SIZE_TYPE:STRING="INT" \
 -D Phalanx_SHOW_DEPRECATED_WARNINGS:BOOL=OFF \
 -D Kokkos_ENABLE_Serial:BOOL=ON \
 -D Kokkos_ENABLE_OpenMP:BOOL=OFF \
 -D Kokkos_ENABLE_Pthread:BOOL=OFF \
\
 ../

# MueLu requires a coarse-level solver. One of the following must be provided:
# (1) SuperLU:
#   -D TPL_ENABLE_SuperLU:STRING=ON \
#   -D SuperLU_INCLUDE_DIRS:STRING="${SUPERLUDIR}/include" \
#   -D SuperLU_LIBRARY_DIRS:STRING="${SUPERLUDIR}/lib" \
# or (2) KLU2:
#   -D Amesos2_ENABLE_KLU2:BOOL=ON \
# You can also include both and choose which one to use at run-time.

# For ctests, include these additional SEACAS-related flags to build the
# install/bin utils exodiff, algebra, and epu, as well as others:
#   -D Trilinos_ENABLE_SEACAS:BOOL=ON \
#   -D TPL_ENABLE_Matio:BOOL=OFF \
#   -D TPL_ENABLE_X11:BOOL=OFF \

# Options for increasing Albany s/w dev efficiency:
#
# Add this line to make a shared-library build.
#   -D BUILD_SHARED_LIBS:BOOL=ON \
# There are two advantages to using a shared-lib build: first, link time of
# large executables is essentially gone; second, incremental rebuilds (in a
# debug-modify-build loop) are much faster: on average, perhaps 5x faster, and
# often up to 20x faster. You will have to add -fPIC to the build flags for
# TPLs to build shared libs.
#   Tip: In SuperLU's make.inc, add -fPIC to these three places:
#     CC           = gcc-4.7 -fPIC
#     FFLAGS       = -O3 -fPIC
#     LOADOPTS     = -fPIC
#
# Add these lines to make a debug build.
#   -D CMAKE_BUILD_TYPE:STRING=DEBUG \
#   -D CMAKE_CXX_FLAGS:STRING="-ggdb" \
# I strongly recommend doing only *shared library* debug builds, as the lib and
# exe files for a static debug build can be huge.
#
# Add this line to enable explicit template instantiation (ETI). (ETI is now
# used in this instructional configuration.) This speeds up compiling certain
# files in Albany by a factor of >3.
#   -D Trilinos_ENABLE_EXPLICIT_INSTANTIATION:BOOL=ON \
# This speedup is particularly evident when doing incremental rebuilds during
# development work.
#
# In summary, the optimal set of optional flags when doing development work in
# Albany is
#   -D Trilinos_ENABLE_EXPLICIT_INSTANTIATION:BOOL=ON \
#   -D BUILD_SHARED_LIBS:BOOL=ON \
# In addition, if the dev work involves gdb-level debugging, include these as
# well:
#   -D CMAKE_BUILD_TYPE:STRING=DEBUG \
#   -D CMAKE_CXX_FLAGS:STRING="-ggdb" \

Frequent Issues

Build time. If you find that rebuilding Albany after making a change to a source file is taking a long time (more than 15s to 1 minute, depending on file), there are typically two basic causes: (1) building Trilinos without explicit template instantiation (ETI) and (2) configuring Trilinos to build static instead of shared libraries. ETI compiles many of Trilinos's headers so that the Albany build does not have to. Building Albany against Trilinos shared libraries means that the linker does not have to build a large static executable. The script above shows how to build with ETI and shared libraries. The runtime cost of using shared libraries is on the order of 1 second at program startup.

Changes by large transition

This is a summary of changes to the Trilinos configuration as a function of large software transitions. You don't need to read this summary if you're building Albany well after these transitions have occurred.

(Nov 2014 transition) If you can already build the Epetra-only Albany, add these lines to your Trilinos configuration file to build the Tpetra-enabled Albany:

    -D Trilinos_ENABLE_Tpetra:BOOL=ON \
    -D Trilinos_ENABLE_Kokkos:BOOL=ON \
    -D Trilinos_ENABLE_Ifpack2:BOOL=ON \
    -D Trilinos_ENABLE_Amesos2:BOOL=ON \
    -D Trilinos_ENABLE_Zoltan2:BOOL=ON \
    -D Trilinos_ENABLE_MueLu:BOOL=ON \

(Feb 2015 transition) If you then can build the Tpetra-enabled Albany, add these lines to build the basic Kokkos-enabled Albany:

    -D Trilinos_ENABLE_Kokkos:BOOL=ON \
    -D Trilinos_ENABLE_KokkosCore:BOOL=ON \
    -D Phalanx_KOKKOS_DEVICE_TYPE:STRING="SERIAL" \
    -D Phalanx_INDEX_SIZE_TYPE:STRING="INT" \
    -D Phalanx_SHOW_DEPRECATED_WARNINGS:BOOL=OFF \
    -D Kokkos_ENABLE_Serial:BOOL=ON \
    -D Kokkos_ENABLE_OpenMP:BOOL=OFF \
    -D Kokkos_ENABLE_Pthread:BOOL=OFF \

Optionally, you can enable MueLu by adding either of these:

    -D Trilinos_ENABLE_MueLu:BOOL=ON \
    -D TPL_ENABLE_SuperLU:STRING=ON \
    -D SuperLU_INCLUDE_DIRS:STRING="${SUPERLUDIR}/include" \
    -D SuperLU_LIBRARY_DIRS:STRING="${SUPERLUDIR}/lib" \

or

    -D Trilinos_ENABLE_MueLu:BOOL=ON \
    -D Amesos2_ENABLE_KLU2:BOOL=ON \

The first requires building SuperLU.

If you are developing Kokkos kernels in Albany, add the following flag to your Albany configuration:

    -D ENABLE_KOKKOS_UNDER_DEVELOPMENT \
Clone this wiki locally
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.