Installation

Cian Wilson edited this page Apr 14, 2018 · 33 revisions

TerraFERMA Home | Wiki Home

TerraFERMA has numerous dependencies so can be a little daunting to install. Here we provide instructions for installing manually, using a script and using docker.

Creating an executable using TerraFERMA is a two step process. The instructions below detail the installation of a standalone library and supporting scripts and other files that, together with various dependencies, provide an interface for the user to describe their model. User input is then required before an executable is generated. For more details on this second step see the running TerraFERMA and documentation sections of the wiki.

Docker

Docker is an open-source project that automates the deployment of Linux applications by building them inside software containers that contain everything the application needs to run. For our purposes, these containers can be thought of like virtual machines but have a much reduced overhead (depending on the operating system they run on). We provide several Docker image files describing different versions of TerraFERMA. These run TerraFERMA inside an Ubuntu container and are automatically built on Docker Hub and tested nightly on our buildbot.

To get started with TerraFERMA through Docker:

  1. Install Docker

  2. In a terminal and with an internet connection run the TerraFERMA image:

     docker run -it --rm terraferma/dev
    

Step 2 above will automatically download the latest terraferma/dev image from Docker Hub, run a container based on that image interactively and with a pseudo-tty connection (-it), and place the user in a bash shell with a full installation of TerraFERMA available. Additionally the container will be automatically removed on exiting the shell (--rm). See the Docker documentation for instructions on retaining, naming and reconnecting to containers.

Files can be shared between the user's computer and the container using the -v option. A mount point is provided in the image at /home/tfuser/shared. For example to run the container, sharing the contents of the current directory on the user's computer use:

docker run -it --rm -v $PWD:/home/tfuser/shared terraferma/dev 

In order to open the TerraFERMA GUI diamond from within a Docker container it is also necessary to share an X connection between the host computer and the container. How this is done depends on the operating system of the host computer.

On Linux use:

docker run -it --rm -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix terraferma/dev

On some systems (and for some users) it may also be necessary to run

xhost local:$USER

before the docker run command, or

xhost local:root

if running Docker using sudo.

On a Mac: these instructions assume that

  • You are using Docker for Mac (not Docker toolbox)
  • You have installed XQuartz (v2.7.10 or later)
  • You have set the XQuartz security preferences to "Allow Connections from Network Clients"

Then start XQuartz, find the machine's IP address, and set the xhost

open -a XQuartz
ip=$(ifconfig en0 | grep inet | awk '$1=="inet" {print $2}')
xhost + ${ip}

Run Docker for Mac with the correct display

docker run -it --rm -e DISPLAY=${ip}:0 -v /tmp/.X11_unix:/tmp terraferma/dev

When you are finished running docker, unset the xhost

xhost - ${ip}

Note that above we have dropped the option to share the host machine's current directory with the container but it can easily be included again by using the option -v $PWD:/home/tfuser/shared before the terraferma/dev argument as before. The complete docker run command would then be:

docker run -it --rm -e DISPLAY=$DISPLAY -v /tmp/.X11-unix:/tmp/.X11-unix -v $PWD:/home/tfuser/shared terraferma/dev

on Linux and:

docker run -it --rm -e DISPLAY=${ip}:0 -v /tmp/.X11_unix:/tmp -v $PWD:/home/tfuser/shared terraferma/dev

on a Mac.

In addition to terraferma/dev, we provide several different Docker configurations that can be used in the more generic command format:

docker run -it --rm terraferma/<image name>:<tag name>

Where <image name> and <tag name> should be chosen from the following lists.

<image name> description
dev A fully functional deployment of TerraFERMA and all its required dependencies.
dev-env Only dependencies installed. TerraFERMA not installed. Intended for developers.
base A stripped down Ubuntu container. No dependencies or TerraFERMA installed.

This image naming structure closely follows that used by FEniCS.

<tag name> description buildbot status
latest The latest version of TerraFERMA. Automatically rebuilt after every update. The tag name defaults to this if unspecified (as above). (unknown status)
v1.0 The first release of TerraFERMA. (unknown status)
v0.9 Final pre-release version of TerraFERMA. (unknown status)
dolfin-master Version of TerraFERMA requiring no modifications of DOLFIN/FEniCS. (unknown status)
debug As latest but with debugging compilation symbols enabled. (unknown status)

To keep up with the most up to date version of TerraFERMA through docker it is necessary to update the image periodically:

docker pull terraferma/dev

as the image is only downloaded automatically the first time it is run.

Once running the terraferma/dev Docker image provides the user with access to a fully functional installation of TerraFERMA. In the home directory the image contains symbolic links to the pre-installed test and tutorial suites where various input files can be browsed and run. Alternatively other files can be imported into, shared with or designed in the container. For more details on building models see the running TerraFERMA and documentation sections of the wiki, particularly the cookbook.

Scripted Installation

Dorsal

Dorsal was the installation script for FEniCS before being retired in favor of HashDist. We still maintain our own branch of Dorsal for TerraFERMA (and dependency) installations and use it to test TerraFERMA on our supported platforms. These builds are generally on clean operating systems so some modification of the instructions below may be necessary.

While more platforms are supported by this installation method, Dorsal will install a large number of dependencies from source that may be unecessary on your system. If you want to link to existing packages already available to you, we suggest you read and appropriately modify the manual installation instructions.

Set-Up

Clone our Dorsal branch into a suitable location:

git clone https://bitbucket.org/tferma/dorsal.git
cd dorsal

Then, either modify dorsal.cfg or add your own local.cfg file (below in the main repository directory:

  • local.cfg:
# Meta-project to build
PROJECT=TerraFERMA
# Where do you want the downloaded source files to go?
DOWNLOAD_PATH=<dorsal download path>  # edit here to set the download path
# Where do you want the compiled software installed?
INSTALL_PATH=<dorsal installation path> # edit here to set the installation path
# How many processes would you like to build using?
PROCS=1
# For TerraFERMA the two following variables should be set to false
STABLE_BUILD=false
USE_SNAPSHOTS=false

where you should edit the DOWNLOAD_PATH and INSTALL_PATH to set the location where packages should be first downloaded and second installed to.

Installation

Once the configuration has been set-up, Dorsal can be started using:

./dorsal.sh

Dorsal will attempt to recognise your platform and, if it is supported, automatically load the instructions for it. If it fails but you believe there is a platform configuration file in TerraFERMA/platforms/supported that should work for your system you may also invoke Dorsal using:

./dorsal.sh TerraFERMA/platforms/supported/<platform name>.platform

where the appropriate platform name should be inserted.

Dorsal will then provide instructions for installation on the selected platform. These generally include installing a range of packages or adding various repositories. If you do not have sufficient privileges on your system to follow these instructions, send them (or the relevant platform file), to your system administrator for installation.

Once the dependencies are installed press enter to continue (or restart Dorsal). Packages being installed from source will then be downloaded and installed to the locations you specified above. This step can take a significant amount of time.

Troubleshooting

If a package fails to install a good place to start troubleshooting is in the source directory for that package. This can be found at in the $DOWNLOAD_PATH/<package name> directory. Each package will generate a configure (dorsal_configure.log) and build (dorsal_build.log) log file that will provide extra information on what has gone wrong. For [supported platforms](Supported Platforms) please report problems on the issue tracker.

Final Steps

Environment

After installing TerraFERMA it is necessary to correctly set your environment before using it. Dorsal will produce both an environment configuration file and an environment module file to do this. Depending on your preferences, either can be used.

Using the configuration file simply source it to set your environment:

source terraferma.conf

If you prefer to use environment modules then, assuming you already have the package installed on your system (see manual installation for an example of this on Ubuntu), simply:

module use <dorsal installation path>/share/dorsal

where you should edit the installation path to match the $INSTALL_PATH used above, and:

module load terraferma.configmodule

Testing

Once installed and the environment is set correctly, we recommend running the TerraFERMA test suite to confirm that everything has been configured correctly. To do this, simply run:

run_tftests

This builds, runs and interrogates a series of short tests, mainly consisting of manufactured solution convergence tests and other simple (but fundamental) problems. This may take some time however at the end the number of passes and failures will be reported along with a summary of the names of any failing tests.

To find out more about this testing stage and to see the current status of our buildbots please see the supported platforms page.

Manual Installation

Ubuntu 16.04

These instructions describe the installation of TerraFERMA on a clean Ubuntu 16.04LTS system however they are known to work with minor modifications on other versions as well.
Installation on a range of Ubuntu versions is also possible using the Dorsal script however by default that method installs a lot of dependencies that may duplicate previously installed packages. This guide is therefore intended to be customizable by more experienced users so that such duplication may be avoided.

Ubuntu is our primary development platform and where we have most experience/resources for fixing problems. If you can`t find your preferred operating system in our list of [supported platforms](Supported Platforms] but want to give TerraFERMA a try we recommend installing an Ubuntu virtual machine as a first step (either manually or using Dorsal). The following steps are what we typically follow on our development platforms.

Dependency Installation

TerraFERMA has three major dependencies:

In the course of installing these we also end up installing some of their dependencies:

though fortunately this is made a lot easier by PETSc (see below).

In addition we have some lighter-weight dependencies on, for example, version control software, various python modules and compilers, amongst other things:

Many of these are dependencies of either PETSc or FEniCS rather than TerraFERMA directly. Luckily, in Ubuntu the installation of these is trival (assuming sudo permissions are available):

sudo apt-get install git \
                     gcc g++ gfortran \
                     cmake \
                     openmpi-bin libopenmpi-dev \
                     flex \
                     libblas-dev \
                     liblapack-dev \
                     python \
                     python-numpy \
                     python-scipy \
                     python-matplotlib \
                     python-ply \
                     python-sympy \
                     python-pytest \
                     python-setuptools \
                     swig3.0 \
                     libeigen3-dev \
                     libboost-dev \
                     libboost-filesystem-dev \
                     libboost-program-options-dev \
                     libboost-iostreams-dev \
                     libboost-timer-dev \
                     libboost-chrono-dev \
                     libvtk5-dev python-vtk \
                     libvtk-java libvtk5-qt4-dev \
                     libhdf5-openmpi-dev \
                     gmsh

This leaves only the major dependencies of PETSc, FEniCS and SPuD.

PETSc

Comprehensive installation instructions for PETSc can be found on the PETSc homepage. Here we outline our typical installation procedure.

We maintain our own fork of PETSc at https://bitbucket.org/tferma/petsc. At the time of writing there are no modifications when compared to the main PETSc repository (except that our fork tends to lag behind the main repository). We maintain a separate fork simply as a method of marking the latest version that TerraFERMA is known to compile with, which is typically on the maint branch.

To get our PETSc fork (and preferred branch), simply clone:

git clone https://bitbucket.org/tferma/petsc.git

into a suitable location.

Once downloaded, change into the petsc directory:

cd petsc

and configure with instructions to download and install the majority of our second tier of dependencies above:

export PETSC_PREFIX=<petsc installation path>   # edit here to enter an installation prefix
./configure --with-clanguage=C++ \
            --with-shared-libraries=1 \
            --download-suitesparse=1 \
            --download-hypre=1 \
            --download-ml=1 \
            --download-parmetis=1 \
            --download-metis=1 \
            --download-blacs=1 \
            --download-scalapack=1 \
            --download-mumps=1 \
            --with-debugging=0 \
            --prefix=$PETSC_PREFIX

Note that we typically provide a prefix for installation, which needs modifying above. Following successful configuration, build PETSc using:

make PETSC_DIR=<petsc clone path> PETSC_ARCH=<architecture> all

where, again, the directory and architecture will need modifying. The PETSc configure process will guide you through the correct values here and it is normally easier just to copy the commands they suggest.

Finally, once the build is complete, install PETSc to your installation prefix:

make PETSC_DIR=<petsc clone path> PETSC_ARCH=<architecture> install

We also recommend you run the PETSc tests:

make PETSC_DIR=$PETSC_PREFIX PETSC_ARCH="" test

to double check that the installation was succesful.

FEniCS

FEniCS provides detailed installation instructions on their homepage, including more automated methods through packages like HashDist and Docker. What follows is our typical manual installation method on our Ubuntu development platforms.

We use six components of FEniCS, which we maintain our own forks and branches of:

Unlike with PETSc, these forks do differ from the main repositories (UFL, FIAT, DIJITSO, Instant, FFC, DOLFIN), containing modifications and extra features to allow them to interact with TerraFERMA. At the time of writing these modifcations exist mostly in DOLFIN, with some minor changes to FFC. This varies as features and bug fixes get merged into the main FEniCS repositories.

For a typical production build we use the tferma-master branch of our forks, which is the default branch after cloning into a suitable location:

git clone https://bitbucket.org/tferma/ufl.git
git clone https://bitbucket.org/tferma/fiat.git
git clone https://bitbucket.org/tferma/dijitso.git
git clone https://bitbucket.org/tferma/instant.git
git clone https://bitbucket.org/tferma/ffc.git
git clone https://bitbucket.org/tferma/dolfin.git

The first five components (UFL, FIAT, Dijitso, Instant and FFC) require only a simple python based configuration process:

export FENICS_PREFIX=<fenics installation path>    # edit here to enter an installation prefix

mkdir -p $FENICS_PREFIX/lib/python2.7/site-packages
export PYTHONPATH=$PYTHONPATH:$FENICS_PREFIX/lib/python2.7/site-packages

cd ufl
python setup.py install --prefix=$FENICS_PREFIX
cd ../

cd fiat
python setup.py install --prefix=$FENICS_PREFIX
cd ../

cd dijitso
python setup.py install --prefix=$FENICS_PREFIX
cd ../

cd instant
python setup.py install --prefix=$FENICS_PREFIX
cd ../

cd ffc
python setup.py install --prefix=$FENICS_PREFIX
cd ../

where the installation path should be modified above.

DOLFIN uses CMake for configuration, which allows out-of-tree builds:

cd dolfin
mkdir build
cd build

In addition we need to set some environment variables so that we can find PETSc and the other FEniCS components:

export PETSC_DIR=$PETSC_PREFIX
export PATH=$FENICS_PREFIX/bin:$PATH

Rather than setting environment variables manually like above, we generally use environment modules to more easily manage these variables (see below) but for simplicity stick to the manual method in this explanation. Once the environment is set by your preferred method, DOLFIN can be configured using:

cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo \
      -DDOLFIN_ENABLE_CHOLMOD:BOOL=OFF \
      -DDOLFIN_ENABLE_UMFPACK:BOOL=OFF \
      -DDOLFIN_ENABLE_TRILINOS:BOOL=OFF \
      -DDOLFIN_ENABLE_SLEPC:BOOL=OFF  \
      -DMETIS_LIBRARY=${PETSC_DIR}/lib/libmetis.so \
      -DPARMETIS_INCLUDE_DIRS=${PETSC_DIR}/include \
      -DPARMETIS_LIBRARY=${PETSC_DIR}/lib/libparmetis.so \
      -DDOLFIN_ENABLE_TESTING:BOOL=ON \
      -DCMAKE_INSTALL_PREFIX=${FENICS_PREFIX} \
      ../.

Once configured, DOLFIN can be built in the build directory:

make

And then installed to the selected installation prefix:

make install

Thereby completing the installation of FEniCS. Setting the following environment variables will allow your installation to be found and used by TerraFERMA:

export LD_LIBRARY_PATH=$FENICS_PREFIX/lib:$LD_LIBRARY_PATH
export PKG_CONFIG_PATH=$FENICS_PREFIX/lib/pkgconfig:$PKG_CONFIG_PATH

SPuD

Detailed installation instructions for SPuD are available on its homepage. On Ubuntu, it can be most easily installed via the available packages.

First, tell apt about the repositories and update the package lists:

sudo apt-add-repository ppa:fluidity-core/ppa
sudo apt-get update

Then, install the spud components:

sudo apt-get install libspud-dev diamond python-spud spudtools

Alternatively, SPuD can be installed from source. First, checkout the source into a suitable location:

git clone git@github.com:FluidityProject/spud.git

The spud source is now ready for configuration, building and installation:

cd spud
export SPUD_PREFIX=<spud installation path>   # edit here to enter an installation prefix
./configure --prefix=$SPUD_PREFIX
make
make install

If installing from source you will need to set:

export SPUD_DIR=$SPUD_PREFIX

in order for TerraFERMA to find the installation in the next step. Additionally, when running TerraFERMA you will need to have set:

export PATH=$SPUD_PREFIX/bin:$PATH
export LD_LIBRARY_PATH=$SPUD_PREFIX/lib:$LD_LIBRARY_PATH
export PYTHONPATH=$SPUD_PREFIX/lib/python2.7/site-packages:$PYTHONPATH

TerraFERMA Installation

Once all the dependencies are installed, we can finally begin building the library component of TerraFERMA. First clone it into a suitable location:

git clone https://github.com/TerraFERMA/TerraFERMA.git

Like FEniCS, TerraFERMA uses CMake so allows out-of-tree builds:

cd TerraFERMA
mkdir build
cd build
export TF_PREFIX=<terraferma installation path>  # edit here to enter an installation prefix
cmake -DCMAKE_INSTALL_PREFIX=${TF_PREFIX} \
      ../.

where a suitable installation path needs to be specified. Building and installing then simply involves:

make
make install

Before using TerraFERMA you need to set some additional environment variables:

export PATH=$TF_PREFIX/bin:$PATH
export PYTHONPATH=$TF_PREFIX/lib/python2.7/site-packages:$PYTHONPATH
export TF_CMAKE_PATH=$TF_PREFIX/share/terraferma/cpp
export CMAKE_MODULE_PATH=$TF_PREFIX/share/buckettools/cmake/modules
export DIAMOND_CONFIG_PATH=$TF_PREFIX/share/buckettools/diamond:$DIAMOND_CONFIG_PATH

Final Steps

Testing

Once installed and the environment is set correctly, we recommend running the TerraFERMA test suite to confirm that everything has been configured correctly. To do this, in the build directory simply run:

make run_shorttests

This builds, runs and interrogates a series of short tests, mainly consisting of manufactured solution convergence tests and other simple (but fundamental) problems. This may take some time however at the end the number of passes and failures will be reported along with a summary of the names of any failing tests.

To find out more about this testing stage and to see the current status of our buildbots please see the supported platforms page.

Environment

Setting all the environment variables above can be annoying, especially as many of them will need to be set every time TerraFERMA is run in a new shell. To make this easier both DOLFIN and TerraFERMA produce configuration files that can be sourced (manually or from your shell configuration file) to correctly set the environment. Details about the location of these files are printed to screen after installation of each package.

In addition to sourcing these files, you should set:

export PETSC_DIR=$PETSC_PREFIX

and, if you installed SPuD from source:

export SPUD_DIR=$SPUD_PREFIX
export PATH=$SPUD_PREFIX/bin:$PATH
export LD_LIBRARY_PATH=$SPUD_PREFIX/lib:$LD_LIBRARY_PATH
export PYTHONPATH=$SPUD_PREFIX/lib/python2.7/site-packages:$PYTHONPATH

This can still be inconvenient if you wish to change environments. To improve this situation we generally use environment modules to manage our shell environment.

Environment modules can be installed on Ubuntu using apt:

sudo apt-get install environment-modules

after which you should open a new terminal.

You then need module files that correctly configure the environment. We split these up by pacakge so that we have at least three files:

  • petsc:
#%Module1.0#####################################################################
proc ModulesHelp { } {
        puts stderr "\tSets environment for PETSc maint (TerraFERMA installation)"
}
module-whatis "PETSc maint (TerraFERMA)"

set petsc_prefix <petsc installation path> # edit here to enter the installation prefix

prepend-path    PATH            $petsc_prefix/bin
prepend-path    LD_LIBRARY_PATH $petsc_prefix/lib
prepend-path    PYTHONPATH      $petsc_prefix/lib/python2.7/site-packages
setenv          PETSC_DIR       $petsc_prefix
  • fenics:
#%Module1.0
proc ModulesHelp { } {
        puts stderr "\tSets environment for FEniCS tferma-master branch with PETSc maint (TerraFERMA installtion)"
}
module-whatis "FEniCS tferma-master (TerraFERMA)"

set fenics_prefix <fenics installation path> # edit here to enter the installation prefix

prepend-path LD_LIBRARY_PATH $fenics_prefix/lib
prepend-path PATH            $fenics_prefix/bin
prepend-path PKG_CONFIG_PATH $fenics_prefix/lib/pkgconfig
prepend-path PYTHONPATH      $fenics_prefix/lib/python2.7/site-packages
prepend-path MANPATH         $fenics_prefix/share/man

module load petsc
  • terraferma:
#%Module1.0
proc ModulesHelp { } {
        puts stderr "Sets environment for TerraFERMA"
}
module-whatis "TerraFERMA"

set tf_prefix <terraferma installation path> # edit here to enter the installation prefix

setenv       TF_CMAKE_PATH         $tf_prefix/share/terraferma/cpp
prepend-path CMAKE_MODULE_PATH     $tf_prefix/share/buckettools/cmake/modules
prepend-path PYTHONPATH            $tf_prefix/lib/python2.7/site-packages
prepend-path DIAMOND_CONFIG_PATH   $tf_prefix/share/buckettools/diamond
prepend-path PATH                  $tf_prefix/bin

module load fenics
#module load spud # uncomment if SPuD was installed from source

Note that a module file for TerraFERMA is also produced automatically during the installation process. See the notes printed to screen after installation for details of its location.

  • spud (if SPuD was installed from source):
#%Module1.0
proc ModulesHelp { } {
        puts stderr "\tSets environment for SPuD (TerraFERMA installtion)"
}
module-whatis "SPuD (TerraFERMA)"

set spud_prefix <spud installation path> # edit here to enter the installation prefix

prepend-path LD_LIBRARY_PATH $spud_prefix/lib
prepend-path PATH            $spud_prefix/bin
prepend-path PYTHONPATH      $spud_prefix/lib/python2.7/site-packages
setenv       SPUD_DIR        $spud_prefix

The above files should be editted (to set the correct installation prefixes and, optionally, to include spud if it was installed from source) and placed in a suitable location (modulefile installation path). They can then be used to set the environment using:

module use <modulefile installation path> # edit here to set installation path
module load terraferma

This can be very useful for running TerraFERMA but can also be used during installation. Loading this module file after the installation of petsc means that all subsequent commands setting the environment detailed above can be skipped during the installation process.

Mac OSX 10.13 (High Sierra)

These instructions describe the installation of TerraFERMA on a clean Max OSX 10.13 High Sierra system using the homebrew package management system. Installation on a range of OSX versions is also possible using the Dorsal script however by default that method installs a lot of dependencies that may duplicate previously installed packages. This guide is therefore intended to be customizable by more experienced users so that such duplication may be avoided.

Dependency Installation

TerraFERMA has three major dependencies:

In the course of installing these we also end up installing some of their dependencies:

though fortunately this is made a lot easier by PETSc (see below).

In addition we have some lighter-weight dependencies on, for example, version control software, various python modules and compilers, amongst other things:

Many of these are dependencies of either PETSc or FEniCS rather than TerraFERMA directly. First, if not already available, install the Homebrew package management system:

/usr/bin/ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Please see the homebrew homepage for more information. Note that you should be careful doing this if you already use an alternative package manage system (e.g. fink or macports) as conflicts may arise.

Once homebrew is installed we can install many of our other dependencies:

brew install coreutils python wget autogen ginac cppunit eigen boost \
             swig cmake vtk openmpi
brew install --with-libglade pygtk
brew install --with-fltk gmsh
brew install brew-pip
pip2 install ply
pip2 install ipython 
sudo pip2 install numpy matplotlib
pip2 install scipy ply py lxml readline sympy 
brew cask install java
brew cask install caskroom/versions/java8
brew install jing-trang

This leaves only the major dependencies of PETSc, FEniCS and SPuD.

PETSc

Comprehensive installation instructions for PETSc can be found on the PETSc homepage. Here we outline our typical installation procedure.

We maintain our own fork of PETSc at https://bitbucket.org/tferma/petsc. At the time of writing there are no modifications when compared to the main PETSc repository (except that our fork tends to lag behind the main repository). We maintain a separate fork simply as a method of marking the latest version that TerraFERMA is known to compile with, which is typically on the maint branch.

To get our PETSc fork (and preferred branch), simply clone:

git clone https://bitbucket.org/tferma/petsc.git

into a suitable location.

Once downloaded, change into the petsc directory:

cd petsc

and configure with instructions to download and install the majority of our second tier of dependencies above:

export PETSC_PREFIX=<petsc installation path>   # edit here to enter an installation prefix
./configure --with-clanguage=C++ \
            --with-shared-libraries=1 \
            --download-suitesparse=1 \
            --download-hypre=1 \
            --download-ml=1 \
            --download-parmetis=1 \
            --download-metis=1 \
            --download-blacs=1 \
            --download-scalapack=1 \
            --download-mumps=1 \
            --with-debugging=0 \
            --prefix=$PETSC_PREFIX

Note that we typically provide a prefix for installation, which needs modifying above. Following successful configuration, build PETSc using:

make PETSC_DIR=<petsc clone path> PETSC_ARCH=<architecture> all

where, again, the directory and architecture will need modifying. The PETSc configure process will guide you through the correct values here and it is normally easier just to copy the commands they suggest.

Finally, once the build is complete, install PETSc to your installation prefix:

make PETSC_DIR=<petsc clone path> PETSC_ARCH=<architecture> install

We also recommend you run the PETSc tests:

make PETSC_DIR=$PETSC_PREFIX PETSC_ARCH="" test

to double check that the installation was succesful.

FEniCS

FEniCS provides detailed installation instructions on their homepage, including more automated methods through packages like HashDist and Docker. What follows is our typical manual installation method.

We use six components of FEniCS, which we maintain our own forks and branches of:

Unlike with PETSc, these forks do differ from the main repositories (UFL, FIAT, DIJITSO, Instant, FFC, DOLFIN), containing modifications and extra features to allow them to interact with TerraFERMA. At the time of writing these modifcations exist mostly in DOLFIN, with some minor changes to FFC. This varies as features and bug fixes get merged into the main FEniCS repositories.

For a typical production build we use the tferma-master branch of our forks, which is the default branch after cloning into a suitable location:

git clone https://bitbucket.org/tferma/ufl.git
git clone https://bitbucket.org/tferma/fiat.git
git clone https://bitbucket.org/tferma/dijitso.git
git clone https://bitbucket.org/tferma/instant.git
git clone https://bitbucket.org/tferma/ffc.git
git clone https://bitbucket.org/tferma/dolfin.git

The first five components (UFL, FIAT, Dijitso, Instant and FFC) require only a simple python based configuration process:

export FENICS_PREFIX=<fenics installation path>    # edit here to enter an installation prefix

mkdir -p $FENICS_PREFIX/lib/python2.7/site-packages
export PYTHONPATH=$PYTHONPATH:$FENICS_PREFIX/lib/python2.7/site-packages

cd ufl
python setup.py install --prefix=$FENICS_PREFIX
cd ../

cd fiat
python setup.py install --prefix=$FENICS_PREFIX
cd ../

cd dijitso
python setup.py install --prefix=$FENICS_PREFIX
cd ../

cd instant
python setup.py install --prefix=$FENICS_PREFIX
cd ../

cd ffc
python setup.py install --prefix=$FENICS_PREFIX
cd ../

where the installation path should be modified above.

DOLFIN uses CMake for configuration, which allows out-of-tree builds:

cd dolfin
mkdir build
cd build

In addition we need to set some environment variables so that we can find PETSc and the other FEniCS components:

export PETSC_DIR=$PETSC_PREFIX
export PATH=$FENICS_PREFIX/bin:$PATH

Rather than setting environment variables manually like above, we generally use environment modules to more easily manage these variables (see below) but for simplicity stick to the manual method in this explanation. Once the environment is set by your preferred method, DOLFIN can be configured using:

cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo \
      -DDOLFIN_ENABLE_CHOLMOD:BOOL=OFF \
      -DDOLFIN_ENABLE_UMFPACK:BOOL=OFF \
      -DDOLFIN_ENABLE_TRILINOS:BOOL=OFF \
      -DDOLFIN_ENABLE_SLEPC:BOOL=OFF  \
      -DMETIS_LIBRARY=${PETSC_DIR}/lib/libmetis.dylib \
      -DPARMETIS_INCLUDE_DIRS=${PETSC_DIR}/include \
      -DPARMETIS_LIBRARY=${PETSC_DIR}/lib/libparmetis.dylib \
      -DDOLFIN_ENABLE_TESTING:BOOL=ON \
      -DCMAKE_INSTALL_PREFIX=${FENICS_PREFIX} \
      ../.

Once configured, DOLFIN can be built in the build directory:

make

And then installed to the selected installation prefix:

make install

Thereby completing the installation of FEniCS. Setting the following environment variables will allow your installation to be found and used by TerraFERMA:

export PKG_CONFIG_PATH=$FENICS_PREFIX/lib/pkgconfig:$PKG_CONFIG_PATH

SPuD

Detailed installation instructions for SPuD are available on its homepage.

First, checkout the source into a suitable location:

git clone git@github.com:FluidityProject/spud.git

The spud source is now ready for configuration, building and installation:

cd spud
export SPUD_PREFIX=<spud installation path>   # edit here to enter an installation prefix
./configure --prefix=$SPUD_PREFIX
make
make install

You will need to set:

export SPUD_DIR=$SPUD_PREFIX

in order for TerraFERMA to find the installation in the next step. Additionally, when running TerraFERMA you will need to have set:

export PATH=$SPUD_PREFIX/bin:$PATH
export PYTHONPATH=$SPUD_PREFIX/lib/python2.7/site-packages:$PYTHONPATH

TerraFERMA Installation

Once all the dependencies are installed, we can finally begin building the library component of TerraFERMA. First clone it into a suitable location:

git clone -b highsierratest https://github.com/TerraFERMA/TerraFERMA.git

Like FEniCS, TerraFERMA uses CMake so allows out-of-tree builds:

cd TerraFERMA
mkdir build
cd build
export TF_PREFIX=<terraferma installation path>  # edit here to enter an installation prefix
cmake -DCMAKE_INSTALL_PREFIX=${TF_PREFIX} \
      ../.

where a suitable installation path needs to be specified. Building and installing then simply involves:

make
make install

Before using TerraFERMA you need to set some additional environment variables:

export PATH=$TF_PREFIX/bin:$PATH
export PYTHONPATH=$TF_PREFIX/lib/python2.7/site-packages:$PYTHONPATH
export TF_CMAKE_PATH=$TF_PREFIX/share/terraferma/cpp
export CMAKE_MODULE_PATH=$TF_PREFIX/share/buckettools/cmake/modules
export DIAMOND_CONFIG_PATH=$TF_PREFIX/share/buckettools/diamond:$DIAMOND_CONFIG_PATH

Final Steps

Testing

Once installed and the environment is set correctly, we recommend running the TerraFERMA test suite to confirm that everything has been configured correctly. To do this, in the build directory simply run:

make run_shorttests

This builds, runs and interrogates a series of short tests, mainly consisting of manufactured solution convergence tests and other simple (but fundamental) problems. This may take some time however at the end the number of passes and failures will be reported along with a summary of the names of any failing tests.

To find out more about this testing stage and to see the current status of our buildbots please see the supported platforms page.

Environment

Setting all the environment variables above can be annoying, especially as many of them will need to be set every time TerraFERMA is run in a new shell. To make this easier both DOLFIN and TerraFERMA produce configuration files that can be sourced (manually or from your shell configuration file) to correctly set the environment. Details about the location of these files are printed to screen after installation of each package.

In addition to sourcing these files, you should set:

export PETSC_DIR=$PETSC_PREFIX
export SPUD_DIR=$SPUD_PREFIX
export PATH=$SPUD_PREFIX/bin:$PATH
export PYTHONPATH=$SPUD_PREFIX/lib/python2.7/site-packages:$PYTHONPATH

This can still be inconvenient if you wish to change environments. To improve this situation we generally use environment modules to manage our shell environment.

Environment modules can be installed using brew with:

brew install modules

after which they need to be initialized using:

. /usr/local/opt/modules/init/bash

It is normally convenient to add this line to your shell initialization script (e.g. ~/.bashrc for bash shells).

You then need module files that correctly configure the environment. We split these up by pacakge so that we have four files:

  • petsc:
#%Module1.0#####################################################################
proc ModulesHelp { } {
        puts stderr "\tSets environment for PETSc maint (TerraFERMA installation)"
}
module-whatis "PETSc maint (TerraFERMA)"

set petsc_prefix <petsc installation path> # edit here to enter the installation prefix

prepend-path    PATH            $petsc_prefix/bin
prepend-path    PYTHONPATH      $petsc_prefix/lib/python2.7/site-packages
setenv          PETSC_DIR       $petsc_prefix
  • fenics:
#%Module1.0
proc ModulesHelp { } {
        puts stderr "\tSets environment for FEniCS tferma-master branch with PETSc maint (TerraFERMA installtion)"
}
module-whatis "FEniCS tferma-master (TerraFERMA)"

set fenics_prefix <fenics installation path> # edit here to enter the installation prefix

prepend-path PATH            $fenics_prefix/bin
prepend-path PKG_CONFIG_PATH $fenics_prefix/lib/pkgconfig
prepend-path PYTHONPATH      $fenics_prefix/lib/python2.7/site-packages
prepend-path MANPATH         $fenics_prefix/share/man

module load petsc
  • terraferma:
#%Module1.0
proc ModulesHelp { } {
        puts stderr "Sets environment for TerraFERMA"
}
module-whatis "TerraFERMA"

set tf_prefix <terraferma installation path> # edit here to enter the installation prefix

setenv       TF_CMAKE_PATH         $tf_prefix/share/terraferma/cpp
prepend-path CMAKE_MODULE_PATH     $tf_prefix/share/buckettools/cmake/modules
prepend-path PYTHONPATH            $tf_prefix/lib/python2.7/site-packages
prepend-path DIAMOND_CONFIG_PATH   $tf_prefix/share/buckettools/diamond
prepend-path PATH                  $tf_prefix/bin

module load fenics
module load spud

Note that a module file for TerraFERMA is also produced automatically during the installation process. See the notes printed to screen after installation for details of its location.

  • spud:
#%Module1.0
proc ModulesHelp { } {
        puts stderr "\tSets environment for SPuD (TerraFERMA installtion)"
}
module-whatis "SPuD (TerraFERMA)"

set spud_prefix <spud installation path> # edit here to enter the installation prefix

prepend-path PATH            $spud_prefix/bin
prepend-path PYTHONPATH      $spud_prefix/lib/python2.7/site-packages
setenv       SPUD_DIR        $spud_prefix

The above files should be editted (to set the correct installation prefixes and placed in a suitable location (modulefile installation path). They can then be used to set the environment using:

module use <modulefile installation path> # edit here to set installation path
module load terraferma

This can be very useful for running TerraFERMA but can also be used during installation. Loading this module file after the installation of petsc means that all subsequent commands setting the environment detailed above can be skipped during the installation process.

You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.
Press h to open a hovercard with more details.