Skip to content

Commit

Permalink
Add full conda build to install instructions
Browse files Browse the repository at this point in the history
  • Loading branch information
kburns committed Aug 8, 2022
1 parent 564a742 commit 8c80451
Show file tree
Hide file tree
Showing 2 changed files with 63 additions and 246 deletions.
228 changes: 0 additions & 228 deletions docs/machines/mac_os/mac_os.rst

This file was deleted.

81 changes: 63 additions & 18 deletions docs/pages/installation.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,64 @@ Documentation for v2 (the latest on PyPI) can be accessed through the sidebar.**

Dedalus is a Python 3 package that includes custom C-extensions (compiled with Cython) and that relies on MPI, FFTW, HDF5, and a basic scientific-Python stack (numpy, scipy, mpi4py, and h5py).

We recommend using conda to build a Python environment with all the necessary prerequisites, as described in the conda instructions below.
This procedure can be easily customized to link to existing MPI/FFTW/HDF5 libraries, which may be preferable when installing Dedalus on a cluster.
The easiest way to install Dedalus and its dependencies is using conda.
The instructions below include options for building the full stack from conda or custom stacks linking to existing MPI/FFTW/HDF5 libraries, which may be preferable when installing on a cluster.
Once you have the necessary C and Python dependencies, you can install and upgrade Dedalus using pip.

Once you have the necessary C dependencies (MPI, FFTW, and HDF5) and Python 3 environment (with properly linked mpi4py and h5py, in particular), you should be able to install Dedalus using pip.

Full-stack conda installation (recommended)
===========================================

Conda installation (recommended)
================================
A full-stack installation of Dedalus v2 is available via conda-forge for Linux and macOS.
This can be used to quickly install all dependencies before installing Dedalus3 via pip.
This installation route is recommended for laptops, workstations, and some cluster environments.
If linking to the existing MPI libraries on your cluster is recommended, see the alternative "Custom conda installation" instructions below.

We recommend installing Dedalus via a conda script that will create a new conda environment with a complete Dedalus installation.
The script allows you to link against custom MPI/FFTW/HDF5 libraries or to install builds of those packages from conda.
First, install conda on your system if you don't already have it (we recommend the `miniforge variant <https://github.com/conda-forge/miniforge/#download>`_).
Then create a new environment for your Dedalus installation (here called ``dedalus3``, but you can pick any name) and activate it::

First, install conda/miniconda for your system if you don't already have it, following the `instructions from conda <https://conda.io/en/latest/miniconda.html>`_.
# Create and activate environment
conda create -n dedalus3
conda activate dedalus3

**Note**: there are currently upstream issues in scipy that prevent Dedalus from natively running on arm64 at this time.
If you are using a Mac with an Apple Silicon processor, you will need to do an extra step to instruct the environment to use x86 packages::

# Macs with Apple Silicon only!
conda config --env --set subdir osx-64

We strongly recommend disabling threading when running Dedalus for maximum performance on all platforms.
The conda environment can be configured to do this automatically with the following commands::

# Disable threading in the environment (strongly recommended for performance)
conda env config vars set OMP_NUM_THREADS=1
conda env config vars set NUMEXPR_MAX_THREADS=1

You can then install Dedalus v2 and all its requirements from the conda-forge channel and upgrade to Dedalus v3 using pip::

# Install Dedalus v2 from conda-forge to build stack
conda install -c conda-forge dedalus

# Upgrade to Dedalus v3
pip uninstall -y dedalus
CC=mpicc pip3 install --no-cache http://github.com/dedalusproject/dedalus/zipball/master/

To use Dedalus, you simply need to activate the new environment.
You can test the installation using the command-line interface::

conda activate dedalus3
python3 -m dedalus test

The Dedalus package within the environment can be updated using pip as described below.


Custom conda installation
=========================

Alternatively, you can use a build script that will create a custom conda environment with all dependencies and then install Dedalus via pip.
This script allows you to optionally link against custom MPI/FFTW/HDF5 libraries, which may provide better performance on some clusters.

First, install conda on your system if you don't already have it (we recommend the `miniforge variant <https://github.com/conda-forge/miniforge/#download>`_).
Then download the Dedalus v3 conda installation script from `this link <https://raw.githubusercontent.com/DedalusProject/dedalus_conda/master/conda_install_dedalus3.sh>`_ or using::

curl https://raw.githubusercontent.com/DedalusProject/dedalus_conda/master/conda_install_dedalus3.sh --output conda_install_dedalus3.sh
Expand All @@ -30,7 +75,8 @@ Then activate the base conda environment and run the script to build a new conda
conda activate base
bash conda_install_dedalus3.sh

To use Dedalus, you simply need to activate the new environment. You can test the installation works using the command-line interface::
To use Dedalus, you simply need to activate the new environment.
You can test the installation using the command-line interface::

conda activate dedalus3
python3 -m dedalus test
Expand All @@ -48,7 +94,7 @@ You can check this by making sure that ``which pip3`` and ``which python3`` resi
If not, use ``python3 -m pip`` instead of ``pip3`` in the following commands.

**Note**: it is strongly recommended that you disable threading, as described on the :doc:`performance_tips` page, when running Dedalus.
This is done automatically when Dedalus is installed using the conda procedure above, but must be done manually otherwise.
This is done automatically when Dedalus is installed using the conda build script described above, but must be done manually otherwise.

Installing from PyPI
--------------------
Expand All @@ -66,33 +112,33 @@ Please build directly from source as described below.**
Building from source
--------------------

To build and install the most recent version of Dedalus, first set the ``MPI_PATH`` and ``FFTW_PATH`` environment variables to the prefix paths for MPI and FFTW::
To build and install the most recent version of Dedalus v3, first set the ``MPI_PATH`` and ``FFTW_PATH`` environment variables to your prefix paths for MPI and FFTW::

export MPI_PATH=/path/to/your/mpi_prefix
export FFTW_PATH=/path/to/your/fftw_prefix

You can then install Dedalus directly from GitHub using pip::
You can then install Dedalus directly from GitHub using pip, ensuring that the C extensions are properly linked to MPI by using ``mpicc``::

pip3 install --no-cache http://github.com/dedalusproject/dedalus/zipball/master/
CC=mpicc pip3 install --no-cache http://github.com/dedalusproject/dedalus/zipball/master/

Alternatively, you can clone the master branch from the source repository and install::
Alternatively, you can clone the master branch from the source repository and install locally using pip::

git clone -b master https://github.com/DedalusProject/dedalus
cd dedalus
pip3 install --no-cache .
CC=mpicc pip3 install --no-cache .

Updating Dedalus
----------------

If Dedalus was installed using the conda script or from GitHub with pip, it can also be updated using pip::

pip3 install --upgrade --force-reinstall --no-deps --no-cache http://github.com/dedalusproject/dedalus/zipball/master/
CC=mpicc pip3 install --upgrade --force-reinstall --no-deps --no-cache http://github.com/dedalusproject/dedalus/zipball/master/

If Dedalus was built from a clone of the source repository, first pull new changes and then reinstall with pip::

cd /path/to/dedalus/repo
git pull
pip3 install --upgrade --force-reinstall --no-deps --no-cache .
CC=mpicc pip3 install --upgrade --force-reinstall --no-deps --no-cache .

**Note**: any custom FFTW/MPI paths set in the conda script or during the original installation will also need to be exported for the upgrade commands to work.

Expand All @@ -118,7 +164,6 @@ These alternative procedures may be out-of-date and are provided for historical
/machines/computecanada/computecanada
/machines/engaging/engaging
/machines/janus/janus
/machines/mac_os/mac_os
/machines/nasa_discover/discover
/machines/nasa_pleiades/pleiades
/machines/savio/savio
Expand Down

0 comments on commit 8c80451

Please sign in to comment.