NERSC recommends the use of anaconda environments to manage python installs, see Brief introduction to Python at NERSC.
There is a conda environment already constructed and maintained for the atom project created using the shareable environment method. You can activate it and run IPS by:
module load python
source activate /global/common/software/atom/cori/ips-framework-new
ips.py --config=simulation.config --platform=platform.conf
This guide will go through creating a conda environment on NERSC and installing the IPS Framework using Option 2: Module + source activate
First, you need to load the python module, then create and activate a new conda environment. This will create the conda environment in your home directory ($HOME/.conda/envs
):
module load python
conda create --name my_ips_env python=3.8 # or any version of python >=3.6
source activate my_ips_env
If you would like the same packages and versions in your conda environment as found in the python modules on Cori, you can clone that environment. In this case using python/3.7-anaconda-2019.10
.
module load python/3.7-anaconda-2019.10
conda create -n my_ips_env --clone base
source activate my_ips_env
Next, install IPS-Framework into the conda environment
python -m pip install ipsframework
To leave your environment
conda deactivate
The example below shows how to select the newly create conda environment in a batch script, see Running Python in a batch job
#!/bin/bash
#SBATCH --constraint=haswell
#SBATCH --nodes=1
#SBATCH --time=5
module load python
source activate my_ips_env
ips.py --config=simulation.config --platform=platform.conf
By default when you create a conda environment it will be created in $HOME/.conda/envs
, to create one elsewhere that can be used by others you can use the --prefix
option, see Creating conda environments.
In this example we are cloning the conda environment from the python/3.7-anaconda-2019.10
module and install ipsframework
.
module load python/3.7-anaconda-2019.10
conda create --prefix /global/common/software/myproject/env --clone base
source activate /global/common/software/myproject/env
python -m pip install ipsframework
The example below shows how to select the newly create conda environment in you batch script, see Running Python in a batch job
#!/bin/bash
#SBATCH --constraint=haswell
#SBATCH --nodes=1
#SBATCH --time=5
module load python
source activate /global/common/software/myproject/env
ips.py --config=simulation.config --platform=platform.conf
To see which packages are currently install in your environment run:
conda list
You can install any other dependencies you need by
conda install numpy matplotlib netcdf4 ...
You should keep your development environment separate from the production environment. If you do development in your my_ips_env
conda environment you can switch between that and the production environment on the atom project by
# switch to production environment
source activate /global/common/software/atom/cori/ips-framework-new
# switch bask to user development environment
source activate my_ips_env
Your bash prompt should be updated to reflect which environment you have active.