Skip to content

Commit

Permalink
Change to install conda via Miniforge
Browse files Browse the repository at this point in the history
  • Loading branch information
@douglatornell
douglatornell committed Jan 6, 2022
1 parent 7b797c8 commit 0da300e
Show file tree
Hide file tree
Showing 2 changed files with 96 additions and 69 deletions.
153 changes: 90 additions & 63 deletions conda_pkg_env_mgr.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -12,55 +12,60 @@
Conda Package and Environment Manager
*************************************

We use `conda`_ software packages,
We use `Conda`_ software packages,
dependencies,
and environments.
:program:`conda` is part of the excellent `Anaconda data science toolkit`_,
but we have found that using :program:`conda` and explicitly managing software environments is more robust over time than using Anaconda.
:program:`conda` was developed as part of the excellent `Anaconda data science toolkit`_,
but we have found that using :program:`conda` and explicitly managing software environments
is more robust over time than using Anaconda.

.. _conda: https://docs.conda.io/en/latest/
.. _Conda: https://docs.conda.io/en/latest/
.. _Anaconda data science toolkit: https://www.anaconda.com/products/individual


TODO: Write an intro to conda


.. _InstallingMiniconda:
.. _InstallingMiniforge:

Installing Miniconda
Installing Miniforge
====================

.. important::
*Do not* install Miniconda or Anaconda on the Compute Canada HPC clusters or other HPC systems!
*Do not* install Miniforge,
Miniconda,
or Anaconda on the Compute Canada HPC clusters or other HPC systems!
It will almost certainly conflict with the HPC-optimized software environments on those system.
If you need to install Python packages on an HPC system,
please contact Doug for advice.

`Miniconda`_ is a minimal installer for conda.
`Miniforge`_ is a minimal installer for `Conda`_ specific to `conda-forge`_.
It installs the :program:`conda` package and environment manager tool,
a recent version of Python, the packages that :program:`conda` depends on,
and a small number of other packages that are essential for creating and managing software environments.

.. _Miniconda: https://docs.conda.io/en/latest/miniconda.html
.. _Miniforge: https://github.com/conda-forge/miniforge
.. _conda-forge: https://conda-forge.org/

The installers for Miniconda are linked on the `Miniconda`_ page.
The installers for Miniforge are linked on the `Miniforge`_ page.
You can download them from there,
and that is what you should do if you are working on Windows.
But for Linux and MacOS you'll be working with :program:`conda` on the command-line in terminals sessions,
so you might as well get started that way by using :program:`wget` to download the installer script into your home directory.

On Linux use:

.. code-block:: bash
$ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
$ wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-Linux-x86_64.sh
On MacOS use:

.. code-block:: bash
$ wget https://repo.anaconda.com/miniconda/Miniconda3-latest-MacOSX-x86_64.sh
$ wget https://github.com/conda-forge/miniforge/releases/latest/download/Miniforge3-MacOSX-x86_64.sh
The installation instructions are linked from https://conda.io/projects/conda/en/latest/user-guide/install/index.html#regular-installation.
The installation instructions are linked from https://github.com/conda-forge/miniforge#unix-like-platforms.
In short:

#. Run the installer script via :program:`bash`:
Expand All @@ -69,13 +74,13 @@ In short:

.. code-block:: bash
$ bash Miniconda3-latest-Linux-x86_64.sh
$ bash Miniforge3-Linux-x86_64.sh
On MacOS use:

.. code-block:: bash
$ bash Miniconda3-latest-MacOSX-x86_64.sh
$ bash Miniforge3-MacOSX-x86_64.sh
Follow the prompts on the screen.
Accept the defaults offered for all of the settings
Expand All @@ -93,43 +98,70 @@ In short:
# packages in environment at /home/dlatorne/miniconda3:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main
brotlipy 0.7.0 py39h27cfd23_1003
ca-certificates 2021.7.5 h06a4308_1
certifi 2021.5.30 py39h06a4308_0
cffi 1.14.6 py39h400218f_0
chardet 4.0.0 py39h06a4308_1003
conda 4.10.3 py39h06a4308_0
conda-package-handling 1.7.3 py39h27cfd23_1
cryptography 3.4.7 py39hd23ed53_0
idna 2.10 pyhd3eb1b0_0
ld_impl_linux-64 2.35.1 h7274673_9
libedit 3.1.20210216 h27cfd23_1
libffi 3.3 he6710b0_2
libgcc-ng 9.1.0 hdf63c60_0
libstdcxx-ng 9.1.0 hdf63c60_0
ncurses 6.2 he6710b0_1
openssl 1.1.1l h7f8727e_0
pip 21.2.4 py37h06a4308_0
pycosat 0.6.3 py39h27cfd23_0
pycparser 2.20 py_2
pyopenssl 20.0.1 pyhd3eb1b0_1
pysocks 1.7.1 py39h06a4308_0
python 3.9.6 h12debd9_1
readline 8.1 h27cfd23_0
requests 2.25.1 pyhd3eb1b0_0
ruamel_yaml 0.15.100 py39h27cfd23_0
setuptools 52.0.0 py39h06a4308_0
six 1.16.0 pyhd3eb1b0_0
sqlite 3.36.0 hc218d9a_0
tk 8.6.10 hbc83047_0
tqdm 4.59.0 pyhd3eb1b0_1
tzdata 2021a h5d7bf9c_0
urllib3 1.26.4 pyhd3eb1b0_0
wheel 0.36.2 pyhd3eb1b0_0
xz 5.2.5 h7b6447c_0
yaml 0.2.5 h7b6447c_0
zlib 1.2.11 h7b6447c_3
_libgcc_mutex 0.1 conda_forge conda-forge
_openmp_mutex 4.5 1_gnu conda-forge
brotlipy 0.7.0 py37h6b43d8f_1003 conda-forge
bzip2 1.0.8 h7f98852_4 conda-forge
c-ares 1.18.1 h7f98852_0 conda-forge
ca-certificates 2021.10.8 ha878542_0 conda-forge
certifi 2021.10.8 py37h9c2f6ca_1 conda-forge
cffi 1.14.6 0_pypy37 conda-forge
charset-normalizer 2.0.8 pyhd8ed1ab_0 conda-forge
colorama 0.4.4 pyh9f0ad1d_0 conda-forge
conda 4.10.3 py37h9c2f6ca_4 conda-forge
conda-package-handling 1.7.3 py37h6b43d8f_1 conda-forge
cryptography 36.0.0 py37h5c3f282_0 conda-forge
expat 2.4.1 h9c3ff4c_0 conda-forge
gdbm 1.18 h0a1914f_2 conda-forge
icu 69.1 h9c3ff4c_0 conda-forge
idna 3.1 pyhd3deb0d_0 conda-forge
krb5 1.19.2 hcc1bbae_3 conda-forge
libarchive 3.5.2 hccf745f_1 conda-forge
libcurl 7.80.0 h2574ce0_0 conda-forge
libedit 3.1.20191231 he28a2e2_2 conda-forge
libev 4.33 h516909a_1 conda-forge
libffi 3.4.2 h7f98852_5 conda-forge
libgcc-ng 11.2.0 h1d223b6_11 conda-forge
libgomp 11.2.0 h1d223b6_11 conda-forge
libiconv 1.16 h516909a_0 conda-forge
libmamba 0.19.0 h3985d26_0 conda-forge
libmambapy 0.19.0 py37h9bd18e5_0 conda-forge
libnghttp2 1.43.0 h812cca2_1 conda-forge
libsolv 0.7.19 h780b84a_5 conda-forge
libssh2 1.10.0 ha56f1ee_2 conda-forge
libstdcxx-ng 11.2.0 he4da1e4_11 conda-forge
libxml2 2.9.12 h885dcf4_1 conda-forge
libzlib 1.2.11 h36c2ea0_1013 conda-forge
lz4-c 1.9.3 h9c3ff4c_1 conda-forge
lzo 2.10 h516909a_1000 conda-forge
mamba 0.19.0 py37h47bf687_0 conda-forge
ncurses 6.2 h58526e2_4 conda-forge
openssl 1.1.1l h7f98852_0 conda-forge
pip 21.3.1 pyhd8ed1ab_0 conda-forge
pybind11-abi 4 hd8ed1ab_3 conda-forge
pycosat 0.6.3 py37h6b43d8f_1009 conda-forge
pyopenssl 21.0.0 pyhd8ed1ab_0 conda-forge
pypy3.7 7.3.7 hbc09475_3 conda-forge
pysocks 1.7.1 py37h9c2f6ca_4 conda-forge
python 3.7.12 0_73_pypy conda-forge
python_abi 3.7 2_pypy37_pp73 conda-forge
readline 8.1 h46c0cb4_0 conda-forge
reproc 14.2.3 h7f98852_0 conda-forge
reproc-cpp 14.2.3 h9c3ff4c_0 conda-forge
requests 2.26.0 pyhd8ed1ab_1 conda-forge
ruamel_yaml 0.15.80 py37h6b43d8f_1006 conda-forge
setuptools 59.4.0 py37h9c2f6ca_0 conda-forge
six 1.16.0 pyh6c4a22f_0 conda-forge
sqlite 3.37.0 h9cd32fc_0 conda-forge
tk 8.6.11 h27826a3_1 conda-forge
tqdm 4.62.3 pyhd8ed1ab_0 conda-forge
urllib3 1.26.7 pyhd8ed1ab_0 conda-forge
wheel 0.37.0 pyhd8ed1ab_1 conda-forge
xz 5.2.5 h516909a_1 conda-forge
yaml 0.2.5 h516909a_0 conda-forge
yaml-cpp 0.6.3 he1b5a44_4 conda-forge
zlib 1.2.11 h36c2ea0_1013 conda-forge
zstd 1.5.0 ha95c52a_0 conda-forge
.. _condaConfiguration:
Expand All @@ -144,24 +176,19 @@ and on the Waterhole workstation that you use
(which will cover all of the Waterhole/Ocean machines).

The :command:`conda config` command is how you interact with the :file:`$HOME/.condarc` file.
Start by telling :program:`conda` to use the `conda-forge`_ channel as its preferred channel to find packages:

.. _conda-forge: https://conda-forge.org/

.. code-block:: bash
$ conda config --prepend channels conda-forge
Also do:
Start by telling :program:`conda` where you want to store your environments:

.. code-block:: bash
$ conda config --prepend envs_dirs $HOME/conda_envs/
$ mkdir $HOME/conda_envs/
The first of those lines tells :program:`conda` that you want to put your environments in a directory called :file:`$HOME/conda_envs/`.
The first of those lines tells :program:`conda` that you want to put your environments
in a directory called :file:`$HOME/conda_envs/`.
The second line creates that directory.
Storing environment directory trees outside of the :file:`$HOME/miniconda3/` directory created by the installer means that if you need to re-install Miniconda you can do so without destroying all of your environments.
Storing environment directory trees outside of the :file:`$HOME/miniforge3/` directory
created by the installer means that if you need to re-install Miniforge
you can do so without destroying all of your environments.

If you want to see all of the :program:`conda` configuration settings
(both the defaults,
Expand Down
12 changes: 6 additions & 6 deletions getting_started.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,7 @@ Here's the Getting Started checklist:
#. :ref:`SetUpSecureRemoteAccess`
#. :ref:`SetUpGit`
#. :ref:`SetUpBash`
#. :ref:`InstallMiniconda`
#. :ref:`InstallMiniforge`
#. :ref:`CreateYourAnalysisRepository`


Expand Down Expand Up @@ -121,16 +121,16 @@ before you can:
on a Waterhole workstation.


.. _InstallMiniconda:
.. _InstallMiniforge:

Install Miniconda
Install Miniforge
=================

You will need to:

#. Learn about :ref:`MOAD-CondaPkgAndEnvMgr`
#. :ref:`Install Miniconda<InstallingMiniconda>` on your laptop
#. :ref:`Install Miniconda<InstallingMiniconda>` in your workspace on the EOAS Ocean collection of Linux computers
#. :ref:`Install Miniforge<InstallingMiniforge>` on your laptop
#. :ref:`Install Miniforge<InstallingMiniforge>` in your workspace on the EOAS Ocean collection of Linux computers


.. _CreateYourAnalysisRepository:
Expand All @@ -141,6 +141,6 @@ Create Your Analysis Repository
You will need to have:

#. :ref:`SetUpGit`
#. :ref:`Installed Miniconda<InstallMiniconda>`
#. :ref:`Installed Miniconda<InstallMiniforge>`

before you can create your :ref:`analysis repository<MOAD-AnalysisRepository>`.

0 comments on commit 0da300e

Please sign in to comment.