Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Switch to Mamba #697

Merged
merged 7 commits into from
Oct 19, 2021
Merged

Switch to Mamba #697

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
0018bec
Switch to mambaforge
manics Jul 31, 2021
d869540
ensure mamba installed, add old miniconda check back
manics Jul 31, 2021
1859a46
Lower test memory to 900m
manics Jul 31, 2021
e3bb87b
Update Amazon docs:
manics Oct 18, 2021
b3c07ba
miniforge FIXME was fixed!
manics Oct 18, 2021
80b3473
Update to mambaforge_version 4.10.3-7
manics Oct 18, 2021
f61b7fb
Fix typo
manics Oct 18, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
6 changes: 3 additions & 3 deletions .github/integration-test.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,9 +29,9 @@ def run_systemd_image(image_name, container_name, bootstrap_pip_spec):
'--mount', 'type=bind,source=/sys/fs/cgroup,target=/sys/fs/cgroup',
'--detach',
'--name', container_name,
# This is the minimum VM size we support.
# If we change this, need to change all other references to this number.
'--memory', '1G',
# A bit less than 1GB to ensure TLJH runs on 1GB VMs.
# If this is changed all docs references to the required memory must be changed too.
'--memory', '900m',
]

if bootstrap_pip_spec:
Expand Down
5 changes: 3 additions & 2 deletions docs/install/amazon.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -76,8 +76,9 @@ Let's create the server on which we can run JupyterHub.
`Next: Configure Instance Details` in the lower right corner.

Check out our guide on How To :ref:`howto/admin/resource-estimation` to help pick
how much Memory / CPU your server needs. You need to have at least **1GB** of
RAM. The smallest instance that can fit this is a **t3.small**.
how much Memory / CPU your server needs.
We recommend you use a server with at leat 2GB of RAM, such as a **t3.small**.
However, if you need to minimise costs you can use a server with **1GB** RAM such as a **t2.micro**, but performance will be limited.

You may wish to consult the listing `here <https://www.ec2instances.info/>`_
because it shows cost per hour. The **On Demand** price is the pertinent cost.
Expand Down
10 changes: 5 additions & 5 deletions tests/test_conda.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,14 +13,14 @@ def prefix():
"""
Provide a temporary directory with a conda environment
"""
miniconda_version = '4.7.10'
miniconda_installer_sha256 = "8a324adcc9eaf1c09e22a992bb6234d91a94146840ee6b11c114ecadafc68121"
installer_url = "https://repo.continuum.io/miniconda/Miniconda3-{}-Linux-x86_64.sh".format(miniconda_version)
mambaforge_version = '4.10.3-3'
installer_sha256 = "a012c24e1cc3bcbe74a1e5693e510830e7c2956e85877b08d1e28707a0bd8d75"
installer_url = "https://github.com/conda-forge/miniforge/releases/download/{v}/Mambaforge-{v}-Linux-x86_64.sh".format(v=mambaforge_version)
with tempfile.TemporaryDirectory() as tmpdir:
with conda.download_miniconda_installer(installer_url, miniconda_installer_sha256) as installer_path:
with conda.download_miniconda_installer(installer_url, installer_sha256) as installer_path:
conda.install_miniconda(installer_path, tmpdir)
conda.ensure_conda_packages(tmpdir, [
'conda==4.8.1'
'conda==4.10.3'
])
yield tmpdir

Expand Down
2 changes: 1 addition & 1 deletion tljh/conda.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,7 @@ def ensure_conda_packages(prefix, packages):
"""
Ensure packages (from conda-forge) are installed in the conda prefix.
"""
conda_executable = [os.path.join(prefix, 'bin', 'python'), '-m', 'conda']
conda_executable = [os.path.join(prefix, 'bin', 'mamba')]
consideRatio marked this conversation as resolved.
Show resolved Hide resolved
abspath = os.path.abspath(prefix)
# Let subprocess errors propagate
# Explicitly do *not* capture stderr, since that's not always JSON!
Expand Down
20 changes: 13 additions & 7 deletions tljh/installer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -172,24 +172,30 @@ def ensure_user_environment(user_requirements_txt_file):

miniconda_old_version = '4.5.4'
miniconda_new_version = '4.7.10'
miniconda_installer_sha256 = "8a324adcc9eaf1c09e22a992bb6234d91a94146840ee6b11c114ecadafc68121"

if conda.check_miniconda_version(USER_ENV_PREFIX, miniconda_new_version):
mambaconda_new_version = '4.10.3'
mambaforge_new_version = '4.10.3-3'
installer_sha256 = "a012c24e1cc3bcbe74a1e5693e510830e7c2956e85877b08d1e28707a0bd8d75"
mamba_version = '0.15.2'
manics marked this conversation as resolved.
Show resolved Hide resolved

if conda.check_miniconda_version(USER_ENV_PREFIX, mambaconda_new_version):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you understand the idea about this? It would be great to have a comment around here regarding these checks.

Copy link
Member Author

@manics manics Oct 18, 2021 •
edited

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did figure it out eventually but I can't remember the details now! It's something along the lines of check the conda version, from which you can infer the miniconda version that was used, and from that you can infer how old the TLJH installation is.

Edit: Or maybe it's the other way round, figure out which version of miniconda is installed, then choose which version of conda we want based on that (maybe because older miniconda versions installed an older version of Python which isn't suportted by newer conda?)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I consider this non-blocking for a merge. I've heard about feature creep, but this is more like a maintenance creep of me to ask about :D

conda_version = '4.10.3'
elif conda.check_miniconda_version(USER_ENV_PREFIX, miniconda_new_version):
conda_version = '4.8.1'
elif conda.check_miniconda_version(USER_ENV_PREFIX, miniconda_old_version):
conda_version = '4.5.8'
# If no prior miniconda installation is found, we can install a newer version
else:
logger.info('Downloading & setting up user environment...')
# FIXME: allow using miniforge
manics marked this conversation as resolved.
Show resolved Hide resolved
installer_url = "https://repo.continuum.io/miniconda/Miniconda3-{}-Linux-x86_64.sh".format(miniconda_new_version)
with conda.download_miniconda_installer(installer_url, miniconda_installer_sha256) as installer_path:
installer_url = "https://github.com/conda-forge/miniforge/releases/download/{v}/Mambaforge-{v}-Linux-x86_64.sh".format(v=mambaforge_new_version)
with conda.download_miniconda_installer(installer_url, installer_sha256) as installer_path:
conda.install_miniconda(installer_path, USER_ENV_PREFIX)
conda_version = '4.8.1'
conda_version = '4.10.3'

conda.ensure_conda_packages(USER_ENV_PREFIX, [
# Conda's latest version is on conda much more so than on PyPI.
'conda==' + conda_version
'conda==' + conda_version,
'mamba==' + mamba_version,
])

conda.ensure_pip_requirements(
Expand Down