Streamlined miniconda install for faster, more direct environment creation

[buhrmann]

Hi,
Perhaps it's something I'm doing wrong, or something I'm missing, but it seems that creating a clean, minimal and up-to-date miniconda install is far from efficient at the moment. Essentially, to get to a final desired environment, python, setuptools, conda etc. may get downloaded and installed in different versions up to 3 times: 1) in the initial miniconda install, 2) in the conda update, and 3) in the environment creation (see below).

Ideally, I would be able to configure the channel and preferred Python version somehow to directly download and install the version of the packages I want from the beginning.

Is there any way to streamline this? E.g. by configuring the miniconda.sh script? Or are there any plans to allow this in the future?

Current Behavior

E.g. to create an install and environment based on conda-forge and Python 3.7.5, the steps below seem to be necessary. The problem is that this leads to many more package downloads and installs than necessary. Packages like conda itself, setuptools, python etc. may get downloaded 3 times in different versions, each time along with any necessary (changed) dependencies. This makes builds in production take much longer than necessary.

Steps to Reproduce

# Install miniconda itself
wget -O miniconda.sh https://repo.continuum.io/miniconda/Miniconda3-latest-Linux-x86_64.sh
chmod +x miniconda.sh && ./miniconda.sh -b && rm miniconda.sh

# Ensure we use conda-forge and get latest versions of everything
conda config --prepend channels conda-forge
conda update -y -n base conda setuptools
conda update -y -n base --all

# Create the environment
conda create -n myenv python=3.7.5

Given the above commands, e.g., the initial miniconda3_latest_xxx.sh will install the following versions (when I wrote this):

conda==4.7.12=py37_0
python==3.7.4=h265db76_1
setuptools==41.4.0=py37_0
...

The update of (the supposed latest conda) will install:

conda==4.8.2=py37_0
setuptools=45.1.0=py37_0
...and 30 other new package versions

Finally, the creation of the environment would install:

python==3.7.5=xxxx
... and any dependencies affected

Expected Behavior

It would be nice if miniconda3_latest_xxx.sh would install always with the latest conda, so that it wouldn't be necessary to always have to update conda first thing. Additionally, any way to configure the install such that the desired python, channel versions etc. are installed directly on the first shot would also be nice.

Would any of those things be feasible?

[forrestwaters]

We have some fairly new documentation here that would likely fit your use-case using our new conda standalone: https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/use-conda-with-travis-ci.html#bootstrap-your-environment

[buhrmann]

Thanks. From the AppVeyor config you linked, and the executable used there (https://repo.anaconda.com/pkgs/misc/conda-execs/conda-latest-linux-64.exe), I understand that this new conda standalone is Window-only for now? Are there plans to provide a Linux version, as a direct replacement for miniconda.sh?

[forrestwaters]

The standalone exists for all 3 platforms (the exe extension is just how we differentiate the standalone from the normal conda entry point). The one you linked is the correct one for Linux

[buhrmann]

Yes, right, thanks for the clarification! I mistakenly downloaded the Linux executable and tried it on Macos. The following seems to work as intended:

./conda-latest-osx-64.exe create -p ~/my/envs/test/ -c conda-forge --override-channels conda python=3.8 ...

As another benefit, the standalone executable seems to be quite a bit smaller than miniconda (10MB vs 70MB).

Is this stable already and safe for use in production?

[github-actions]

Hi there, thank you for your contribution!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.

If you would like this issue to remain open please:

1. Verify that you can still reproduce the issue at hand
2. Comment that the issue is still reproducible and include:
   - What OS and version you reproduced the issue on
   - What steps you followed to reproduce the issue

NOTE: If this issue was closed prematurely, please leave a comment.

Thanks!