`conda env create --file` errors one package at a time

[mivade]

When trying to create a new environment from a environment YAML file that was created on one operating system, I get errors such as the following when running on another OS:

ResolvePackageNotFound: 
  - pango 1.39.0 1

The problem is that as soon as I comment the offending package out (since it is likely not a hard requirement, but rather just something that happened to be installed by whomever created the environment file), I just get another similar error for another package.

Considering it can take 10s of seconds before getting this error, it would be especially helpful to be presented with a list of all packages that can't be found instead of having to hunt them one at a time.

[MichaelJendryke]

+1

[matthiasroder]

I run into the same issue.

conda env create -f environment.yml -v

gives me:

Traceback (most recent call last): File "/home/ubuntu/anaconda2/lib/python2.7/site-packages/conda/exceptions.py", line 640, in conda_exception_handler return_value = func(*args, **kwargs) File "/home/ubuntu/anaconda2/lib/python2.7/site-packages/conda_env/cli/main_create.py", line 108, in execute installer.install(prefix, pkg_specs, args, env) File "/home/ubuntu/anaconda2/lib/python2.7/site-packages/conda_env/installers/conda.py", line 32, in install channel_priority_map=_channel_priority_map) File "/home/ubuntu/anaconda2/lib/python2.7/site-packages/conda/plan.py", line 475, in install_actions_list dists_for_envs = determine_all_envs(r, specs, channel_priority_map=channel_priority_map) File "/home/ubuntu/anaconda2/lib/python2.7/site-packages/conda/plan.py", line 540, in determine_all_envs spec_for_envs = tuple(SpecForEnv(env=p.preferred_env, spec=p.name) for p in best_pkgs) File "/home/ubuntu/anaconda2/lib/python2.7/site-packages/conda/plan.py", line 540, in <genexpr> spec_for_envs = tuple(SpecForEnv(env=p.preferred_env, spec=p.name) for p in best_pkgs) File "/home/ubuntu/anaconda2/lib/python2.7/site-packages/conda/plan.py", line 539, in <genexpr> best_pkgs = (r.index[r.get_dists_for_spec(s, emptyok=False)[-1]] for s in specs) File "/home/ubuntu/anaconda2/lib/python2.7/site-packages/conda/resolve.py", line 563, in get_dists_for_spec raise ResolvePackageNotFound([(ms,)]) ResolvePackageNotFound: - backports 1.0 py27h6492d98_1

I suspect that this has something to do with "channels" and how conda sorts dependencies. I therefore wanted to see whether the situation would be solved by using the old "sorting algorithm":

conda config --set channel_priority false

and then again:

conda env create -f environment.yml -v

to no avail.

[colllin]

Is anyone looking into this? I am experiencing the same problem after exporting a conda environment on macOS and then trying to reproduce the environment on Ubuntu 16.04. Actually this is literally the only thing I did.

• I started on macOS,
• installed some dependencies,
• realized the data was too big,
• exported the environment,
• spun up an Ubuntu instance on EC2,
• cloned the repo,
• tried to re-create the conda environment.

I exported using conda env export > environment.yml, and tried to re-create using conda env create --name my-project --file environment.yml. I receive the same error as reported by OP:

ResolvePackageNotFound: 
  - ca-certificates 2017.08.26 ha1e5d58_0

I also tried to first create a conda environment without the file, then install from the file, with a different error:

$ conda create --name my-project `grep -o 'python=[0-9.]\+' environment.yml`
$ conda install --file environment.yml
CondaValueError: could not parse 'name: my-project' in: environment.yml

This is the same error I receive if I try to create the environment from the file using conda create rather than conda env create.

[kalefranz]

Oftentimes build numbers don't match across platforms. Use the --no-builds flag with conda env export.

conda env export --no-builds > environment.yml

[mivade]

What I have generally been doing to work around this is to hand craft environment files rather than using conda env export and only including packages that I'm explicitly depending on. This is not always ideal (especially for things that might have a lot of dependencies), but it has been working well enough so far.

[colllin]

Thanks for the --no-builds tip, @kalefranz. That seemed promising. I believe this changes the problem, but I still have similar looking errors, albeit a much smaller list of failed packages:

ResolvePackageNotFound: 
  - appnope=0.1.0
  - libcxx=4.0.1
  - libgfortran=3.0.1
  - libcxxabi=4.0.1

Note: I also ran conda update conda, which updated me to 4.4.7 (miniconda and anaconda don't ship with latest conda?).

Is there a simple explanation here? The linux channels are not in-sync with the macOS channels? Or... I'm fairly new to python... Can package dependencies vary depending on the destination platform?

@mivade, good idea... Glad to hear it's a working path forward. It kind of defeats the purpose of using an environment manager, though — even if I am open to it, it would be ridiculous to try to get my team on-board, and expect them to learn how to do it and get it right, with complexities such as channels, etc. But I'm not sure what other option I have right now!

For reference, I believe npm install --save and yarn add behave similarly to what @mivade describes. When you explicitly install/add a package, it only writes that package to your manifest, but will also install any dependencies. The dependencies don't need to be written, because they are just that — dependencies — and can be determined dynamically at install-time. I believe this is a better pattern than what is currently followed by conda env export, which appears to be more equivalent to a lock-file, which works great in node-land, but apparently doesn't work so well in python-land.

I have already wasted too much time on these issues to dig any further on my own. If you have other ideas, I will definitely give them a shot.

[kalefranz]

The reason this issue is tagged as a feature request and with the “environment-spec” tag is because we know there’s quite a bit of work we need to do here to improve the user experience.

Indeed conda env export should only include, or have a mode that only includes, “user-provided” specs, and not the equivalent of a lock file that lists every package in the environment.

[maxwellmckinnon]

I also am blocked by the libcxx issue

Using Anaconda API: https://api.anaconda.org
Fetching package metadata .............

ResolvePackageNotFound:
  - libcxx 4.0.1*`

Like the original poster said, very aggravating that it takes 10s to see the failure, then fixing it just brings on more.

Can't there be a switch to just install everything possible from the environment, then alert which ones failed afterward? This defeats the #1 promise of anaconda being a solution to the "runs on my computer" problem. The biggest/only thing I want from anaconda is the export reqs and import reqs to work flawlessly.

Is there another workflow I'm missing that is the intended path of deploying code to another computer? I thought it was supposed to be conda env export > env.yml on one, conda create env -f env.yml on the other.

[gabegm]

Have the same problem, had to manually remove the dependencies causing the error from the env file.

ResolvePackageNotFound:

• libcxxabi=4.0.1
• libcxx=4.0.1

Exported the env from macOS and tried to import on Linux

[pattersongp]

+1

[leemengtw]

Exactly same problem like @gabegm even if I used

conda env export --no-builds > environment.yaml

Had to manually remote the dependencies

libcxxabi=4.0.1
libcxx=4.0.1

which should be handle more gracefully by conda.

[ryanrunchey]

Same problem

conda env create --force -f=/Users/ryanrunchey/Downloads/gcloud_ryanrunchey_starling_conda_env_base_2018-07-25/environment.yml -n gcloud_starling_base

Using Anaconda API: https://api.anaconda.org
Fetching package metadata .............

ResolvePackageNotFound:

• gensim 3.5.0 py36_0