Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Conda env create fails using environment.yml file. #9480

Closed
sorenwacker opened this issue May 29, 2018 · 31 comments
Closed

Conda env create fails using environment.yml file. #9480

sorenwacker opened this issue May 29, 2018 · 31 comments

Comments

@sorenwacker
Copy link

@sorenwacker sorenwacker commented May 29, 2018

I exported an environment using conda env export > environment.yml.
I installed miniconda3 on a different system, updated conda, installed pip.
Then conda env create -f environment.yml -n py3

Actual Behavior

It gives me this error message:

Solving environment: failed
ResolvePackageNotFound:
  - brewer2mpl==1.4=py36_0
  - ggplot==0.9.4=py36_0

Expected Behavior

The environment should be installed with the same packages indicated in the yml file. All necessary information, such as channels, should be provided by the yml file. And it should give a warning about packages from locally installed sources. I noted that e.g. the packages pyqsar from my source system is not available on the target system. Though conda did not raise a warning.

Steps to Reproduce

TARGET SYSTEM
     active environment : None
       user config file : /home/swacker/.condarc
 populated config files :
          conda version : 4.5.4
    conda-build version : not installed
         python version : 3.6.5.final.0
       base environment : /home/swacker/miniconda3  (writable)
           channel URLs : https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/linux-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/pro/linux-64
                          https://repo.anaconda.com/pkgs/pro/noarch
          package cache : /home/swacker/miniconda3/pkgs
                          /home/swacker/.conda/pkgs
       envs directories : /home/swacker/miniconda3/envs
                          /home/swacker/.conda/envs
               platform : linux-64
             user-agent : conda/4.5.4 requests/2.18.4 CPython/3.6.5 Linux/3.10.0-693.17.1.el7.x86_64 centos/7 glibc/2.17
                UID:GID : 16608087:16608087
             netrc file : None
           offline mode : False

# packages in environment at /home/swacker/miniconda3:
#
# Name                    Version                   Build  Channel
asn1crypto                0.24.0                   py36_0    defaults
ca-certificates           2018.03.07                    0    defaults
certifi                   2018.4.16                py36_0    defaults
cffi                      1.11.5           py36h9745a5d_0    defaults
chardet                   3.0.4            py36h0f667ec_1    defaults
conda                     4.5.4                    py36_0    defaults
conda-env                 2.6.0                h36134e3_1    defaults
cryptography              2.2.2            py36h14c3975_0    defaults
idna                      2.6              py36h82fb2a8_1    defaults
libedit                   3.1                  heed3624_0    defaults
libffi                    3.2.1                hd88cf55_4    defaults
libgcc-ng                 7.2.0                hdf63c60_3    defaults
libstdcxx-ng              7.2.0                hdf63c60_3    defaults
ncurses                   6.0                  h9df7e31_2    defaults
openssl                   1.0.2o               h20670df_0    defaults
pip                       9.0.3                    py36_0    defaults
pycosat                   0.6.3            py36h0a5515d_0    defaults
pycparser                 2.18             py36hf9f622e_1    defaults
pyopenssl                 17.5.0           py36h20ba746_0    defaults
pysocks                   1.6.8                    py36_0    defaults
python                    3.6.5                hc3d631a_0    defaults
readline                  7.0                  ha6073c6_4    defaults
requests                  2.18.4           py36he2e5f8d_1    defaults
ruamel_yaml               0.15.35          py36h14c3975_1    defaults
setuptools                39.0.1                   py36_0    defaults
six                       1.11.0           py36h372c433_1    defaults
sqlite                    3.23.1               he433501_0    defaults
tk                        8.6.7                hc745277_3    defaults
urllib3                   1.22             py36hbe7ace6_0    defaults
wheel                     0.31.0                   py36_0    defaults
xz                        5.2.3                h55aa19d_2    defaults
yaml                      0.1.7                had09818_2    defaults
zlib                      1.2.11               ha838bed_2    defaults
environment.yml
cat environment.yml
name: py3
channels:
- conda-forge
- bioconda
- bokeh
- defaults
dependencies:
- p7zip=15.09=3
- brewer2mpl=1.4=py36_0
- ggplot=0.9.4=py36_0
- jupyter_dashboards=0.7.0=py36_0
- _tflow_180_select=1.0=gpu
- absl-py=0.2.1=py36_0
- asn1crypto=0.24.0=py36_0
- astor=0.6.2=py36_0
- backcall=0.1.0=py36_0
- backports=1.0=py36hfa02d7e_1
- backports.weakref=1.0.post1=py36h39d5b32_0
- blas=1.0=mkl
- bleach=1.5.0=py36_0
- blosc=1.14.3=hdbcaa40_0
- bokeh=0.12.16=py36_0
- bzip2=1.0.6=h14c3975_5
- ca-certificates=2018.03.07=0
- certifi=2018.4.16=py36_0
- cffi=1.11.5=py36h9745a5d_0
- chardet=3.0.4=py36h0f667ec_1
- click=6.7=py36h5253387_0
- cloudpickle=0.5.3=py36_0
- colorcet=1.0.0=py36_0
- cryptography=2.2.2=py36h14c3975_0
- cudatoolkit=9.0=h13b8566_0
- cudnn=7.1.2=cuda9.0_0
- cupti=9.0.176=0
- cycler=0.10.0=py36h93f1223_0
- cython=0.28.2=py36h14c3975_0
- cytoolz=0.9.0.1=py36h14c3975_0
- dask=0.17.5=py36_0
- dask-core=0.17.5=py36_0
- datashader=0.6.5=py36_0
- datashape=0.5.4=py36h3ad6b5c_0
- dbus=1.13.2=h714fa37_1
- decorator=4.3.0=py36_0
- distributed=1.21.8=py36_0
- entrypoints=0.2.3=py36h1aec115_2
- expat=2.2.5=he0dffb1_0
- fontconfig=2.12.6=h49f89f6_0
- freetype=2.8=hab7d2ae_1
- gast=0.2.0=py36_0
- glib=2.56.1=h000015b_0
- gmp=6.1.2=h6c8ec71_1
- grpcio=1.11.0=py36hf484d3e_0
- gst-plugins-base=1.14.0=hbbd80ab_1
- gstreamer=1.14.0=hb453b48_1
- h5py=2.7.1=py36h3585f63_0
- hdf5=1.10.1=h9caa474_1
- heapdict=1.0.0=py36_2
- html5lib=0.9999999=py36_0
- icu=58.2=h9c2bf20_1
- idna=2.6=py36h82fb2a8_1
- imageio=2.3.0=py36_0
- inflection=0.3.1=py36_0
- intel-openmp=2018.0.0=8
- ipykernel=4.8.2=py36_0
- ipython=6.4.0=py36_0
- ipython_genutils=0.2.0=py36hb52b0d5_0
- ipywidgets=7.2.1=py36_0
- jedi=0.12.0=py36_1
- jinja2=2.10=py36ha16c418_0
- jpeg=9b=h024ee3a_2
- jsonschema=2.6.0=py36h006f8b5_0
- jupyter=1.0.0=py36_4
- jupyter_client=5.2.3=py36_0
- jupyter_console=5.2.0=py36he59e554_1
- jupyter_core=4.4.0=py36h7c827e3_0
- jupyterlab=0.32.1=py36_0
- jupyterlab_launcher=0.10.5=py36_0
- keras=2.1.5=py36_0
- kiwisolver=1.0.1=py36h764f252_0
- libedit=3.1.20170329=h6b74fdf_2
- libffi=3.2.1=hd88cf55_4
- libgcc-ng=7.2.0=hdf63c60_3
- libgfortran-ng=7.2.0=hdf63c60_3
- libpng=1.6.34=hb9fc6fc_0
- libprotobuf=3.5.2=h6f1eeef_0
- libsodium=1.0.16=h1bed415_0
- libstdcxx-ng=7.2.0=hdf63c60_3
- libtiff=4.0.9=he85c1e1_1
- libxcb=1.13=h1bed415_1
- libxml2=2.9.8=h26e45fe_1
- llvmlite=0.23.1=py36hdbcaa40_0
- locket=0.2.0=py36h787c0ad_1
- lzo=2.10=h49e0be7_2
- markdown=2.6.11=py36_0
- markupsafe=1.0=py36hd9260cd_1
- matplotlib=2.2.2=py36h0e671d2_1
- mistune=0.8.3=py36h14c3975_1
- mkl=2018.0.2=1
- mkl_fft=1.0.1=py36h3010b51_0
- mkl_random=1.0.1=py36h629b387_0
- mock=2.0.0=py36h3c5bf6c_0
- more-itertools=4.1.0=py36_0
- msgpack-python=0.5.6=py36h6bb024c_0
- multipledispatch=0.5.0=py36_0
- nbconvert=5.3.1=py36hb41ffb7_0
- nbformat=4.4.0=py36h31c9010_0
- ncurses=6.1=hf484d3e_0
- networkx=2.1=py36_0
- nose=1.3.7=py36hcdf7029_2
- notebook=5.5.0=py36_0
- numba=0.38.0=py36h637b7d7_0
- numexpr=2.6.5=py36h7bf3b9c_0
- numpy=1.14.3=py36hcd700cb_1
- numpy-base=1.14.3=py36h9be14a7_1
- olefile=0.45.1=py36_0
- openssl=1.0.2o=h20670df_0
- packaging=17.1=py36_0
- pandas=0.23.0=py36h637b7d7_0
- pandoc=1.19.2.1=hea2e7c5_1
- pandocfilters=1.4.2=py36ha6701b7_1
- param=1.6.1=py36_0
- parso=0.2.0=py36_0
- partd=0.3.8=py36h36fd896_0
- patsy=0.5.0=py36_0
- pbr=4.0.3=py36_0
- pcre=8.42=h439df22_0
- pexpect=4.5.0=py36_0
- pickleshare=0.7.4=py36h63277f8_0
- pillow=5.1.0=py36h3deb7b8_0
- pip=10.0.1=py36_0
- plotly=2.6.0=py36_0
- prompt_toolkit=1.0.15=py36h17d85b1_0
- protobuf=3.5.2=py36hf484d3e_0
- psutil=5.4.5=py36h14c3975_0
- ptyprocess=0.5.2=py36h69acd42_0
- pycparser=2.18=py36hf9f622e_1
- pygments=2.2.0=py36h0d3125c_0
- pyopenssl=18.0.0=py36_0
- pyparsing=2.2.0=py36hee85983_1
- pyqt=5.9.2=py36h751905a_0
- pysocks=1.6.8=py36_0
- pytables=3.4.3=py36h02b9ad4_0
- python=3.6.5=hc3d631a_2
- python-dateutil=2.7.3=py36_0
- pytz=2018.4=py36_0
- pywavelets=0.5.2=py36he602eb0_0
- pyyaml=3.12=py36hafb9ca4_1
- pyzmq=17.0.0=py36h14c3975_0
- qt=5.9.5=h7e424d6_0
- qtconsole=4.3.1=py36h8f73b5b_0
- quandl=3.3.0=py36_0
- readline=7.0=ha6073c6_4
- requests=2.18.4=py36he2e5f8d_1
- scikit-image=0.13.1=py36h14c3975_1
- scikit-learn=0.19.1=py36h7aa7ec6_0
- scipy=1.1.0=py36hfc37229_0
- seaborn=0.8.1=py36hfad7ec4_0
- send2trash=1.5.0=py36_0
- setuptools=39.1.0=py36_0
- simplegeneric=0.8.1=py36_2
- sip=4.19.8=py36hf484d3e_0
- six=1.11.0=py36h372c433_1
- snappy=1.1.7=hbae5bb6_3
- sortedcontainers=1.5.10=py36_0
- sqlalchemy=1.2.7=py36h6b74fdf_0
- sqlite=3.23.1=he433501_0
- statsmodels=0.9.0=py36h3010b51_0
- tblib=1.3.2=py36h34cf8b6_0
- tensorboard=1.8.0=py36hf484d3e_0
- tensorflow=1.8.0=hb11d968_0
- tensorflow-base=1.8.0=py36hc1a7637_0
- tensorflow-tensorboard=1.5.1=py36hf484d3e_1
- termcolor=1.1.0=py36_1
- terminado=0.8.1=py36_1
- testpath=0.3.1=py36h8cadb63_0
- tk=8.6.7=hc745277_3
- toolz=0.9.0=py36_0
- tornado=5.0.2=py36_0
- tqdm=4.23.3=py36_0
- traitlets=4.3.2=py36h674d592_0
- urllib3=1.22=py36hbe7ace6_0
- wcwidth=0.1.7=py36hdf4376a_0
- werkzeug=0.14.1=py36_0
- wheel=0.31.1=py36_0
- widgetsnbextension=3.2.1=py36_0
- xarray=0.10.4=py36_0
- xz=5.2.4=h14c3975_4
- yaml=0.1.7=had09818_2
- zeromq=4.2.5=h439df22_0
- zict=0.1.3=py36h3a3bf81_0
- zlib=1.2.11=ha838bed_2
- pip:
  - args==0.1.0
  - chembl-webresource-client==0.9.31
  - clint==0.5.1
  - dill==0.2.7.1
  - easydict==1.7
  - enum34==1.1.6
  - floyd-cli==0.10.31
  - gevent==1.3.1
  - gevent-openssl==1.2
  - greenlet==0.4.13
  - grequests==0.2.0
  - holoviews (/mnt/c/Users/soere/workspace/holoviews)==1.10.4.post10+g8711663
  - ipython-genutils==0.2.0
  - jupyter-client==5.2.3
  - jupyter-console==5.2.0
  - jupyter-core==4.4.0
  - jupyter-dashboards==0.7.0
  - jupyterlab-launcher==0.10.5
  - lightgbm==2.0.11
  - lxml==4.2.1
  - marshmallow==2.15.0
  - mkl-fft==1.0.0
  - mkl-random==1.0.1
  - molvs==0.1.1
  - mytoolbox==0+untagged.3.gde0cad4
  - oauthlib==2.0.6
  - pathlib2==2.3.0
  - prompt-toolkit==1.0.15
  - pyqsar-ML-publication==1.ge51d073.dirty
  - pyteomics==3.4.2
  - raven==6.5.0
  - requests-cache==0.4.13
  - requests-oauthlib==0.8.0
  - requests-toolbelt==0.8.0
  - tables==3.4.3
  - tabulate==0.8.2
  - tweepy==3.5.0
  - unittest2py3k==0.5.1
  - unittest2six==0.0.0
  - versioneer==0.18
  - xgboost==0.7.post3
prefix: /home/swacker/miniconda3/envs/py3
@SKalt
Copy link

@SKalt SKalt commented Jun 5, 2018

Seems like those packages don't exist at that combination of version and build on the channels searched by your target system. I found broadening my search by eliminating build info helped. The command I used was conda env export --no-builds > environment.yml.

@sorenwacker
Copy link
Author

@sorenwacker sorenwacker commented Jun 6, 2018

Ok, I will try that. Though, I don't understand why the packages should not be available, since I also installed them at the source system.

@sorenwacker
Copy link
Author

@sorenwacker sorenwacker commented Jun 11, 2018

The error changed a bit.

conda env create -f environment.yml -n py3
Solving environment: failed

ResolvePackageNotFound:
  - brewer2mpl==1.4=py36_0

@SKalt
Copy link

@SKalt SKalt commented Jun 11, 2018

That was my experience as well: fixed around half the unresolved dependencies. FWIW, my eventual fix was installing the stubborn ones via pip.

@mahboobmm
Copy link

@mahboobmm mahboobmm commented Oct 10, 2018

Any updates on this? Going through the same issue.

@vv111y
Copy link

@vv111y vv111y commented Oct 20, 2018

same here

@chauek
Copy link

@chauek chauek commented Oct 26, 2018

This helped me: datitran/object_detector_app#41

@lennbo
Copy link

@lennbo lennbo commented Dec 4, 2018

Had that issue and solved it by first conda update --all on the host sytem, then conda export --no-builds > env.yml and regular installation via conda env create -f env.yml on the remote machine.

Guess for me the problem was that the base env conda on remote had packages installed that were not working with the older packages on the host system.

@6Hhcy
Copy link

@6Hhcy 6Hhcy commented Apr 8, 2019

Does anyone here know why we have any Python packages in the top section "dependencies". It seems like it only causes problems. Why not put them all under "pip"

@TejasAvinashShetty
Copy link

@TejasAvinashShetty TejasAvinashShetty commented May 5, 2019

I think there is a small error in https://github.com/ContinuumIO/anaconda-issues/issues/9480#issuecomment-444117929 namely that

conda export --no-builds > env.yml

should instead be written as

conda env export --no-builds > env.yml

@TejasAvinashShetty
Copy link

@TejasAvinashShetty TejasAvinashShetty commented May 5, 2019

One could also try adding a conda channel if the said packages are not present in the default channels. E.g. qutip package appears in conda-forge channel. One can add the channel by typing the following command.
conda config --append channels conda-forge Off course one needs to replace the name conda-forge by the appropriate channel name.

@TejasAvinashShetty
Copy link

@TejasAvinashShetty TejasAvinashShetty commented May 5, 2019

I think it also helps to remove any packages which would not be necessary to prevent conflicts. One thing that we must try is by removing the version numbers associated with the packages in the .yml file.

@iggy12345
Copy link

@iggy12345 iggy12345 commented Jun 10, 2019

Does anyone here know why we have any Python packages in the top section "dependencies". It seems like it only causes problems. Why not put them all under "pip"

putting them under dependencies causes anaconda to try and keep the versions compatible after installation, so if you were to install any further packages, then conda would keep them compatible. If you put them all under pip, then they are all installed via pip and would no longer be compatibility controlled by anaconda.

You could theoretically do that if you knew that you were never going to need to install another package, but otherwise, I'd say it's probably a bad idea.

@glinskyc
Copy link

@glinskyc glinskyc commented Jun 18, 2019

Beware of dependencies in your environment.yml that are specific to your dev machine, e.g. Windows, compared to what dependencies may actually be required or available on the remote machine, e.g. Linux. I've had to comment out such dependencies in environment.yml.

@EthiopianOne
Copy link

@EthiopianOne EthiopianOne commented Jul 27, 2019

Guys, Simple Answer,
Just open the .yml file with text editor and put all dependencies under "- pip:" and it will Work.

@msarahan
Copy link
Contributor

@msarahan msarahan commented Jul 27, 2019

Please see @iggy12345 answer above on why not to do this.

@mcubeddu
Copy link

@mcubeddu mcubeddu commented Aug 27, 2019

any updates on this??

@Gammaglobulino
Copy link

@Gammaglobulino Gammaglobulino commented Sep 17, 2019

solved with the EthiopianOne solution:

Guys, Simple Answer,
Just open the .yml file with text editor and put all dependencies under "- pip:" and it will Work.

@iggy12345
Copy link

@iggy12345 iggy12345 commented Sep 17, 2019

Please see my above response on why that doesnt solve the issue

@sorenwacker
Copy link
Author

@sorenwacker sorenwacker commented Sep 19, 2019

It would be possible to export the env without over-specific version numbers to make it machine un-specific?

@forrestwaters
Copy link

@forrestwaters forrestwaters commented Sep 19, 2019

@soerendip there is a new flag in conda 4.7.12 that allows environments to be exported based on the history of what packages were explicitly requested (versions are only included if you explicitly specified a version when installing that package). Please try:

conda env export --from-history

@ashleymaeconard
Copy link

@ashleymaeconard ashleymaeconard commented Oct 11, 2019

Had that issue and solved it by first conda update --all on the host sytem, then conda export --no-builds > env.yml and regular installation via conda env create -f env.yml on the remote machine.

Guess for me the problem was that the base env conda on remote had packages installed that were not working with the older packages on the host system.

I believe you meant conda env export --no-builds > env.yml

@hardianlawi
Copy link

@hardianlawi hardianlawi commented Nov 20, 2019

@forrestwaters conda env export --from-history doesn't seem to export dependencies installed using pip

@windspirit95
Copy link

@windspirit95 windspirit95 commented Mar 11, 2020

https://docs.conda.io/projects/conda/en/latest/user-guide/tasks/manage-environments.html
https://repo.anaconda.com/pkgs/
These already help me to reconfigure the environments file. Maybe you can check again whether those packages exist or not.
In my case is Win64, after creating myenv follow the guide in first link, the myenv folder will be at Anaconda3\envs\myenv. You can search for .json files inside that folder.

@Oktai15
Copy link

@Oktai15 Oktai15 commented Mar 19, 2020

To sum up, working solution (without typo):
conda env export --no-builds > env.yml
conda env create -f env.yml

@Xinxin-Lai
Copy link

@Xinxin-Lai Xinxin-Lai commented Jun 7, 2020

conda env create -f env.yml

Hi, I have tried your command line, but it still gave me such an error:

CondaEnvException: Unable to determine environment

Please re-run this command with one of the following options:

* Provide an environment name via --name or -n
* Re-run this command inside an activated conda environment.

Any help would be appreciated, thanks!

Sophia

abegehr added a commit to abegehr/ComE_BGMM that referenced this issue Jul 21, 2020
conda env export --no-builds > environment.yml
ContinuumIO/anaconda-issues#9480 (comment)
@giuliamoscoso
Copy link

@giuliamoscoso giuliamoscoso commented Nov 3, 2020

For those who were still lost about this, I solved mine by importing the environment with Anaconda Navigator, it works :)

@DanielIAvila
Copy link

@DanielIAvila DanielIAvila commented Dec 7, 2020

The general idea is to make it work from anaconda prompt not to import it. Does anyone have updates on this?

@paapu88
Copy link

@paapu88 paapu88 commented Jan 22, 2021

I got this:

conda env create -f environment.yaml
Collecting package metadata (repodata.json): done
Solving environment: failed
ResolvePackageNotFound: 
  - libgfortran-ng==7.5.0

Problem solved by removing '=hdf63c60_6' in file environment.yaml from the original line
- libgfortran-ng=7.5.0=hdf63c60_6

@skwde
Copy link

@skwde skwde commented Feb 22, 2021

To sum up, the following (as suggested above) still doesn't solve the issue for me:

conda env export --no-builds > test.yaml
conda env create --name test --name test --file test.yaml

Results in ResolvePackageNotFound, removing the problematic packages from test.yaml still finds conflicts and fails after trying to resolve them.

Note the same is true when trying to create the environment via

# after adjusting prefix and name in test.yaml
conda env create -f test.yaml

@mcan06
Copy link

@mcan06 mcan06 commented Jun 28, 2021

One could also try adding a conda channel if the said packages are not present in the default channels. E.g. qutip package appears in conda-forge channel. One can add the channel by typing the following command.
conda config --append channels conda-forge Off course one needs to replace the name conda-forge by the appropriate channel name.

What do you mean in hear, could you expand your words for qutip?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet