Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

solving environment for 6 hours #7690

Closed
s-sajid-ali opened this issue Aug 25, 2018 · 37 comments
Closed

solving environment for 6 hours #7690

s-sajid-ali opened this issue Aug 25, 2018 · 37 comments

Comments

@s-sajid-ali
Copy link

Current Behavior

Conda install hangs at solving environment.

Steps to Reproduce

(py27) conda install -c conda-forge boost
Solving environment: \ | ^Z

Expected Behavior

Conda solves the environment and installs the package.

Environment Information

`conda info`

(py27) conda info

     active environment : py27
    active env location : /home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/miniconda3-4.5.4-cn4y5gakwtozuu6thdozzmdekeo36qdg/envs/py27
            shell level : 1
       user config file : /home/sajid/.condarc
 populated config files : /home/sajid/.condarc
          conda version : 4.5.4
    conda-build version : not installed
         python version : 3.6.5.final.0
       base environment : /home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/miniconda3-4.5.4-cn4y5gakwtozuu6thdozzmdekeo36qdg  (writable)
           channel URLs : https://conda.anaconda.org/intel/linux-64
                          https://conda.anaconda.org/intel/noarch
                          https://repo.anaconda.com/pkgs/main/linux-64
                          https://repo.anaconda.com/pkgs/main/noarch
                          https://repo.anaconda.com/pkgs/free/linux-64
                          https://repo.anaconda.com/pkgs/free/noarch
                          https://repo.anaconda.com/pkgs/r/linux-64
                          https://repo.anaconda.com/pkgs/r/noarch
                          https://repo.anaconda.com/pkgs/pro/linux-64
                          https://repo.anaconda.com/pkgs/pro/noarch
                          https://conda.anaconda.org/conda-forge/linux-64
                          https://conda.anaconda.org/conda-forge/noarch
          package cache : /home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/miniconda3-4.5.4-cn4y5gakwtozuu6thdozzmdekeo36qdg/pkgs
                          /home/sajid/.conda/pkgs
       envs directories : /home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/miniconda3-4.5.4-cn4y5gakwtozuu6thdozzmdekeo36qdg/envs
                          /home/sajid/.conda/envs
               platform : linux-64
             user-agent : conda/4.5.4 requests/2.18.4 CPython/3.6.5 Linux/3.10.0-693.el7.x86_64 centos/7 glibc/2.17
                UID:GID : 1003:1003
             netrc file : None
           offline mode : False

`conda config --show-sources`

(py27) conda config --show-sources
==> /home/sajid/.condarc <==
channels:
  - intel
  - defaults
  - conda-forge
report_errors: True

`conda list --show-channel-urls`

# packages in environment at /home/sajid/packages/spack/opt/spack/linux-centos7-x86_64/gcc-4.8.5/miniconda3-4.5.4-cn4y5gakwtozuu6thdozzmdekeo36qdg/envs/py27:
#
# Name                    Version                   Build  Channel
alabaster                 0.7.11                   py27_0    defaults
anaconda                  custom           py27h4a00acb_0    defaults
anaconda-client           1.7.1                    py27_0    defaults
anaconda-navigator        1.8.7                    py27_0    defaults
anaconda-project          0.8.2                    py27_0    defaults
appdirs                   1.4.3            py27h28b3542_0    defaults
argparse                  1.4.0                     <pip>
asn1crypto                0.24.0                   py27_0    defaults
astroid                   1.6.5                    py27_0    defaults
astropy                   2.0.8            py27h035aef0_0    defaults
atomicwrites              1.1.5                    py27_0    defaults
attrs                     18.1.0                   py27_0    defaults
automat                   0.7.0                    py27_0    defaults
babel                     2.6.0                    py27_0    defaults
backports                 1.0                      py27_1    defaults
backports.functools_lru_cache 1.5                      py27_1    defaults
backports.shutil_get_terminal_size 1.0.0                    py27_2    defaults
backports_abc             0.5                      py27_0    defaults
beautifulsoup4            4.6.3                    py27_0    defaults
bitarray                  0.8.3            py27h14c3975_0    defaults
bkcharts                  0.2                      py27_0    defaults
blas                      1.0                         mkl    defaults
blaze                     0.11.3                   py27_0    defaults
bleach                    2.1.3                    py27_0    defaults
blinker                   1.4                      py27_0    defaults
blosc                     1.14.4               hdbcaa40_0    defaults
bokeh                     0.13.0                   py27_0    defaults
boto                      2.49.0                   py27_0    defaults
boto3                     1.7.62                   py27_1    defaults
botocore                  1.10.62                  py27_0    defaults
bottleneck                1.2.1            py27h035aef0_1    defaults
bz2file                   0.98                     py27_1    defaults
bzip2                     1.0.6                h14c3975_5    defaults
ca-certificates           2018.03.07                    0    defaults
cairo                     1.14.12              h8948797_3    defaults
cdecimal                  2.3              py27h14c3975_3    defaults
certifi                   2018.8.13                py27_0    defaults
cffi                      1.11.5           py27h9745a5d_0    defaults
chardet                   3.0.4                    py27_1    defaults
clangdev                  6.0.0                 default_0    conda-forge
click                     6.7                      py27_0    defaults
cloudpickle               0.5.3                    py27_0    defaults
clyent                    1.2.2                    py27_1    defaults
colorama                  0.3.9                    py27_0    defaults
configparser              3.5.0                    py27_0    defaults
constantly                15.1.0           py27h28b3542_0    defaults
contextlib2               0.5.5                    py27_0    defaults
cryptography              2.3              py27hb7f436b_0    defaults
cryptography-vectors      2.3                      py27_0    defaults
cudatoolkit               9.0                  h13b8566_0    defaults
curl                      7.61.0               h84994c4_0    defaults
cycler                    0.10.0                   py27_0    defaults
cython                    0.28.5           py27hf484d3e_0    defaults
cytoolz                   0.9.0.1          py27h14c3975_1    defaults
daal                      2018.0.3.20180405               0    intel
dask                      0.18.2                   py27_0    defaults
dask-core                 0.18.2                   py27_0    defaults
datashape                 0.5.4                    py27_1    defaults
dbus                      1.13.2               h714fa37_1    defaults
decorator                 4.3.0                    py27_0    defaults
distributed               1.22.1                   py27_0    defaults
docutils                  0.14                     py27_0    defaults
entrypoints               0.2.3                    py27_2    defaults
enum34                    1.1.6                    py27_1    defaults
et_xmlfile                1.0.1                    py27_0    defaults
expat                     2.2.5                he0dffb1_0    defaults
expresso                  0.2                       <pip>
fastcache                 1.0.2            py27h14c3975_2    defaults
filelock                  3.0.4                    py27_0    defaults
flask                     1.0.2                    py27_1    defaults
flask-cors                3.0.6                    py27_0    defaults
fontconfig                2.13.0               h9420a91_0    defaults
freetype                  2.9.1                h8a8886c_0    defaults
fribidi                   1.0.4                h14c3975_0    defaults
funcsigs                  1.0.2                    py27_0    defaults
functools32               3.2.3.2                  py27_1    defaults
futures                   3.2.0                    py27_0    defaults
gensim                    3.4.0            py27h14c3975_0    defaults
get_terminal_size         1.0.0                haa9412d_0    defaults
gevent                    1.3.5            py27h14c3975_0    defaults
git                       2.18.0          pl526hb37396a_0    defaults
glib                      2.56.1               h000015b_0    defaults
glob2                     0.6                      py27_0    defaults
gmp                       6.1.2                h6c8ec71_1    defaults
gmpy2                     2.0.8            py27h10f8cd9_2    defaults
graphite2                 1.3.11               h16798f4_2    defaults
greenlet                  0.4.14           py27h14c3975_0    defaults
grin                      1.2.1                    py27_4    defaults
gst-plugins-base          1.14.0               hbbd80ab_1    defaults
gstreamer                 1.14.0               hb453b48_1    defaults
h5py                      2.8.0            py27h8d01980_0    defaults
harfbuzz                  1.8.4                hec2c2bc_0    defaults
hdf5                      1.10.2               hba1933b_1    defaults
heapdict                  1.0.0                    py27_2    defaults
html5lib                  1.0.1                    py27_0    defaults
hyperlink                 18.0.0                   py27_0    defaults
icc_rt                    2018.0.3                intel_0  [intel]  intel
icu                       58.2                 h9c2bf20_1    defaults
idna                      2.7                      py27_0    defaults
imageio                   2.3.0                    py27_0    defaults
imagesize                 1.0.0                    py27_0    defaults
impi_rt                   2018.0.3                intel_0  [intel]  intel
incremental               17.5.0                   py27_0    defaults
intel-openmp              2018.0.3                      0    defaults
intelpython               2018.0.3                      0    intel
ipaddress                 1.0.22                   py27_0    defaults
ipykernel                 4.8.2                    py27_0    defaults
ipython                   5.8.0                    py27_0    defaults
ipython_genutils          0.2.0                    py27_0    defaults
ipywidgets                7.4.0                    py27_0    defaults
islpy                     2018.1                   py27_0    conda-forge
isort                     4.3.4                    py27_0    defaults
itsdangerous              0.24                     py27_1    defaults
jbig                      2.1                  hdba287a_0    defaults
jdcal                     1.4                      py27_0    defaults
jedi                      0.12.1                   py27_0    defaults
jinja2                    2.10                     py27_0    defaults
jmespath                  0.9.3                    py27_0    defaults
jpeg                      9b                   h024ee3a_2    defaults
jsonschema                2.6.0                    py27_0    defaults
jupyter                   1.0.0                    py27_4    defaults
jupyter_client            5.2.3                    py27_0    defaults
jupyter_console           5.2.0                    py27_1    defaults
jupyter_core              4.4.0                    py27_0    defaults
jupyterlab                0.33.11                  py27_0    defaults
jupyterlab_launcher       0.11.2                   py27_0    defaults
keyring                   13.2.1                   py27_0    defaults
kiwisolver                1.0.1            py27hf484d3e_0    defaults
lazy-object-proxy         1.3.1            py27h14c3975_2    defaults
libcurl                   7.61.0               h1ad7b7a_0    defaults
libedit                   3.1.20170329         h6b74fdf_2    defaults
libffi                    3.2.1                hd88cf55_4    defaults
libgcc                    7.2.0                h69d50b8_2    defaults
libgcc-ng                 8.2.0                hdf63c60_1    defaults
libgfortran               3.0.0                         1    anaconda
libgfortran-ng            7.3.0                hdf63c60_0    defaults
libhwloc                  1.11.9                        0    conda-forge
libiconv                  1.15                 h63c8f33_5    defaults
libpng                    1.6.34               hb9fc6fc_0    defaults
libsodium                 1.0.16               h1bed415_0    defaults
libssh2                   1.8.0                h9cfc8f7_4    defaults
libstdcxx-ng              8.2.0                hdf63c60_1    defaults
libtiff                   4.0.9                he85c1e1_1    defaults
libtool                   2.4.6                h544aabb_3    defaults
libuuid                   1.0.3                   intel_2  [intel]  intel
libxcb                    1.13                 h1bed415_1    defaults
libxml2                   2.9.8                h26e45fe_1    defaults
libxslt                   1.1.32               h1312cb7_0    defaults
linecache2                1.0.0                    py27_0    defaults
llvm-meta                 6.0.0                         0    conda-forge
llvmdev                   6.0.0                h1a6f6a4_4    defaults
llvmlite                  0.24.0           py27hdbcaa40_0    defaults
locket                    0.2.0                    py27_1    defaults
lxml                      4.2.4            py27hf71bdeb_0    defaults
lzo                       2.10                 h49e0be7_2    defaults
mako                      1.0.7                    py27_0    defaults
markupsafe                1.0              py27h14c3975_1    defaults
matplotlib                2.2.3            py27hb69df0a_0    defaults
mccabe                    0.6.1                    py27_1    defaults
mistune                   0.8.3            py27h14c3975_1    defaults
mkl                       2018.0.3                      1    defaults
mkl-service               1.1.2            py27h651fb7a_4    defaults
mkl_fft                   1.0.4            py27h4414c95_1    defaults
mkl_random                1.0.1            py27h4414c95_1    defaults
more-itertools            4.3.0                    py27_0    defaults
mpc                       1.1.0                h10f8cd9_1    defaults
mpfr                      4.0.1                hdf1c602_3    defaults
mpi                       1.0                       mpich    conda-forge
mpi4py                    2.0.0                    py27_2    defaults
mpich                     3.2.1                h26a2512_4    conda-forge
mpich2                    1.4.1p1                       0    defaults
mpmath                    1.0.0                    py27_2    defaults
msgpack-python            0.5.6            py27h6bb024c_1    defaults
multipledispatch          0.6.0                    py27_0    defaults
navigator-updater         0.2.1                    py27_0    defaults
nbconvert                 5.3.1                    py27_0    defaults
nbformat                  4.4.0                    py27_0    defaults
ncurses                   6.1                  hf484d3e_0    defaults
networkx                  2.1                      py27_0    defaults
nltk                      3.3.0                    py27_0    defaults
nose                      1.3.7                    py27_2    defaults
notebook                  5.6.0                    py27_0    defaults
numba                     0.39.0           py27h04863e7_0    defaults
numexpr                   2.6.7            py27h00253c6_0    defaults
numpy                     1.14.5           py27h1b885b7_4    defaults
numpy-base                1.14.5           py27hdbf6ddf_4    defaults
numpydoc                  0.8.0                    py27_0    defaults
oauthlib                  2.1.0                    py27_0    defaults
ocl-icd                   2.2.9                h470a237_4    conda-forge
odo                       0.5.1                    py27_0    defaults
olefile                   0.45.1                   py27_0    defaults
openmp                    2018.0.3                intel_0    intel
openpyxl                  2.5.5                    py27_0    defaults
openssl                   1.0.2p               h14c3975_0    defaults
packaging                 17.1                     py27_0    defaults
pandas                    0.23.4           py27h04863e7_0    defaults
pandoc                    2.2.1                h629c226_0    defaults
pandocfilters             1.4.2                    py27_1    defaults
pango                     1.42.3               h8589676_0    defaults
parso                     0.3.1                    py27_0    defaults
partd                     0.3.8                    py27_0    defaults
patchelf                  0.9                  hf484d3e_2    defaults
path.py                   11.0.1                   py27_0    defaults
pathlib2                  2.3.2                    py27_0    defaults
patsy                     0.5.0                    py27_0    defaults
pcre                      8.42                 h439df22_0    defaults
pep8                      1.7.1                    py27_0    defaults
perl                      5.26.2               h14c3975_0    defaults
pexpect                   4.6.0                    py27_0    defaults
pickleshare               0.7.4                    py27_0    defaults
pillow                    5.2.0            py27heded4f4_0    defaults
pip                       10.0.1                   py27_0    defaults
pixman                    0.34.0               hceecf20_3    defaults
pkginfo                   1.4.2                    py27_1    defaults
pluggy                    0.7.1                    py27_0    defaults
ply                       3.11                     py27_0    defaults
pocl                      1.1                           2    conda-forge
prometheus_client         0.3.1                    py27_0    defaults
prompt_toolkit            1.0.15                   py27_0    defaults
psutil                    5.4.6            py27h14c3975_0    defaults
pthread-stubs             0.3                  h0ce48e5_1    defaults
ptyprocess                0.6.0                    py27_0    defaults
py                        1.5.4                    py27_0    defaults
pyasn1                    0.4.4                    py27_0    defaults
pyasn1-modules            0.2.2                    py27_0    defaults
pycairo                   1.15.4           py27h1b9232e_1    defaults
pycodestyle               2.4.0                    py27_0    defaults
pycosat                   0.6.3            py27h14c3975_0    defaults
pycparser                 2.18                     py27_1    defaults
pycrypto                  2.6.1            py27h14c3975_9    defaults
pycurl                    7.43.0.2         py27hb7f436b_0    defaults
pydaal                    2018.0.3.20180405 np114py27_intel_0  [intel]  intel
pyfftw                    0.10.4           py27h7eb728f_3    conda-forge
pyflakes                  2.0.0                    py27_0    defaults
pygments                  2.2.0                    py27_0    defaults
PyHamcrest                1.9.0                     <pip>
pyjwt                     1.6.4                    py27_0    defaults
pylint                    1.9.2                    py27_0    defaults
pyodbc                    4.0.23           py27hf484d3e_0    defaults
pyopencl                  2018.1.1                 py27_2    conda-forge
pyopenssl                 18.0.0                   py27_0    defaults
pyparsing                 2.2.0                    py27_1    defaults
pyqt                      5.9.2            py27h22d08a2_0    defaults
pysocks                   1.6.8                    py27_0    defaults
pytables                  3.4.4            py27ha205bf6_0    defaults
pytest                    3.7.1                    py27_0    defaults
python                    2.7.15               h1571d57_0    defaults
python-crfsuite           0.9.5            py27h6bb024c_0    defaults
python-dateutil           2.7.3                    py27_0    defaults
pytools                   2018.5.2                   py_0    conda-forge
pytz                      2018.5                   py27_0    defaults
pywavelets                0.5.2            py27h035aef0_2    defaults
pyyaml                    3.13             py27h14c3975_0    defaults
pyzmq                     17.1.2           py27h14c3975_0    defaults
qt                        5.9.6                h52aff34_0    defaults
qtawesome                 0.4.4                    py27_0    defaults
qtconsole                 4.4.0                    py27_0    defaults
qtpy                      1.4.2                    py27_1    defaults
quantities                0.12.2             pyh24bf2e0_0    conda-forge
readline                  7.0                  ha6073c6_4    defaults
requests                  2.19.1                   py27_0    defaults
requests-oauthlib         1.0.0                    py27_1    defaults
rope                      0.11.0                   py27_0    defaults
ruamel_yaml               0.15.46          py27h14c3975_0    defaults
s3transfer                0.1.13                   py27_0    defaults
scandir                   1.9.0            py27h14c3975_0    defaults
scikit-image              0.14.0           py27hf484d3e_1    defaults
scikit-learn              0.19.1           py27hedc7406_0    defaults
scipy                     1.1.0            py27hc49cb51_0    defaults
seaborn                   0.9.0                    py27_0    defaults
SecretStorage             2.3.1                     <pip>
send2trash                1.5.0                    py27_0    defaults
service_identity          17.0.0           py27h28b3542_0    defaults
setuptools                40.0.0                   py27_0    defaults
simplegeneric             0.8.1                    py27_2    defaults
singledispatch            3.4.0.3                  py27_0    defaults
sip                       4.19.8           py27hf484d3e_0    defaults
six                       1.11.0                   py27_1    defaults
smart_open                1.6.0                    py27_0    defaults
snappy                    1.1.7                   intel_0  [intel]  intel
snowballstemmer           1.2.1                    py27_0    defaults
sortedcollections         1.0.1                    py27_0    defaults
sortedcontainers          2.0.4                    py27_0    defaults
sphinx                    1.7.6                    py27_0    defaults
sphinxcontrib             1.0                      py27_1    defaults
sphinxcontrib-websupport  1.1.0                    py27_1    defaults
spyder                    3.3.1                    py27_1    defaults
spyder-kernels            0.2.6                    py27_0    defaults
sqlalchemy                1.2.10           py27h14c3975_0    defaults
sqlite                    3.24.0               h84994c4_0    defaults
ssl_match_hostname        3.5.0.1                  py27_2    defaults
statsmodels               0.9.0            py27h035aef0_0    defaults
subprocess32              3.5.2            py27h14c3975_0    defaults
sympy                     1.2                      py27_0    defaults
system                    5.8                           2    anaconda
tbb                       2018.0.4             h6bb024c_1    defaults
tblib                     1.3.2                    py27_0    defaults
tcl                       8.6.4                  intel_19  [intel]  intel
terminado                 0.8.1                    py27_1    defaults
testpath                  0.3.1                    py27_0    defaults
tk                        8.6.7                hc745277_3    defaults
toolz                     0.9.0                    py27_0    defaults
tornado                   5.1              py27h14c3975_0    defaults
traceback2                1.4.0                    py27_0    defaults
traitlets                 4.3.2                    py27_0    defaults
twisted                   18.7.0           py27h14c3975_1    defaults
twython                   3.7.0                    py27_0    defaults
typing                    3.6.4                    py27_0    defaults
unicodecsv                0.14.1                   py27_0    defaults
unittest2                 1.1.0                    py27_0    defaults
unixodbc                  2.3.6                h1bed415_0    defaults
urllib3                   1.23                     py27_0    defaults
wcwidth                   0.1.7                    py27_0    defaults
webencodings              0.5.1                    py27_1    defaults
werkzeug                  0.14.1                   py27_0    defaults
wheel                     0.31.1                   py27_0    defaults
widgetsnbextension        3.4.0                    py27_0    defaults
wrapt                     1.10.11          py27h14c3975_2    defaults
xlrd                      1.1.0                    py27_1    defaults
xlsxwriter                1.0.5                    py27_0    defaults
xlwt                      1.3.0                    py27_0    defaults
xorg-libxau               1.0.8                h470a237_6    conda-forge
xorg-libxdmcp             1.1.2                h470a237_7    conda-forge
xz                        5.2.4                h14c3975_4    defaults
yaml                      0.1.7                had09818_2    defaults
zeromq                    4.2.5                hf484d3e_0    defaults
zict                      0.1.3                    py27_0    defaults
zlib                      1.2.11               ha838bed_2    defaults
zope                      1.0                      py27_0    defaults
zope.interface            4.5.0            py27h14c3975_0    defaults

@s-sajid-ali s-sajid-ali mentioned this issue Aug 25, 2018
2 tasks
@opoplawski
Copy link
Contributor

I'm seeing the same - also with the intel channel, though not conda-forge. Dropping defaults and just using "main" seems to have helped.

@beilpig
Copy link

beilpig commented Nov 30, 2018

I experienced similar issue installing cuda development packages to fresh install - fixed it by creating new environment and activating it then installing new packages there without issue.

@rsjeffers
Copy link

This is also an issue for me now that I am trying to install mox :(

@opoplawski what do you mean by just using main? (still a beginner with conda)

Thank you in advance :)

@opoplawski
Copy link
Contributor

@jeffginger by default conda will search a few different channels as shown by "conda info". If you can limit yourself to just some of those you may be able to speed up the depsolving. But this is still a huge pain for us - I see conda update all in a python2 environment taking between 30 and 290 minutes on fast hardware.

@opoplawski
Copy link
Contributor

So to be more explicit, you could try something like:

conda install --override-channels -c main -c conda-forge boost

or

conda install --override-channels -c main package_in_main

I really think that some channels just need to start pruning some old packages.

@Fireflaker
Copy link

Same issue here, after installing conda-forge boost.

conda install --override-channels -c main package_in_main did not work. It said
(User) C:\Users\Administrator>conda install --override-channels -c main package_in_main
Collecting package metadata: done
Solving environment: failed

in like 3 seconds.

@rvalenzuelar
Copy link

Same issue here when using:

conda create -n opencv_py3 -c conda-forge python opencv xarray cartopy ipython matplotlib pandas fiona descartes x264=20131218

Solving environment is taking forever. It's been there for at least 15-20 minutes.

@rvalenzuelar
Copy link

Same issue here when using:

conda create -n opencv_py3 -c conda-forge python opencv xarray cartopy ipython matplotlib pandas fiona descartes x264=20131218

Solving environment is taking forever. It's been there for at least 15-20 minutes.

It ended up taking ~20 minutes. Probably not an issue given the long list of modules.

@rajasekarv
Copy link

same here:
conda install -c anaconda keras-gpu
I let it run overnight. Still solving environment.

@lijecaru
Copy link

lijecaru commented Mar 6, 2019

I had the same problem. I cancelled the installation and used the conda navigator instead: anaconda-navigator

@yvesgrenier
Copy link

yvesgrenier commented Mar 7, 2019

I have the same problem: "solving the enviroment" is still continuing after 40 minutes.
Moreover, I cannot use anaconda-navigator but this is another problem (or not?):

  File "/home/grenier/anaconda2/bin/anaconda-navigator", line 7, in <module>
    from anaconda_navigator.app.main import main
  File "/home/grenier/anaconda2/lib/python2.7/site-packages/anaconda_navigator/app/main.py", line 22, in <module>
    from anaconda_navigator.utils.conda import is_conda_available
  File "/home/grenier/anaconda2/lib/python2.7/site-packages/anaconda_navigator/utils/__init__.py", line 15, in <module>
    from qtpy.QtGui import QIcon
  File "/home/grenier/anaconda2/lib/python2.7/site-packages/qtpy/QtGui.py", line 20, in <module>
    from PyQt5.QtGui import *
ImportError: /usr/lib/x86_64-linux-gnu/libharfbuzz.so.0: undefined symbol: FT_Get_Var_Blend_Coordinates

@matanox
Copy link

matanox commented Mar 13, 2019

I observe also #7239 which has been closed for commenting. I understand this is one priority issue for the development team. However I would be really happy if someone could say what are currently the factors affecting the duration ― does it depend on which package you are trying to install, your choice of channels (e.g. conda-forge), or other factors altogether.

@msarahan
Copy link
Contributor

https://www.anaconda.com/understanding-and-improving-condas-performance/

For solve times beyond one or two minutes, it is almost always some problem with metadata that leads conda into awkward loops where there is not a clear "right" answer. For example, things on defaults are built with openssl 1.1.1, whereas things from conda-forge are only starting to be available with that. Because conda-forge doesn't have a complete set of packages using openssl 1.1.1, conda can get into awkward solutions that end up not really converging. Strict channel priority is helpful for forcing these situations to error out quickly instead of churning, though the error messages are not always very helpful.

If you identify packages that create problems, it is helpful to file issues using the "speed complaint" template. Even if there's nothing conda can do, we can perhaps help fix the metadata in our packages or on conda-forge so that other people don't get stuck.

@dgg32
Copy link

dgg32 commented Mar 20, 2019

same here:
conda install -c anaconda keras-gpu
I let it run overnight. Still solving environment.

I had this exact problem. But then I created an empty env and went into it to run the same command, it let me install it after a short wait.

@megies
Copy link
Contributor

megies commented Apr 12, 2019

I'm starting to think that dropping all default channels and only using conda-forge might be the only solution. It's weird if you think about it -- using Anaconda but not using Anaconda official channels -- but nowadays I find myself uninstalling and reinstalling complete envs just because I want one additional package (and solver for conda install ... goes on forever).

@msarahan
Copy link
Contributor

Did you see the blog post at https://www.anaconda.com/understanding-and-improving-condas-performance/ ?

Have you tried the suggestions there?

Rather than dropping defaults completely, you should first try redefining default_channels in condarc to exclude "free" - that particular part of defaults is probably the issue.

I have a talk on this at Anacondacon. Hopefully the recording will be up soon and hopefully I can put my slides up somewhere.

@opoplawski
Copy link
Contributor

Using --strict-channel-priority seems to help me out a lot with speed. Though it's going to require reworking how I've configured various environments.

@fspaolo
Copy link

fspaolo commented May 17, 2019

How is this issue still happening?! Having to search online and read a ton of text just to understand what's going on with the hanging "Solving environment" is unacceptable in a production environment.

Is there a simple solution to this (i.e. to hit conda install xxx and get the package installed right away)? If not, I will have to drop conda from all our systems...

@rvalenzuelar
Copy link

How is this issue still happening?! Having to search online and read a ton of text just to understand what's going on with the hanging "Solving environment" is unacceptable in a production environment.

Is there a simple solution to this (i.e. to hit conda install xxx and get the package installed right away)? If not, I will have to drop conda from all our systems...

I've been using a simple conda install [package] and it seems faster. Not 100% sure because I have not performed any experiment.

@msarahan
Copy link
Contributor

I'm sorry there's been so much discussion. We attempted to consolidate information at https://www.anaconda.com/understanding-and-improving-condas-performance/.

There is no single simple solution to this because there are many ways the problem can happen. Efforts to improve things are ongoing. Conda 4.7.0 was just released on the conda-canary channel, and it might improve the situation for you.

conda update -c conda-canary conda

@fspaolo
Copy link

fspaolo commented May 17, 2019

Also, let me add that I am trying the suggested Speed_complaint.md and the CONDA_INSTRUMENTATION_ENABLED=1 <your conda command> -vv | tee conda_debug_output.txt itself is taking forever. I will have to kill it.

@msarahan
Copy link
Contributor

Yes, that can't go any faster than conda otherwise would. Hopefully it's telling you something about what part of conda is getting hung, though.

@caroso1222
Copy link

I can confirm that creating a new env "fixed" the issue. Not sure if this is relevant, but the first environment I created was with python=3.5. I later realized the base environment was on 3.7 so I just created my new one with 3.7 as well, this one worked fine. conda create -n myenv python=3.7.

@michaelaye
Copy link

can confirm that conda 4.7, installed as advised above, reduced solving time by many factors for me on my CentOS 6.x.

@msarahan
Copy link
Contributor

Glad to hear that. I just uploaded 4.7.2 to conda-canary. It fixes a few things that earlier releases broke. I think this one might actually make it out of canary.

@msarahan
Copy link
Contributor

msarahan commented Jun 11, 2019

Some early benchmarks:

with @rvalenzuelar example:

conda create -n opencv_py3 -c conda-forge python opencv xarray cartopy ipython matplotlib pandas fiona descartes x264=20131218

I changed the x264 to be unpinned, so that conda 4.7.2 can use its smaller package collection of only the latest versions.

conda create -n opencv_py3 -c conda-forge python opencv xarray cartopy ipython matplotlib pandas fiona descartes x264

This took 320s with conda 4.6.14, and 39s with conda 4.7.2.

If you specify the minor version for python, things get a whole lot faster:

conda create -n opencv_py3 -c conda-forge python=3.7 opencv xarray cartopy ipython matplotlib pandas fiona descartes x264

This takes 30s on conda 4.6.14, and 22s on conda 4.7.2

@michaelaye
Copy link

@msarahan did you forget to add the time for the first example? And what is this x264 wizardry? I'd like to know more about that.

@michaelaye
Copy link

Ah, x264 is just a package name, and the equal sign is just pinning it to that date? Didn't know date is working instead of version numbers.

@msarahan
Copy link
Contributor

It's not x264 wizardry. It's that conda-build is creating a separate index with only the latest versions of everything. That's a lot smaller. Conda 4.7 can use it, but only when all the specs are satisfiable with that new smaller index. Otherwise, conda falls back to the full index (and is then slower because it has wasted time)

@msarahan
Copy link
Contributor

It doesn't. That's a terrible package where they version by date.

@michaelaye
Copy link

oh nevermind. ;)

@michaelaye
Copy link

where should I put some problem reports with conda 4.7.2?

@msarahan
Copy link
Contributor

open new issues, please

@smcinerney
Copy link

If this issue has been solved, suggest you close this bug and add very explicit recommendations in the doc to a) upgrade to 4.7.2+ b) recommend users never put both Python and R in the same/base env or channel c) The doc needs tips on which channels to subscribe to, pinning, and how to partition envs to keep Python and R separate, and how to debug performance issues in envs and channels. The bottom line is that the out-of-the-box performance should never get worse, and there should be unit-tests for that.

If you specify the minor version for python, things get a whole lot faster [10x on 4.6.14, 1.5x on 4.7.2]

Isn't that a serious bug? Why can't Python 3 just simply default to 3.7 (switch to 3.8 when it comes out 10/2019)? If people want 3.6 or earlier they can specify it. Make the common case fast, by default.

@msarahan
Copy link
Contributor

add very explicit recommendations in the doc

which doc?

a) upgrade to 4.7.2+

conda 4.7.2 is on our early release channel. I recommend that people test it, but it has not been released for general availability yet. It can't go into docs until then.

b) recommend users never put both Python and R in the same/base env or channel

This is an arbitrary constraint that has underlying reasons that are more relevant. Mismatching dependencies are the underlying problem - say rstudio needing a different Qt version than matplotlib. Those are things that we hope our improved conflict messaging will help explain.

c) The doc needs tips on which channels to subscribe to, pinning, and how to partition envs to keep Python and R separate, and how to debug performance issues in envs and channels.

conda can't and won't endorse channels other than the official "defaults" channel that conda ships with. We refer people to conda-forge as the de-facto community channel. Partitioning R and Python is not necessary, as I mentioned above. That's a false, or at least oversimplified, solution. Debugging performance issues is the topic of the blog post I referenced above, and there will be another blog post on how conda 4.7.2 has improved things.

The bottom line is that the out-of-the-box performance should never get worse, and there should be unit-tests for that.

The out-of-the-box performance will always get worse, because the size of the ecosystem is growing. The problem size is always getting larger. I also reject "performance should never get worse", because there are things that are important enough that a speed tradeoff is worthwhile.

Isn't that a serious bug? Why can't Python 3 just simply default to 3.7 (switch to 3.8 when it comes out 10/2019)? If people want 3.6 or earlier they can specify it. Make the common case fast, by default.

That's a misunderstanding of how the solver works. The "solver" is an optimizer that involves solving at many steps. If you feed it broad specs, you make the problem harder. If you force python 3.x by default in any way when the user hasn't specified it, you are cutting out solutions that may be optimal in other ways. Conda does default to the python version that matches your root environment, but it does so in a way that does not preclude other python versions from being possible answers. I think that's important, because some packages may only be available for use with an earlier python, and if we're defaulting to python 3.7, people will think the environment is just unsatisfiable by default. Explicit is better than implicit.

@zilben
Copy link

zilben commented Jul 16, 2019

The following line (as suggested in earlier posts) solved this issue for me:
:)

conda install --override-channels -c main -c anaconda pandasql

The line below would get stuck in 'solving environment':

conda install -c anaconda pandasql

even after running:

conda config --set channel_priority strict

@msarahan
Copy link
Contributor

conda 4.7 has hopefully dramatically improved the situation. Please avoid -c anaconda - that negates the improvement that conda 4.7 made with removing the free channel. Closing this - if there are any new speed issues, please file a new issue using the speed complaint template.

@conda conda locked as resolved and limited conversation to collaborators Jul 23, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests