conda update anaconda vs. conda update --all

[sensharma]

Hi, when I try conda update anaconda, it gives me the following:

The following packages will be downloaded:

package	build
hdf5-1.8.15.1	1 1.5 MB
llvmlite-0.5.0	py34_0 5.9 MB
bcolz-0.9.0	np19py34_0 393 KB
bottleneck-1.0.0	np19py34_0 153 KB
numba-0.19.1	np19py34_0 1.1 MB
numexpr-2.4.3	np19py34_p0 106 KB
blz-0.6.2	np19py34_1 327 KB
xlwt-1.0.0	py34_0 158 KB
h5py-2.5.0	np19py34_3 688 KB
pytables-3.2.0	np19py34_0 1.6 MB
anaconda-2.3.0	np19py34_0 3 KB

------------------------------------------------------------
                                       Total:        11.8 MB

The following NEW packages will be INSTALLED:

bottleneck:   1.0.0-np19py34_0  
xlwt:         1.0.0-py34_0      

The following packages will be UPDATED:

anaconda:     2.2.0-np19py34_0   --> 2.3.0-np19py34_0  
bcolz:        0.8.1-np19py34_0   --> 0.9.0-np19py34_0  
blz:          0.6.2-np19py34_0   --> 0.6.2-np19py34_1  
h5py:         2.5.0-np19py34_2   --> 2.5.0-np19py34_3  
hdf5:         1.8.14-0           --> 1.8.15.1-1        
llvmlite:     0.4.0-py34_0       --> 0.5.0-py34_0      
numba:        0.18.2-np19py34_1  --> 0.19.1-np19py34_0 
numexpr:      2.3.1-np19py34_p0  [mkl] --> 2.4.3-np19py34_p0  [mkl]
pytables:     3.1.1-np19py34_2   --> 3.2.0-np19py34_0  
scikit-learn: 0.15.2-np19py34_p0 [mkl] --> 0.16.1-np19py34_p0 [mkl]

The following packages will be DOWNGRADED:

bokeh:        0.9.1-np19py34_0   --> 0.9.0-np19py34_0  
cffi:         1.1.2-py34_0       --> 1.1.0-py34_0      
matplotlib:   1.4.3-np19py34_3   --> 1.4.3-np19py34_2  
mistune:      0.6-py34_0         --> 0.5.1-py34_1      
openpyxl:     2.0.2-py34_0       --> 1.8.5-py34_0      
pillow:       2.9.0-py34_0       --> 2.8.2-py34_0      
pip:          7.1.0-py34_0       --> 7.0.3-py34_0      
py:           1.4.30-py34_0      --> 1.4.27-py34_0     
pytest:       2.7.2-py34_0       --> 2.7.1-py34_0      
setuptools:   18.0.1-py34_0      --> 17.1.1-py34_0     
sqlalchemy:   1.0.6-py34_0       --> 1.0.5-py34_0      

Proceed ([y]/n)? y

Whereas if I do conda update --all, it gives me:

The following packages will be downloaded:

package	build
pillow-2.9.0	py34_0 482 KB
py-1.4.30	py34_0 127 KB
cffi-1.1.2	py34_0 169 KB
pytest-2.7.2	py34_0 189 KB
setuptools-18.0.1	py34_0 345 KB
sqlalchemy-1.0.6	py34_0 1.4 MB
pip-7.1.0	py34_0 1.4 MB
bokeh-0.9.1	np19py34_0 15.1 MB

------------------------------------------------------------
                                       Total:        19.2 MB

The following packages will be UPDATED:

bokeh:      0.9.0-np19py34_0 --> 0.9.1-np19py34_0
cffi:       1.1.0-py34_0     --> 1.1.2-py34_0    
pillow:     2.8.2-py34_0     --> 2.9.0-py34_0    
pip:        7.0.3-py34_0     --> 7.1.0-py34_0    
py:         1.4.27-py34_0    --> 1.4.30-py34_0   
pytest:     2.7.1-py34_0     --> 2.7.2-py34_0    
setuptools: 17.1.1-py34_0    --> 18.0.1-py34_0   
sqlalchemy: 1.0.5-py34_0     --> 1.0.6-py34_0

It seems conda update anaconda and conda install --all are trying to upgrade and downgrade the exact opposite set of packages. What is the reason for this behaviour? If I want to keep all packages "consistent" at the highest possible level, which of the commands is preferable? Its a strange behavious since I'd have expected both commands to be able to do this.

Thanks.

[ilanschnell]

To keep everything consistent, use "conda update anaconda", as Anaconda is the consistent set of packages for each Anaconda release.

[sensharma]

Thank you. For my understanding, how is conda update --all different? And why does it seem to be in conflict with conda update anaconda? I would like to know if this is a temporary hitch or if there is some fundamental feature of the two commands. Thanks again.
N.B.: In the past, conda update anaconda has failed to work a few times even when there have been consistent upgrades to install.

[ijstokes]

I may be wrong about the exact details, but the basic idea is that conda update anaconda will make your current environment match the exact set of packages plus versions of the latest Anaconda release. The latest Anaconda release may have been 5 months ago, and some of the package versions in the latest Anaconda are not the latest version that is available for the package itself. The nice thing about conda update anaconda is that all the packages will work together (modulo any missed dependency tests, which we do our best to avoid for prior to release), whereas conda update --all will try to update every package you currently have in your environment in such a way that all dependencies are satisfied. Because of the way conda works, this will normally be an impossible task if your environment has more than a few dozen packages in it. Plus even if it is possible the dependency solver may take ~forever to complete.

[asmeurer]

Because of the way conda works, this will normally be an impossible task if your environment has more than a few dozen packages in it. Plus even if it is possible the dependency solver may take ~forever to complete.

This is generally not true any more.

[ijstokes]

@asmeurer I'm assuming that your comment is mostly targeted towards the "~forever" comment, but if in fact you mean that by some magic conda update --all has found a way to overcome the common "Error: Unsatisfiable package specification" message (I can think of a few ways you may have achieved that), then I'd love to know.

[asmeurer]

I mean both the forever part and the impossible part. Things generally work now.

[asmeurer]

A few changes have been made in the past few months:

• conda update --all completely ignores the anaconda metapackage. Since anaconda almost always prevents updates, it makes little sense to include it. As noted, conda update anaconda and conda update --all are mutually exclusive ways of keeping things up-to-date.
• conda update --all no longer requires that each package that is installed be updated only. That is, it can downgrade a package, if it leads to the overall satisfiability of packages. Since conda already tries to make everything as up-to-date as possible, this results in far fewer unsatisfiability issues, which were usually caused because one package's dependency specifications required another to be downgraded.
• The unsatisfiable package specification hint generation is now very fast, even for a large number of packages. It no longer hangs or bails with no hint.
• The algorithm can still be a little slow. Not many changes have been made in the past six months there (one memory leak in Python 2 has been fixed recently). It should generally finish, though, unless you run out of memory, or are too impatient. For conda update --all, it shouldn't take more than a minute at the absolute worst on a reasonably fast machine.
• A few instances of over specified dependencies in the Anaconda metadata have been fixed.

[sensharma]

Guys, thanks a lot for your responses. Sorry I had disappeared for a bit. I have been taking the "conda update --all" approach for now. So far, so good. Doesn't take unbearably long and so far haven't faced any consistency issues.
Should I close this post? Once closed, if there are problems in future, is there a way to re-open the thread so it helps with the context? Thanks again.

[sensharma]

Sorry - another question - does "conda update --all" and "conda update anaconda" update all the pip installed packages as well (they do show up when I do a "conda list")?

[drandreaskrueger]

Hi!
I have found this issue googling because I was walso wondering about all those to-be-downgraded packages when doing

conda update conda
conda update anaconda

I had been excessively using pip in between (e.g. whenever I was not able to find a package on anaconda, or sometimes when a package readme explicitly told me so). And I guess that had led to all those newer packages (that anaconda now wants to downgrade) - right?

Can I for the future go for this strategy:

pip install whatever
conda update conda
conda update --all

and if I really run into problems, then do

conda update conda
conda update anaconda

to downgrade everything to a consistent set?

...

Perhaps ... is there a need / should I consider to have 2 parallel installations of anaconda, one with only conda update anaconda and never using pip, and the other one with conda update --all and in there I am allowed to be using pip ? But I guess long term, I will then probably only use that 2nd branch ...

TL;DR: I need to better understand the relation of conda and pip, I guess. How compatible are they?

Thanks!

[drandreaskrueger]

Addendum: I have just tried to additionally upgrade via pip after

conda update conda
conda update --all

via

pip freeze > requirements.txt
pip install -r requirements.txt --upgrade

which however only results in a long list of

Requirement already up-to-date: ...

so I guess - upgrading via pip is like conda update --all ?

[endolith]

So what's the recommended way to keep things up to date?

There is now an Anaconda Navigator that has all the packages listed under Environments. I can select "Upgradable" and I can upgrade them all from there, Is that the same as update -all? Will it break things?

[whimian]

I thinks using conda update anaconda may be the better way to keep everything working well.

[KrOstir]

Does conda update --all include also conda update conda? In other words, is conda update --all everything an average user should care about?

[nateGeorge]

Are conda update conda and conda update anaconda the same thing?

[kalefranz]

conda update conda updates conda. The package manager.

conda update anaconda updates Anaconda, the single metapackage with explicitly pinned versions of several hundred individual packages.
https://docs.continuum.io/anaconda/pkg-docs

[Inferno-P]

What does 'superseded by a higher channel' mean ? I am doing conda update anaconda.

[codeling]

conda update anaconda updates Anaconda, the single metapackage with explicitly pinned versions of several hundred individual packages.

This information (and the information I found everywhere else, e.g. here: https://stackoverflow.com/questions/45197777/how-do-i-update-anaconda) seems to be at least misleading; I read it that conda update anaconda should update to the latest "stable release" of Anaconda. This is however not at all what happens.

conda update anaconda does NOT try to install the latest "stable release", at least in none of the many times I have tried it. For example, with a just now newly installed version of 2019.07, conda update anaconda proposes to update several packages, and would "downgrade" anaconda (2019.07-py37_0 --> custom-py37_1). custom-py37_1 does not really strike me as an official "stable release" tag, right?

Curiously enough, even when specifying specific version to conda install (e.g. conda install anaconda=2019.07), strange things happen - when I do that on my freshly installed 2019.07, it proposes to update conda to 4.7.12. And when I do that, and then run conda update anaconda, I get the "dreaded" message about an "inconsistent environment" (for packages anaconda and numba).

So, after reading quite a bit about the "correct" procedure of how to update anaconda (conda update conda followed by conda update anaconda, or conda update --all, or conda install anaconda, ...); and playing around a lot with combinations of those commands; my current conclusion still is that the safest (and probably fastest) way to get a working, up-to-date Anaconda environment is to completely uninstall Anaconda, and then reinstalling everything...

[spencermathews]

I agree @codeling , this has been one of the biggest headaches caused by conda in my extensive time using it. I try to be gentle with my base environment, but even so I've had to resort to all manner of contortions to fix broken environments and get conda to update the way I want. Reconciling the actual behavior with the scattered and often contradictory docs is quite frustrating.

There have been times I've had to resort to completely reinstalling Anaconda. However, lately I've had success with less drastic solutions, including:

1. Explicitly install the custom Anaconda version. This can resolve consistency issues, then you can conda update --all to your hearts content:

    conda install anaconda=custom
   

2. Uninstall anaconda metapackage, then reinstall it. All of anaconda's dependencies might be remove in the process, but reinstalling will add them back. This might be useful if you just want to return to a consistent state:

    conda remove anaconda
    conda install anaconda[=version]
   

As of now, I can confirm your observation that (with conda version 4.7.12) starting in an env with anconda=2019.07 that conda update anaconda does want to update to custom anaconda. I think the catch is that conda update anaconda does not, as if often advertised, update to the latest anaconda release along with its version constrained dependencies. Instead, as is mentioned in ContinuumIO/anaconda-issues#9328, it will update all of anaconda's dependencies to their latest versions.

So forget what you've been told. The advice in Updating from older versions is misleading. Particularly, and I quote:

• FALSE: conda update anaconda grabs the latest release of the Anaconda metapackage.
• TRUE: conda update --all updates all packages in the current environment to the latest version.
• TRUE: conda update --all may not be able to make everything the latest versions because you may have conflicting constraints in your environment.
• FALSE: If you have the Anaconda metapackage version 2019.03, conda update --all should not update any dependencies that are in that metapackage because that metapackage constrains the solution.
• NONSENSE: With Anaconda 2019.07’s newer Anaconda metapackage, conda update --all should update you to that version, along with all of that version’s pins.

So in summary, there currently seems little use for conda update anaconda. Instead use conda update --all to keep packages up to date, and use conda install anaconda=<version> to explicitly update to a new anaconda release.

As a side note, the notion that the custom anaconda package is lower in version ordering than any actual release is bizarre. This may be true on initial install, but does not seem to hold for updates.

[msarahan]

Thank you for pointing these out. We'll try to both improve the docs and make the behavior more defined and predictable.

As a side note, the notion that the custom anaconda package is lower in version ordering than any actual release is bizarre. This may be true on initial install, but does not seem to hold for updates.

It is counterintuitive, but I don't think bizarre is the right word. The ideal behavior is a way to keep to the "stable" track, and if the anaconda "custom" version were highest, then there would have to be some kind of hack to exclude the "custom" package if staying on the stable track. An unintuitive consequence of this is that any package you want to update later may conflict with the stable release (for example, packages that have very tight pinning such as boost or hdf5), which would prevent you from adding on packages that share dependencies with the metapackage. The locked-down metapackage really is primarily only good for use by itself, and because of its size and specificity, it very quickly grows the need to become flexible ("custom") in real use with packages outside of its set.

Also, bear in mind that although it is absolutely true that the "custom" version is lower than any number version (see https://github.com/conda/conda/blob/master/conda/models/version.py#L47-L155 for how this works), it is also true that a conda update anaconda operation may end up changing from a numbered version to the custom version. Conda will do so in order to maximize the version of other things - the anaconda metapackage does not exist in isolation. This leads to people being confused about a "downgrade message" (ContinuumIO/anaconda-issues#11243), when what is really happening is just a relaxation of constraints to allow upgrades.

So, regarding your statements:

FALSE: conda update anaconda grabs the latest release of the Anaconda metapackage.

This is a bug. We need to fix it. This should be the stable track route.

FALSE: If you have the Anaconda metapackage version 2019.03, conda update --all should not update any dependencies that are in that metapackage because that metapackage constrains the solution.

This is a docs problem. conda update --all will unpin everything and make everything as new as it can.

NONSENSE: With Anaconda 2019.07’s newer Anaconda metapackage, conda update --all should update you to that version, along with all of that version’s pins.

Same as last one, docs need fix. conda update --all will almost certainly make the metapackage go to the custom version in order to update other specs. It is conceivable that it would pick something like 2019.07, but only if 2019.07 coincided with optimal versions of everything else.

CC @rrigdon @csoja

[github-actions]

Hi there, thank you for your contribution to Conda!

This issue has been automatically locked since it has not had recent activity after it was closed.

Please open a new issue if needed.