Add "upgrade" and "upgrade-all" commands

[vbabiy]

(Updated to reflect reality in April 2020)

This issue was initially scoped for changing the behavior of pip install --upgrade, by adding a new upgrade command with a different behavior when upgrading packages. The older recursive "eager" default caused grief for many (#304) However, after a lot of discussion on this, the approach taken was to add the --upgrade-strategy flag. The flag initially had a default of "eager" which was changed to a better default in #4500.

The tracking issue for "upgrade all packages" functionality is #4551.

[vbabiy]

"upgrade" is a trivial alias for "install --upgrade". Need to think a bit more
about "upgrade-all"; for one thing, I presume it would only upgrade packages
inside sys.prefix? (i.e. if you're inside a virtualenv, it wouldn't try to
upgrade global packages). That would be a reason to move
UninstallPathSet.can_uninstall() to a more generically-named function (or
method of InstallRequirement), so it provides generic "can I touch this?"
decisions.

---

Original Comment By: Carl Meyer

[vbabiy]

For the record, I think that seems like good idea, given the ability to
uninstall before upgrading. Although I'd prefer an --all option for
upgrade instead of an own upgrade-all command.

For the matter of can_uninstall(), I agree.. this is probably handy to have
globally anyway.

---

Original Comment By: Jannis Leidel

[vbabiy]

I'm not entirely unopposed to upgrade as an alias for install --upgrade. But
it seems a bit trivial.

upgrade-all requires you to figure out what is "upgradable". Probably one
prerequesite is that it lives in <sys.prefix>/lib/pythonX.Y/site-packages
(simply under sys.prefix isn't enough).

If we allow something like "zip import" (to bring a package from the parent
environment into a virtualenv environment) then probably packages in that
parent environment shouldn't be upgraded, but it's not 100% clear that is what
the user will expect.

I tried uninstalling an editable package with "pip uninstall" and it quite
reasonably offered to remove the .egg-link and update easy-install.pth. But it
couldn't have reasonably upgraded the package, so can_uninstall is somewhat
different from can_upgrade.

---

Original Comment By: Ian Bicking

[vbabiy]

Yeah, you're right that can_uninstall and can_upgrade are different.

I would think if we had "pip import" we still wouldn't want to upgrade
imported global packages; but (along with editables) it might be worth a "not
upgrading this" warning to the console.

---

Original Comment By: Carl Meyer

[vbabiy]

+1 for this bug

---

Original Comment By: smyrman

[vbabiy]

Issue #167 was marked as a duplicate of this issue.

---

Original Comment By: Carl Meyer

[vbabiy]

1

(echo pip; pip freeze | awk 'BEGIN{FS="=="}{print $1}') | xargs sudo pip

install -U

This should upgrade upgrade all installed packages (including pip itself). If
you run it in virtualenv you probably don't need to use sudo.

Of course it has high risk of failure -- if upgrading one of the packages
fails the whole process will fail (it's similar to port upgrade outdated in
MacPorts).

---

Original Comment By: Tomasz Elendt

[vbabiy]

+1 for upgrade --all

Why at the moment all Python module management facilities have to suck? Why no
one provides simple upgrade + upgrade --all command?

---

Original Comment By: Anonymous

[vbabiy]

I wouldn't mind taking a shot at an implementation, but first a few questions.

Is the general consensus that a new "upgrade" command that supports a '--all'
option be added to pip?

Running pip upgrade should only affect the environment it is running in. If
run from a virtualenv then only packages local to that env will be upgraded;
same for non-virtualenv's

---

Original Comment By: Kelsey Hightower

[vbabiy]

Kelsey: from my reading of the above discussion, I don't see any real
opposition to it. I think it's a fine addition to make. The main edge case is
editable VCS installs - as Ian suggested, I think an "upgrade" command
shouldn't touch those. Defining what "upgrade" means in the context of all the
editable possibilities (including local repos installed editable that have no
origin) would be next to impossible, I think, and even if some halfway-working
definition could be put together, it would only increase the maintenance
burden of the already-fragile VCS backends. But for non-editables -- go for
it!

---

Original Comment By: Carl Meyer

[vbabiy]

Carl: Cool, I will get started and update this ticket with the results.

---

Original Comment By: Kelsey Hightower

[vbabiy]

While working on the upgrade command the following questions came up:

• What methods should pip upgrade support to specify which packages to
  upgrade? Should we support a requirements file?
• How should pip upgrade handle packages that are not already installed?
  Should we install missing packages?

pip upgrade use cases and how to handle them:

# pip upgrade some_installed_package

Try and locate a package that satisfies the requirement. If the

requirement is not satisfied upgrade to the requested version. This includes
upgrading to an older version.

# pip upgrade --all

Locate all installed packages (non-editables) and update them to a new

version if available.

# pip upgrade some_other_package

Warning: some_other_package not installed, use pip install

some_other_package.

My goals are to keep the upgrade command really simple. You can upgrade
specific non-editable packages to a new or older version; or you can upgrade
all non-editable packages to a newer version.

Thoughts?

---

Original Comment By: Kelsey Hightower

[vbabiy]

I think "pip upgrade" should be an exact alias for "pip install --upgrade" as
it works now. That implies that yes, it installs requested packages if they
aren't installed, and yes, it accepts requirements files with -r. His should
be doable with almost no new code.

Upgrade --all will require some code for finding the list of currently
installed upgradable packages; then it should just pass that list to install
--upgrade, as above.

---

Original Comment By: Carl Meyer

[vbabiy]

Carl, thanks for the reply. I have pretty much taken the path you have
described. Later today I should be able to post some example runs.

---

Original Comment By: Kelsey Hightower

[vbabiy]

Got most of the code done, just a little polish left before I post the code
for review.

TODO:

• Tests
• Filter requirements file; add non-editables to list of packages to
  upgrade.

Running pip using the upgrade command

# pip upgrade --all

All packages up-to-date


# pip upgrade --all -v

Packages installed at latest version:

  pip: 0.8.2 (latest)

  distribute: 0.6.14 (latest)

  Python: 2.7.1 (latest)

  wsgiref: 0.1.2 (latest)

All packages up-to-date


# pip upgrade PyYAML

Package updates available:

  PyYAML: N/A (installed) 3.09 (latest)

Downloading/unpacking PyYAML

  Downloading PyYAML-3.09.tar.gz (238Kb): 238Kb downloaded

....

Successfully installed PyYAML

Cleaning up...


# pip upgrade --all -v

Packages installed at latest version:

  pip: 0.8.2 (latest)

  distribute: 0.6.14 (latest)

  PyYAML: 3.09 (latest)

  Python: 2.7.1 (latest)

  wsgiref: 0.1.2 (latest)

All packages up-to-date


# pip upgrade PyYAML==3.08

Downloading/unpacking PyYAML==3.08

....

Successfully installed PyYAML

Cleaning up...


# pip upgrade --all -v

Packages installed at latest version:

  pip: 0.8.2 (latest)

  distribute: 0.6.14 (latest)

  Python: 2.7.1 (latest)

  wsgiref: 0.1.2 (latest)

Package updates available:

  PyYAML: 3.08 (installed) 3.09 (latest)

Downloading/unpacking PyYAML

...

Successfully installed PyYAML

Cleaning up...

  Removing temporary dir /root/upgrade_env/build...

---

Original Comment By: Kelsey Hightower