Add option to uninstall packages not in requirements file

[wodow]

Currently a requirements file is used to specify what packages should be installed (and how, in some cases).

I believe it would be useful to have an additional option to uninstall all packages that do not appear in a supplied requirements file.

This would be useful for tightly controlling installing to a virtualenv or indeed any situation where you want to be absolutely confident that you have only the packages you desire.

Discussion on Stack Overflow: http://stackoverflow.com/questions/13176968/how-can-i-use-a-pip-requirements-file-to-uninstall-as-well-as-install-packages/

[fuhrysteve]

This should accomplish the functionality you're looking for in most cases (note that this doesn't seem to work with -e stuff like git/svn repositories and the like):

pip freeze | grep -v -f requirements.txt - | xargs pip uninstall -y

[keleshev]

I would love to see such option implemented. I even think that the main use-case of requirements file is to ensure that only the listed packages are installed, not more, not less.

[Lucretiel]

pip sync, perhaps?

[shuhaowu]

+1 for pip sync

[ThisGuyCodes]

This would be useful in continuous integration, currently many setups start with a totally clean environment, which works, but requires re-installing everything which takes time, this would make it easy to re-use the same virtual environment, only accounting for changes in stated requirements.

Also: @fuhrysteve your provided option pip freeze | grep -v -f requirements.txt - | xargs pip uninstall -y works, sorta. It doesn't account for dependencies of dependencies (ie: mongoengine depends on pymongo, but pymongo may not be in requirements.txt, thus pymongo and mongoengine will be uninstalled.)

Thus I think something implemented in pip itself would be more prudent. As an aside, I with Github had some sort of 'vote for issues' feature.

[shuhaowu]

I thought pip freeze frozed all of the packages? Including dependencies of dependencies?

[Lucretiel]

Pip freeze just dumps out a list of all your installed packages and their
exact versions. It's used so you can reproduce your environment later.
On Aug 25, 2014 11:32 AM, "Shuhao Wu" notifications@github.com wrote:

I thought pip freeze frozed all of the packages? Including dependencies of
dependencies?

—
Reply to this email directly or view it on GitHub
#716 (comment).

[shuhaowu]

Exactly. Doesn't that mean the scenario that @conslo described will never happen unless you manually edit your requirements.txt file?

[ThisGuyCodes]

@shuhaowu pip freeze lists all packages installed yes, including dependencies of dependencies. But dependencies of dependencies are NOT required to be stated in a requirements.txt file.

If I have mongengine stated as a requirement in my requirements.txt file, and I install with pip install -r requirements.txt, pymongo will also be installed (because mongoengine depends on it.) If I run @Lucretiel 's command pip freeze | grep -v -f requirements.txt - | xargs pip uninstall -y the pip freeze will list pymongoas well as mongoengine, but pymongo isn't in the requirements.txt file I'm filtering with, so it'll be passed to xargs pip uninstall -y, thus pymongo will be uninstalled, also causing mongoengine to be uninstalled.

I'm not sure what you mean by never happen unless you manually edit your requirements.txt file?, in what project are you coding that you don't control your own requirements?

[Lucretiel]

You also have to consider the reverse case- packages installed that aren't
necessarily dependencies. For instance, I typically install IPython and
Ipdb into my local environment. Even if I manually remove them from
requirements.txt, their dependencies may still remain.
On Aug 25, 2014 11:55 AM, "Travis Johnson" notifications@github.com wrote:

@shuhaowu https://github.com/shuhaowu pip freeze lists all packages
installed yes, including dependencies of dependencies. But dependencies of
dependencies are NOT required to be stated in a requirements.txt file.

If I have mongengine stated as a requirement in my requirements.txt file,
and I install with pip install -r requirements.txt, pymongo will also be
installed (because mongoengine depends on it.) If I run @Lucretiel
https://github.com/Lucretiel 's command pip freeze | grep -v -f
requirements.txt - | xargs pip uninstall -y the pip freeze will list
pymongoas well as mongoengine, but pymongo isn't in the requirements.txt
file I'm filtering with, so it'll be passed to xargs pip uninstall -y,
thus pymongo will be uninstalled, also causing mongoengine to be
uninstalled.

I'm not sure what you mean by never happen unless you manually edit your
requirements.txt file?, in what project are you coding that you don't
control your own requirements?

—
Reply to this email directly or view it on GitHub
#716 (comment).

[ThisGuyCodes]

@Lucretiel I'm not sure I understand. In that case wouldn't we want pip sync to remove them? If the job of pip sync is to remove packages not stated in requirements, then... packages not in requirements should be removed o.0

[Lucretiel]

Yes, that's what I'd like to see happen. I'm using Ipython as another
example for why the pipeline workaround falls short.
On Aug 25, 2014 12:04 PM, "Travis Johnson" notifications@github.com wrote:

@Lucretiel https://github.com/Lucretiel I'm not sure I understand. In
that case wouldn't we want pip sync to remove them? If the job of pip sync
is to remove packages not stated in requirements, then... packages not in
requirements should be removed o.0

—
Reply to this email directly or view it on GitHub
#716 (comment).

[arahayrabedian]

+1

my use case for this would be something like when you swap out a package for a fork of that package (e.g: PIL/pillow), when updating servers with pip install -r reqs.txt successively we can come across issues where both packages remain installed (and cause conflicts). Something like pip sync (or just an option/switch on pip install) makes perfect sense to uninstall the old (and whatever dependencies) and install the new. (someone please correct me if there's a way to do this as-is)

@Lucretiel @conslo I think the issue can be overcome by splitting of dev/prod requirements, pip sync (or whatever solution) would be a good step towards encouraging this.

[piotr-dobrogost]

There's pip-sync doing this, currently under development in the https://github.com/nvie/pip-tools/ project. See http://nvie.com/posts/better-package-management/

[pczerkas]

I think another good thing would be possibility to specify in requirements that specific package should be uninstalled. I came across situation, where having django-storages==1.1.8 in requirements I decided to upgrade to django-storages-redux==1.2.2. Simple editing of requirements file is not enough, because when installing requirements the second time, both packages have the same destination directory "storages", and pip freeze reports them both instaled.

[msabramo]

Cc @nvie who is working on pip-sync, part of pip-tools, as @piotr-dobrogost pointed out.

[pradyunsg]

Hey everyone!

I'll humbly request you to please refrain from posting a "+1" comment and use the new GitHub reactions feature on issue description to signify your support for getting this feature into pip.

Thanks a ton for understanding!

[pradyunsg]

@xavfernandez @pfmoore @dstufft Could one of you tag this as "enhancement proposal"?

[hellyworld]

Hello guys!

I have a work around!
If you have more packages installed in your venvs and the version you just received from git has less packages, you don't want to uninstall them manually, one by one!

Let's say that from git you have a requirements.txt with only 5 packages. On local venv you have 17 packages and some of them are not in the requirements.txt

1. Create a new file with your actual packages:
   pip freeze > uninstall_packages.txt
2. Uninstall all your packages from venv:
   pip uninstall -r uninstall_packages.txt -y
3. Install the packages from requirements.txt:
   pip install -r requirements.txt

Now, with three commands you have your packages, and only them!

Have fun with code!
Hellyworld

[DylanYoung]

Seems like you could just make the requirements optional for pip uninstall, no?

Then it's as simple as pip uninstall -y && pip install -r REQ_FILE

It does a bit of extra work... but hey, it's a computer ;)

[hellyworld]

@DylanYoung if I try your solution I have this error:
You must give at least one requirement to uninstall (see "pip help uninstall")

If you want a one line command, combining multiple command it can be as this:
pip freeze > uninstall_packages.txt && pip uninstall -r uninstall_packages.txt -y && pip install -r requirements.txt

This command works but I will not recommend it because it will be very easy to make some mistyping errors and you can't use the autocomplete tool to fast typing that uninstall_packages.txt doesn't exists.

P.S. I still have work to do on compacting and cleaning my code. I have only few months as a python developer and the road is long!

[hellyworld]

For those that are trying to find a copy/paste command solution I built and simplified this one:

pip freeze > uninstall_packages.txt && pip uninstall -r uninstall_packages.txt -y && pip install -r requirements.txt && rm uninstall_packages.txt

This command works if you have a requirements.txt that contain the correct packages and you must clean your environment! It is a command line combined from four commands as:

1. pip freeze > uninstall_packages.txt - will create an uninstall_packages.txt with the packages from the environment
2. pip uninstall -r uninstall_packages.txt -y - will uninstall the packages from the environment conform uninstall_packages.txt
3. pip install -r requirements.txt - will install the correct packages in the environment from requirements.txt
4. rm uninstall_packages.txt - will delete the previous created uninstall_packages.txt

Have fun with code!
Hellyworld

[DylanYoung]

Sorry for the confusion! It wasn't a workaround, but a suggested implementation for the devs. Here's a way where you can avoid the temp file if you like: `pip freeze | xargs pip uninstall -y && pip install -r requirements.txt` Happy Coding!

[mitchmieuxplacer]

There are cases when this feature would be very useful. Pandas installation takes about 15 minutes, so cleaning virtualenv before installing requirements file is very time consuming.

[chrahunt]

Given that pip will generate a wheel on first install and reuse that for subsequent installs, I think the "use a new virtualenv" argument is probably good enough for most cases (and what many projects do every day). If this isn't working and you don't know why please create a separate issue so we can help!

[pradyunsg]

I've labelled this issue as a "deferred till PR".

This label is essentially for indicating that further discussion related to this issue should be deferred until someone files a PR for it. This does not mean that the said PR would be accepted - it has not been determined whether this is a useful change to pip and that decision has been deferred until the PR is made.

[uranusjr]

For anyone interested, this feature could be generalised into “a switch to uninstall packages not depended by the requested packages.” The design would be something like pip install --unused-strategy=uninstall ...things_to_install. The switch would default to keep (the current behaviour).

[jannikmi]

This functionality is being provided by the command pip-sync of the pip-tools package: https://pypi.org/project/pip-tools/

[Regressor]

Recently I have a case: We have a large legacy project and replaced django-rest-auth/djangorestframework-jwt packets (and their dependency PyJWT-1.7.1) with dj-rest-auth/djangorestframework-simplejwt packets. Stage/prod docker images are built from scratch and there is no problem, but developers envs have old packets installed and pip install -r requirements.txt fails to finish because of PyJWT-1.7.1/2.1.0 conflict. To get it back they have to remove packages by hand or recreate env. And there is a pain switching branches.

Why there is no future to add some packages to requirement.txt as uninstall_if_exists ?

[pradyunsg]

Because this functionality exists in the pip-sync command, that’s developed externally to pip.

[uranusjr]

Do we still want to work on this if pip sync is available? See #10636