Do not exclude legitimately installed packages from pip freeze #1610

Closed
piotr-dobrogost opened this Issue Mar 2, 2014 · 5 comments

Comments

Projects
None yet
7 participants

Silently excluding legitimately installed packages like setuptools, distribute and pip is not ok. For instance, as requirements file can contain options for pip, someone might want to ensure that the proper version of pip (able to understand these options) would be available. Also, @qwcode gave rationale for not excluding setuptools in issue #1606:

with pip 1.5 able to install from wheel without setuptools, I can see a real case where it's meaningful for setuptools to be listed, i.e. when one of the packages needs setuptools due to an import of pkg_resources, but the install is just using pip (and no setuptools) to install from wheel.

Reasons for having setuptools as a requirement were given in answers to my recent question Is there any sense to put setuptools as a requirement? on distutils-sig mailing list.

There should be an option so that users could decide themselves what packages if any they want to exclude. In any case there should be information about excluded packages presented in the output of every command affected.

By legitimately installed I mean all outside of stdlib as excluding stdlib packages with metadata is another subject – see issue #1570.

Owner

dstufft commented Mar 2, 2014

Putting pip inside of a requirements.txt file is almost never what you want. For instance, you stated:

For instance, as requirements file can contain options for pip, someone might want to ensure that the proper version of pip (able to understand these options) would be available.

However the problem is, if you have a requirements file like that, then an incompatible version of pip will bomb out trying to read that requirements.txt file before it even has a chance to upgrade pip. Even if everything goes without a hitch and you successfully upgrade/downgrade pip to a different version via the requirements.txt file it won't actually take any effect until the next invocation of pip (because the current version will have already been imported). If we want to handle the case where you modify the version of pip in a single invocation then we're going to need additional code to handle that.

In the past there was the same sort of problem with setuptools. You couldn't import pip without setuptools installed and you couldn't upgrade setuptools in the same process and have it take effect in the installation of the rest of the packages in that installation run. Now a days this is no longer the case, however I would still argue that it should be excluded by default because rarely does someone actually want to assert a particular version of setuptools (and in fact it can be harmful, especially on platforms like Heroku).

I do not believe we should be putting pip into the output of pip freeze by default, and I feel pretty strongly about that. setuptools/distribute being included by default I still don't think is generally applicable and can be harmful, but I could probably be convinced otherwise. One option is to include setuptools/distribute by default, but have it commented out so that it's obvious that it's there and what version was installed, but that it doesn't affect anything without end user work. Another option is to make the list of what to ignore configurable. If the user hasn't specified any ignores it uses our default list. If they configure any at all they are responsible for configuring them all.

Member

pfmoore commented Mar 2, 2014

I don't have a strong opinion about setuptools, but I agree with @dstufft that including pip is wrong. And I do not want to see distribute included by default.

People can manually edit requirements files however they like - as far as I can see this is purely about using freeze and not reviewing or amending its output in any way. I think I'd need to see a specific, clear use case where the current behaviour causes an error and where manual editing of the requirements file is not practical, before I'd think this was worth it.

Contributor

qwcode commented Mar 2, 2014

One option is to include setuptools/distribute by default,
but have it commented out so that it's obvious that it's there and what version was installed
Another option is to make the list of what to ignore configurable.
If the user hasn't specified any ignores it uses our default list.

I'd be ok with either or both of these.

Member

Ivoz commented Mar 19, 2014

I think part of the issue is that pip and setuptools are pretty much "build environment" requirements, not regular "install" requirements; but requirements.txt' use case was only ever consciously designed for the latter.

If you separate these two concepts, you come up with a modicum solution:

pip install -r build_requirements.txt
pip install -r requirements.txt

Where build_requirements.txt had something like:

setuptools==<version>
pip==<version>

taion commented Oct 8, 2015

As a reference point for how other package managers handle this, NPM explicitly splits out:

I think in general something like pip freeze that locks down the full set of dependencies and sub-dependencies is not the right way to handle development dependencies anyway.

@xavfernandez xavfernandez added a commit to xavfernandez/pip that referenced this issue Oct 10, 2015

@xavfernandez xavfernandez Do not skip anything
in case a skipped package is in the requirement file
Add an option to whitelist some packages - closes #1610
48c26e1

@xavfernandez xavfernandez added a commit to xavfernandez/pip that referenced this issue Oct 21, 2015

@xavfernandez xavfernandez Do not skip anything
in case a skipped package is in the requirement file
Add an option to whitelist some packages - closes #1610
cb87243

@xavfernandez xavfernandez added a commit to xavfernandez/pip that referenced this issue Oct 22, 2015

@xavfernandez xavfernandez Do not skip anything
in case a skipped package is in the requirement file
Add an option to whitelist some packages - closes #1610
4b0c00f

@xavfernandez xavfernandez added a commit to xavfernandez/pip that referenced this issue Nov 13, 2015

@xavfernandez xavfernandez Do not skip anything
in case a skipped package is in the requirement file
Add an option to whitelist some packages - closes #1610
e7526fd

@xavfernandez xavfernandez added a commit to xavfernandez/pip that referenced this issue Nov 13, 2015

@xavfernandez xavfernandez Do not skip anything
in case a skipped package is in the requirement file
Add an option to whitelist some packages - closes #1610
e5a9cc4

@xavfernandez xavfernandez added a commit to xavfernandez/pip that referenced this issue Nov 29, 2015

@xavfernandez xavfernandez Do not skip anything
in case a skipped package is in the requirement file
Add an option to whitelist some packages - closes #1610
8a3ef2a

@xavfernandez xavfernandez added a commit to xavfernandez/pip that referenced this issue Dec 3, 2015

@xavfernandez xavfernandez Do not skip anything
in case a skipped package is in the requirement file
Add an option to whitelist some packages - closes #1610
1cb937c

@xavfernandez xavfernandez added a commit to xavfernandez/pip that referenced this issue Jan 2, 2016

@xavfernandez xavfernandez Do not skip anything
in case a skipped package is in the requirement file
Add an option to whitelist some packages - closes #1610
684f570

@xavfernandez xavfernandez added a commit to xavfernandez/pip that referenced this issue Jan 4, 2016

@xavfernandez xavfernandez Do not skip anything
in case a skipped package is in the requirement file
Add an option to whitelist some packages - closes #1610
8a2b79a

@xavfernandez xavfernandez added a commit to xavfernandez/pip that referenced this issue Jan 13, 2016

@xavfernandez xavfernandez Do not skip anything
in case a skipped package is in the requirement file
Add an option to whitelist some packages - closes #1610
4e0e13b

@xavfernandez xavfernandez added a commit to xavfernandez/pip that referenced this issue Jan 15, 2016

@xavfernandez xavfernandez Do not skip anything
in case a skipped package is in the requirement file
Add an option to whitelist some packages - closes #1610
49d53ad

@xavfernandez xavfernandez added a commit to xavfernandez/pip that referenced this issue Jan 19, 2016

@xavfernandez xavfernandez Do not skip anything
in case a skipped package is in the requirement file
Add an option to whitelist some packages - closes #1610
8b3fade

@xavfernandez xavfernandez added a commit to xavfernandez/pip that referenced this issue Jan 26, 2016

@xavfernandez xavfernandez Do not skip anything
in case a skipped package is in the requirement file
Add an option to whitelist some packages - closes #1610
332170b

@xavfernandez xavfernandez added a commit to xavfernandez/pip that referenced this issue Jan 31, 2016

@xavfernandez xavfernandez Do not skip anything
in case a skipped package is in the requirement file
Add an option to whitelist some packages - closes #1610
8177c9f

@xavfernandez xavfernandez added a commit to xavfernandez/pip that referenced this issue Feb 4, 2016

@xavfernandez xavfernandez Do not skip anything
in case a skipped package is in the requirement file
Add an option to whitelist some packages - closes #1610
9b4c4d0

@xavfernandez xavfernandez added a commit to xavfernandez/pip that referenced this issue Feb 6, 2016

@xavfernandez xavfernandez Add --all option to pip freeze
In order to include pip/setuptools/wheel in the freeze output
Closes #1610
772fda0

@xavfernandez xavfernandez added a commit to xavfernandez/pip that referenced this issue Feb 17, 2016

@xavfernandez xavfernandez Add --all option to pip freeze
In order to include pip/setuptools/wheel in the freeze output
Closes #1610
2956a3e

dstufft closed this in #3458 Mar 3, 2016

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment