Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it possible to forward --allow-unsafe to pip-compile #157

Merged
merged 4 commits into from
Sep 12, 2019
Merged

Make it possible to forward --allow-unsafe to pip-compile #157

merged 4 commits into from
Sep 12, 2019

Conversation

bhearsum
Copy link
Contributor

@bhearsum bhearsum commented Sep 12, 2019

This seems to be a common issue for projects that use --generate-hashes and use packages that include setuptools in their requirements file (such as pytest and Markdown). Those in situation end up with unusable requirements files due to missing hashes on `setuptools. See jazzband/pip-tools#806 and jazzband/pip-tools#814 for more on this.

This PR simply permits --allow-unsafe to be forwarded to pip-compile to ensure it's included in the resulting requirement file(s).

I think this will fix #152, although it doesn't add the globbing support which that issue talks about (I'm not really sure I understand the purpose of that idea, to be honest).

@peterdemin
Copy link
Owner

Thanks, Ben, I appreciate your clear PR. Did you verify that the new feature works as expected?

@peterdemin
Copy link
Owner

On the second thought, since requirements file with hashes and without unsafe packages is uninstallable, doesn't it make sense to just fix the root cause in pip-tools?

@peterdemin
Copy link
Owner

(Don't worry about travis failure, it's a known issue fixed in another branch)

@codecov

This comment has been minimized.

@codecov

This comment has been minimized.

@peterdemin
Copy link
Owner

peterdemin commented Sep 12, 2019

I pushed updated dependencies to your branch. Note how new hash files contain commented unsafe packages:

+# WARNING: The following packages were not pinned, but pip requires them to be
+# pinned when the requirements file includes hashes. Consider using the --allow-unsafe flag.
+# pip==19.2.3               # via pipdeptree
+# setuptools==41.2.0        # via twine

But when I run

(tmp) pip-compile-multi 17:53:23$ python -V
Python 2.7.10
(tmp) pip-compile-multi 17:53:26$ pip install -r requirements/local27.hash

It succeeds having a note about setuptools:

Requirement already satisfied: setuptools>=0.7.0 in /Users/peter.demin/.virtualenvs/tmp/lib/python2.7/site-packages (from twine==1.14.0->-r requirements/local27.hash (line 137)) (41.2.0)

twine itself has hash:

37 twine==1.14.0 \
138     --hash=sha256:b2cec0dc1ac55bd74280d257f43763cf0cf928bdcd0de0fd70be70aa1195e3b0 \
139     --hash=sha256:e37d5a73d77b095b85314dde807bfb85b580b5b9d137f5b21332f4636990d97a

My question is, how to reproduce the issue with uninstallable hashed requirements file?

@peterdemin
Copy link
Owner

Ah, I see, when I downgraded setuptools to 0.6rc11, installation failed with:

Collecting setuptools>=0.7.0 (from twine==1.14.0->-r requirements/local27.hash (line 137))
ERROR: In --require-hashes mode, all requirements must have their versions pinned with ==. These do not:
    setuptools>=0.7.0 from https://files.pythonhosted.org/packages/b2/86/095d2f7829badc207c893dd4ac767e871f6cd547145df797ea26baea4e2e/setuptools-41.2.0-py2.py3-none-any.whl#sha256=4380abcf2a4ffd1a5ba22d687c6d690dce83b2b51c70e9c6d09f7e8c7e8040dc (from twine==1.14.0->-r requirements/local27.hash (line 137))

@peterdemin peterdemin merged commit fbb3f2e into peterdemin:master Sep 12, 2019
@peterdemin
Copy link
Owner

I confirmed that --allow-unsafe works as expected and actually started using it in tox.ini. I'll make a new release ASAP. Thanks again, Ben, great work!

@bhearsum
Copy link
Contributor Author

Sorry for my lack of response, I pushed this and ran away for the day :). It looks like everything is all clear now, but just to be 100% clear: the problem this is fixing only shows up with --generate-hashes, and is most easily reproducible with a fresh virtualenv (to make sure you've got a clean slate).

I don't think it can be fixed in pip-tools because they've made the explicit decision to not pin setuptools and a few other packages unless --allow-unsafe is specified. (I really don't understand why they do this when --generate-hashes is used, but that's a separate question).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

add --allow-unsafe per environment option
2 participants