Support pip 8's hash checking mode #303

Closed
aaugustin opened this Issue Jan 20, 2016 · 8 comments

Projects

None yet

7 participants

@aaugustin

See https://pip.pypa.io/en/stable/reference/pip_install/#hash-checking-mode

It would be nice if pip-tools generated the hashes automatically in its output.

That would increase security without adding any overhead for the developer.

@jezdez
Member
jezdez commented Jan 21, 2016

Yay! \o/

@nvie
Member
nvie commented Jan 23, 2016

This looks really interesting. Is there currently a way of generating those lines in some way? Or does one currently really have to copy the hash from the download URL and manually paste it into the requirements.txt?

@ubernostrum

I've been using hashin for semi-automated hash generation, but pip hash will at least get you the hash to stdout.

@jezdez
Member
jezdez commented Feb 28, 2016

@nvie The hash created by the hash checking mode is not identical to the MD5 hash generated and offered by PyPI download URLs. "Manually pasting" is basically the workflow that is expected for the hash checking feature, since it's basically part of the manual vetting process. You should use the hash subcommand as @ubernostrum indicated for that. I'm personally not convinced the hashin package is useful in that regard since it kind of defeats the purpose of the whole feature. Here's what I'd do right now (without pip-tools):

  • pip download <package-name>
  • vet the file that got downloaded that it's what you'd expect
  • pip hash <file that got downloaded>
  • copy the hash to requirements file

The features that would be useful of pip-tools to have is:

  • don't delete the --hash .. lines in the requirements
  • offer optional command line option e.g. --auto-hash to the pip-compile tool so it can be used like hashin
@edmorley
edmorley commented Mar 2, 2016

I'm personally not convinced the hashin package is useful in that regard since it kind of defeats the purpose of the whole feature.

I agree that skipping the 'vet the package' step does increase the risk, however I'm not convinced that it's by a huge amount.

For me, the major advantage of the new pip 8 hashing feature is that it protects against:

  • An active MITM attacker (they'd need to MITM the connection to PyPI from every one of: my local dev environment where the hashes were generated, Travis and the stage/prod deployment -- otherwise there would be a hash mismatch against a subset of them)
  • Accidental modification of an existing package (eg uploading a newer version over an older one, which actually happened in one of our dependencies last year, and thankfully was caught by peep)
  • Malicious modification of an existing package version (eg PyPI compromise, or package owner PyPI credential leak)

By not vetting a new package (or a new version release of an existing package), there is a risk that either:
a) the package author is trying to sneak malicious code into the package
b) someone other than the author created a fake new version release (either via PyPI compromise or package owner credential leak)

However for (a), I struggle to believe that most people would review the code thoroughly enough to spot intentionally obfuscated malicious code in a large package (even if diffing against a known good version, the changes could be hidden in a refactor; a backdoor can be implemented in very few lines).

And for (b), you'd have to manually update to the new version of the package prior to the package owner realising that someone had just created a new release on their behalf. (Side note: perhaps PyPI should email all the package owners every time a new release is created? Edit: Filed pypa/warehouse#997)

@aaugustin

I agree with @edmorley. That would be the most convenient way to take advantage of this feature.

It improves security in the same way HSTS does. If you're constantly victim of a MITM, you're still in trouble. If you're generally accessing the Internet without anything interfering, you'll notice if something does, whether voluntarily or not.

@pjenvey
Contributor
pjenvey commented Jan 18, 2017

This is now supported as of #383 so this can be closed, although it could also use some documentation

@davidovich
Contributor

Closing as it has been implemented.

@davidovich davidovich closed this Feb 19, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment