We really need to automate this and have an official workflow and policy for security releases.
Is the right thing to do here just to rewrite cURL's Perl script that converts the Mozilla text file into a .pem file? I took a look and it shouldn't be too awful to do.
the problem with mozilla's cert list is that it includes all of the blacklisted certs. We need to automate the removal of those.
The code at #1460 is interesting —it claims to be both a valid python file and a valid pem file at the same time.
Oh, I didn't spot he had a parser for Mozilla's certdata. That's interesting. I'll take a look into using that as a springboard. /me assigns to self.
This came up during the review/discussion of PEP 453 (the now accepted PEP to add an ensurepip module to bootstrap pip in Python 3.4). However, that acceptance is currently conditional on meeting an integration timeline that includes resolving this issue as part of the PEP's security considerations: http://www.python.org/dev/peps/pep-0453/#security-considerations
While ensurepip isn't affected by this (since it doesn't talk to the internet), pip is affected when installing from PyPI (including to update itself).
The target date in the PEP is currently December 29th (a bit over two months from now)
Will Pip be relying on Requests' CA Bundle (and to-be-introduced infrastructure)? I'm assuming so, since it needs to make calls out to the internet at large in addition to just PyPi.
What's the intended workflow for Pip? When a new cert gets blacklisted, will users need to run $ pip install pip --upgrade, or would an automatic CA Bundle update be preferred?
$ pip install pip --upgrade
Pip will be relying on the CA bundle shipped with requests at this point in time. I have a todo list item to see about the possibility of getting it to use system certificates (would that be something requests itself would be interested in?) but either way it'll need to use the bundled certs on AIX and HP-UX.
As of right now there is no plans for an automatic CA Bundle update, in order to get a new bundle they'll need to get a new pip.
@dstufft requests explicitly prefers not to rely on the system certs due to massive compatibility headaches the variance in distros often cause. I believe we have a utility function to enable this functionality built-in though. Will update shortly.
So, at this point in time, Pip is blocked only by our currently outdated CA Bundle, nothing else?
On the requests side that's all we require for ensurepip yes. The development branch of pip already is using requests with the older ca bundle.
@dstufft one thing to clarify here: historically, we only provide security releases for the latest release. We now have rigid API stability, so this shouldn't effect Pip at all. But, it is something to be aware of.
e.g. if/when Requests v3.0.0 comes around (not currently roadmapped), I will recommend Pip upgrade to it.
That sounds more or less reasonable as long as it doesn't remove anything we depend on :)
I'd rather not make this integration this rigid, but, if Pip wanted to provide an expected "api contract", it wouldn't be unwelcome.
The core of the SSL issue here is summed up in this conversation: http://twitter.theinfo.org/164062050971160576#id164067268924420098
I doubt you'll remove any of it, off the top of my head it's timeouts, TLS verification, http proxy support, CONNECT proxy support, and some system for allowing us to treat file:// urls similarly to http urls (the adaper thing works fine for this).
@dstufft are you aware of any sort of alert system for cert blacklisting? Is it possible to automate this?
I may make a simple service for this.
I'm not aware of anything.
FWIW, Christian Heimes mentioned to me that instead of blacklisting we want white listing based on the CKA_TRUST_* settings and enhanced key usage OIDs. Apparently there are certificates in the root store that are trusted, but not for HTTPS but only for code signing and such.
agl is a security person on the Chromium project, here's something they wrote to extract mozilla's certificates https://github.com/agl/extract-nss-root-certs/blob/master/convert_mozilla_certdata.go
It's written in go though.
Christian Heimes would be a good person to ping about this - he's been looking into a bunch of SSL issues for CPython, which is how the topic came up in relation to PEP 453 (see https://mail.python.org/pipermail/python-dev/2013-October/129755.html)
https://mail.python.org/pipermail/python-dev/2013-October/129738.html is the original post from Christian regarding cert management that prompted me to ask him to take a look at pip's SSL cert handling (which ended up resulting in the addition to the PEP 453 security considerations section already mentioned above)
http://www.egenix.com/company/news/eGenix-pyOpenSSL-Distribution-0.13.2.1.0.1.5.html has more details regarding what eGenix ended up doing for their cert bundles.
@dstufft interesting. What "transformations" need to be done against Mozilla's raw cert file?
I believe the workflow I used was taking the certs provided by curl's caextract and manually removing blacklisted certs.
I'm trying to grok this go code to see what agl is doing to extract a list of trusted certificates.
Go is perfect if it functions properly. @dstufft if you're up for it, could you vet it?
I'm reading it but I don't really know go, but to be honest agl is an actual expert in this area. I'd trust anything he had to say or wrote to do this way more then I'd trust myself.
You know who does know go, right? @lukasa that's who. ;)
@agl, any input you can provide would be much appreciated!
Going by the README, agl/extract-nss-root-certs appears to be perfect.
Yea, it appears to be exactly what requests wants here. Wrap it up with a little something that just polls Mozilla and compares the list existing list of trusted against the polled list of trusted and you'll end up with a pretty nice solution. If there's anything wrong with extract-nss-root-certs it'd probably be reasonable to file bugs.
Tentative new cert is available here:
+1 from @dstufft, @agl, or @tiran would be appreciated :)
Once I receive a +1, a new release will be cut.
So, the old CA Bundle is 211KB, and the new one is 1.4MB. This doesn't seem right :)
Oh, it looks like the go code didn't actually do anything.
Really? Seems like on my machine it removed some untrusted certs. I'll build a gist to show you what I mean.
The code doesn't create a file with PEM encoded certificates...
Apologies, because the files are massive, but they're here.
An example, the explicitly distrusted certificate "MD5 Collisions Forged Rogue CA 25c3" is not present in the second file (or doesn't appear to be).
@tiran Really? I don't know my PEM well enough, but it looks fine to me based on that Gist...
The file contains DER-encoded certificates in a octal notation. PEM certificates are base64 encoded DER blocks with limited line length and surrounded by
I think my gist isn't very clear: the first file is the original, the second (at the bottom) is the output. Direct link here. That looks like PEM to me. =)
How exactly did you create the file? On my machine the go script doesn't return a PEM encoded file.
$ curl https://hg.mozilla.org/mozilla-central/raw-file/tip/security/nss/lib/ckfw/builtins/certdata.txt -o certdata.txt
$ go run convert_mozilla_certdata.go
Oh, that's interesting. I used a different URL: https://mxr.mozilla.org/mozilla/source/security/nss/lib/ckfw/builtins/certdata.txt?raw=1. This is the URL given in the README for that project.
I suspect the program is tripping up on a difference between the two files, but not logging the error.
That URL points to an outdated bundle from 2012
Yeah, I just realised. =) Still, I think the diagnosis is solid. Just trying to work out what execution path we're taking through the Go code to make the mistake.
Oh, this is obvious. To get the whole license block, the code looks through for the line beginning CVS_ID. That line no longer exists in the new certificate file. I'll see if I can tweak it.
@tiran noticed the CVS_ID was missing, I guessed my way to https://gist.github.com/dstufft/7137007
@dstufft Exactly what I did.Lukasa/extract-nss-root-certs@cdead23
Heh, probably don't need all three solutions. =D
If three people arrive at the same conclusion, it must have a good chance of being right.. right?
Agreed. =) I think that @tiran's solution is the best for Pull-Request action, because it's a lot smarter than mine. =) Gotta fix up the slightly inconsistent indentation in it though (L162).
@kennethreitz If you're interested, I can try to wrap this code up into a nice binary that does all of the work itself: run one command and you're done. Sound good?
@lukasa i think this will work fine, but i won't stop you :)
My only concern now is being altered of new mistrusts.
Seems like the easiest way to do that would be to have an automated job that runs this process overnight and diffs against the current version in the repository. Any trust changes = time for new security release.
I'll setup an alert system that just checks for changes in mozilla's file
We can now update this at any time by running $ make certs
$ make certs