Support verifying downloads using PGP signatures and the signers Key ID

[postmodern]

It is problematic to keep updating MD5 / SHA1 checksums when a new package is available. Instead of constantly updating the md5/sha1 declarations, why not support verifying the download against a PGP ascii-armored signature (.asc). This could be done by specifying the path to the .asc file, or suffixing the download URL with .asc. The signers Key ID would also have to be provided in order for homebrew to download the signers key.

[jacknagel]

It is problematic to keep updating MD5 / SHA1 checksums when a new package is available.

Not really: the download URL has to be updated anyway! And adding a field for the signature means that would have to be updated as well, so there isn't any maintenance gain here.

But an even bigger reason is that checksums (well, message digests in our case) and cryptographic signatures are solving different problems.

We use checksums to ensure download integrity, and we absolutely need them as a mechanism to detect broken downloads. But they do not to ensure authenticity: by using Homebrew, you are placing your implicit trust in the maintainers and the idea that we are not evil and will not maliciously replace URLs and checksums.

Digital signatures are about message authentication. It involves your explicit trust in the signer of the message, i.e. your trust in the key that generated the signature. The careful user is free to check any downloaded tarballs against a signature if the developer provides one.

[jacknagel]

It is problematic to keep updating MD5 / SHA1 checksums when a new package is available.

Not really: the download URL has to be updated anyway! And adding a field for the signature means that would have to be updated as well, so there isn't any maintenance gain here.

But an even bigger reason is that checksums (well, message digests in our case) and cryptographic signatures are solving different problems.

We use checksums to ensure download integrity, and we absolutely need them as a mechanism to detect broken downloads. But they do not to ensure authenticity: by using Homebrew, you are placing your implicit trust in the maintainers and the idea that we are not evil and will not maliciously replace URLs and checksums.

Digital signatures are about message authentication. It involves your explicit trust in the signer of the message, i.e. your trust in the key that generated the signature. The careful user is free to check any downloaded tarballs against a signature if the developer provides one.

[jacknagel]

(The takeaway is that while I'm not opposed to the idea of adding support for signature verification, it will definitely not replace checksums.)

[jacknagel]

(The takeaway is that while I'm not opposed to the idea of adding support for signature verification, it will definitely not replace checksums.)

[postmodern]

I should rephrase what I said. Updating the URL and the md5/sha1 is troublesome for me. :) As part of my release process my Makefile automatically generates PGP signatures, which I later upload along with the .tar.gz. Having to run md5sum is yet another step for me.

[postmodern]

I should rephrase what I said. Updating the URL and the md5/sha1 is troublesome for me. :) As part of my release process my Makefile automatically generates PGP signatures, which I later upload along with the .tar.gz. Having to run md5sum is yet another step for me.

[jacknagel]

Sorry, but any other type of download verification would have to be an in-addition-to and opt-in type thing; i.e. the package would need a checksum in addition to the signature. Homebrew uses the checksums for cache invalidation, and we don't want to add any base dependencies to Homebrew other than Xcode/CLT.

[jacknagel]

Sorry, but any other type of download verification would have to be an in-addition-to and opt-in type thing; i.e. the package would need a checksum in addition to the signature. Homebrew uses the checksums for cache invalidation, and we don't want to add any base dependencies to Homebrew other than Xcode/CLT.

[adamv]

I don't want to add a PGP dependency for Homebrew.

[adamv]

I don't want to add a PGP dependency for Homebrew.

[MikeMcQuaid]

Seems the consensus is to close this. I agree with what @jacknagel and @adamv have said. Sorry!

[MikeMcQuaid]

Seems the consensus is to close this. I agree with what @jacknagel and @adamv have said. Sorry!

[andschwa]

@jacknagel @adamv @MikeMcQuaid

Before I go work on it, would you guys be opposed to development of automatic (but optional, i.e. if included by the formula, supplied by the developer, and with gpg installed by the user, not intended to replace checksums) GPG signature verification? It would display an error message (think along the lines of caveats) if the signature failed, so the curious (and prepared) user could investigate the discrepancy.

[andschwa]

@jacknagel @adamv @MikeMcQuaid

Before I go work on it, would you guys be opposed to development of automatic (but optional, i.e. if included by the formula, supplied by the developer, and with gpg installed by the user, not intended to replace checksums) GPG signature verification? It would display an error message (think along the lines of caveats) if the signature failed, so the curious (and prepared) user could investigate the discrepancy.

[MikeMcQuaid]

@andschwa Unfortunately those requirements contradict; we're not going to include it in the formula if it requires gpg required by the user. I'd suggest the best thing to do would be to write an external command like brew gpg-install which wrapped the current installation process and had metadata stored elsewhere mapping gpg files to formula.

[MikeMcQuaid]

@andschwa Unfortunately those requirements contradict; we're not going to include it in the formula if it requires gpg required by the user. I'd suggest the best thing to do would be to write an external command like brew gpg-install which wrapped the current installation process and had metadata stored elsewhere mapping gpg files to formula.

[andschwa]

I’m suggesting that after the hash check, if a GPG key is supplied in a formula, home brew check if gpg is installed, if so, verify the key, if gpg is not installed (or a gpg key was never supplied to begin with), home brew will continue on its merry way. That’s not a requirement of the user, it’s an option for them to have it installed, and therefore does not become a dependency. However, I would not be opposed to gpg-install, but brew would still need to handle formulae that supply GPG files (this may be trivial), but having externally mapped metadata would be a PIA that’s not necessary.

On Oct 24, 2013, at 1:23 AM, Mike McQuaid notifications@github.com wrote:

@andschwa Unfortunately those requirements contradict; we're not going to include it in the formula if it requires gpg required by the user. I'd suggest the best thing to do would be to write an external command like brew gpg-install which wrapped the current installation process and had metadata stored elsewhere mapping gpg files to formula.

—
Reply to this email directly or view it on GitHub.

[andschwa]

I’m suggesting that after the hash check, if a GPG key is supplied in a formula, home brew check if gpg is installed, if so, verify the key, if gpg is not installed (or a gpg key was never supplied to begin with), home brew will continue on its merry way. That’s not a requirement of the user, it’s an option for them to have it installed, and therefore does not become a dependency. However, I would not be opposed to gpg-install, but brew would still need to handle formulae that supply GPG files (this may be trivial), but having externally mapped metadata would be a PIA that’s not necessary.

On Oct 24, 2013, at 1:23 AM, Mike McQuaid notifications@github.com wrote:

@andschwa Unfortunately those requirements contradict; we're not going to include it in the formula if it requires gpg required by the user. I'd suggest the best thing to do would be to write an external command like brew gpg-install which wrapped the current installation process and had metadata stored elsewhere mapping gpg files to formula.

—
Reply to this email directly or view it on GitHub.

[MikeMcQuaid]

We're not going to add GPG keys to formula, sorry.

[MikeMcQuaid]

We're not going to add GPG keys to formula, sorry.

[andschwa]

@MikeMcQuaid I'm not asking you to do so, I was offering to do it myself. I just looked at the formula code, and indeed supplying a key would be trivial, I would just add an optional parameter to the class, gpg_url (actually it could simply be a flag, as the url would just be either url+'.asc' or url+'.sig'). If that's supplied, and then if it can import a GPG module successfully (which would raise an exception if GPG isn't installed), it would do the verification, otherwise it would catch the exception and not bother about GPG.

I'm just checking that you guys would potentially accept a merge that enables this functionality, before I start working on it. Because honestly, as a user in this day and age, I would feel much better if signatures were also verified. They provide a different level of authenticity than hashes (they are capable of replacing hashes as they do provide the same integrity check, but it's already been made clear that hashes are not to be replaced in home brew), that we would very much like to have.

[andschwa]

@MikeMcQuaid I'm not asking you to do so, I was offering to do it myself. I just looked at the formula code, and indeed supplying a key would be trivial, I would just add an optional parameter to the class, gpg_url (actually it could simply be a flag, as the url would just be either url+'.asc' or url+'.sig'). If that's supplied, and then if it can import a GPG module successfully (which would raise an exception if GPG isn't installed), it would do the verification, otherwise it would catch the exception and not bother about GPG.

I'm just checking that you guys would potentially accept a merge that enables this functionality, before I start working on it. Because honestly, as a user in this day and age, I would feel much better if signatures were also verified. They provide a different level of authenticity than hashes (they are capable of replacing hashes as they do provide the same integrity check, but it's already been made clear that hashes are not to be replaced in home brew), that we would very much like to have.

[MikeMcQuaid]

@andschwa By we're not going to add them I mean no-one is going to add them to formula for optional functionality. The same discussion was had around descriptions with @telemachus‎ and descriptions and he built an external command and descriptions repository which seems to be working well. I suggest you take the same approach:
https://github.com/telemachus/homebrew-desc

If you build it as an external command and cover a significant number of formula (which I think you may have slightly underestimated how huge an undertaking that is) then we might consider adding it in future. We won't accept a merge currently that allows or requires changes to formula.rb or Formula files themselves, sorry.

[MikeMcQuaid]

@andschwa By we're not going to add them I mean no-one is going to add them to formula for optional functionality. The same discussion was had around descriptions with @telemachus‎ and descriptions and he built an external command and descriptions repository which seems to be working well. I suggest you take the same approach:
https://github.com/telemachus/homebrew-desc

If you build it as an external command and cover a significant number of formula (which I think you may have slightly underestimated how huge an undertaking that is) then we might consider adding it in future. We won't accept a merge currently that allows or requires changes to formula.rb or Formula files themselves, sorry.

[telemachus]

you may have slightly underestimated how huge an undertaking that is

For what it's worth, by my count, there are currently 2505 formulas in mxcl/master. (On the plus side homebrew-desc only has 501 formulas left to go before it has complete coverage.)

[telemachus]

you may have slightly underestimated how huge an undertaking that is

For what it's worth, by my count, there are currently 2505 formulas in mxcl/master. (On the plus side homebrew-desc only has 501 formulas left to go before it has complete coverage.)

[sethladd]

Is there a doc where we can learn about what the process is to verify that a formula "does the right thing" and "comes from the right place" ? For example, how to the homebrew maintainers verify that a formula is pulling bits from the right place? Would love to learn more. Thanks!

[sethladd]

Is there a doc where we can learn about what the process is to verify that a formula "does the right thing" and "comes from the right place" ? For example, how to the homebrew maintainers verify that a formula is pulling bits from the right place? Would love to learn more. Thanks!

[MikeMcQuaid]

@sethladd There isn't one I'm afraid. There's basically just a small number of trusted core maintainers who have each worked on the project for years who use their judgement and discretion.

[MikeMcQuaid]

@sethladd There isn't one I'm afraid. There's basically just a small number of trusted core maintainers who have each worked on the project for years who use their judgement and discretion.

[postmodern]

@MikeMcQuaid is there some sort of process that the maintainers follow to verify a PR for an updated formula is authentic, even if it's submitted by a random user?

[postmodern]

@MikeMcQuaid is there some sort of process that the maintainers follow to verify a PR for an updated formula is authentic, even if it's submitted by a random user?

[MikeMcQuaid]

@postmodern Common sense, basically. If it gets pointed to some random new URL then we won't merge it without doing our own research. Pretty much everything in Homebrew is submitted by random users.

[MikeMcQuaid]

@postmodern Common sense, basically. If it gets pointed to some random new URL then we won't merge it without doing our own research. Pretty much everything in Homebrew is submitted by random users.

[postmodern]

@MikeMcQuaid I think attackers are usually a little more subtle than using "totally random new URLs". They will likely typo-squat the URL, either in the domain or if it's github/bitbucket the user/repo name. Sounds like you need a process to follow, instead of just trusting maintainers not get fooled.

[postmodern]

@MikeMcQuaid I think attackers are usually a little more subtle than using "totally random new URLs". They will likely typo-squat the URL, either in the domain or if it's github/bitbucket the user/repo name. Sounds like you need a process to follow, instead of just trusting maintainers not get fooled.

[MikeMcQuaid]

@postmodern And the diff will show that the URL root has been changed and we won't merge it. We're running Homebrew fine, thanks.

[MikeMcQuaid]

@postmodern And the diff will show that the URL root has been changed and we won't merge it. We're running Homebrew fine, thanks.