Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/build: publish GPG signatures of downloads for releases #14739

Closed
brianredbeard opened this issue Mar 10, 2016 · 26 comments

Comments

Projects
None yet
@brianredbeard
Copy link

commented Mar 10, 2016

While providing SHA256 sums is helpful, it would be preferable to also validate that binary blobs are created by a trusted entity. Utilizing a GPG signing key and providing detached signatures (in .asc or .sig format) would allow users to automatically validate the heritage of a Golang tarball/MSI.

@bradfitz

This comment has been minimized.

Copy link
Member

commented Mar 10, 2016

We do sign our *.msi and *.pkg files, because those OSes basically require it, and Google has infrastructure for signing binary releases for those.

I don't think Google has an existing mechanism for doing GPG signing, but I'm likely wrong, and that's not really the point. We could make it work if required, but I don't think GPG signing adds much anyway. We already have an API now to let you fetch (over HTTPS, with Google certs) the SHA-256 of the downloads, and then you can verify your curl with sha256sum or whatever. But even that is a little redundant, since we offer the downloads over https too.

Put another way: what's the threat you're concerned about?

@dominikh

This comment has been minimized.

Copy link
Member

commented Mar 10, 2016

(Dup of #12749)

@bradfitz

This comment has been minimized.

Copy link
Member

commented Mar 10, 2016

would allow users to automatically validate the heritage of a Golang tarball/MSI.

If either

  1. the download came from https://storage.googleapis.com/golang/*
  2. the SHA-256 of any set of bytes downloaded from any mirror (are there any mirrors?) matches the SHA-256 published at https://golang.org/dl/,

then its heritage is the Go team.

You used the word "automatically".

For 1), use curl without --insecure and curl will automatically validate Google's cert.

For 2), just append ".sha256" to URLs. Here is how to verify a hypothetical HTTP download against the HTTPS-verified SHA-256:

$ curl --silent -O http://storage.googleapis.com/golang/go1.6.src.tar.gz
$ (curl --silent https://storage.googleapis.com/golang/go1.6.src.tar.gz.sha256; echo "  go1.6.src.tar.gz") > sums
$ sha256sum --check sums
go1.6.src.tar.gz: OK

It's hypothetical because you can also just download it over https.

@brianredbeard

This comment has been minimized.

Copy link
Author

commented Mar 11, 2016

Sorry, my concern is that of a hostile network with a compromised CA certificate. Merely trusting the CA bundle alone has proved to be a questionable practice especially as the hashes are not signed either. Signing hashes would be sufficient, but at that point why not just sign the binaries as well.

State actors, Bluecoat, etc are all capable of targeting users.

SSL added and removed here

@bradfitz

This comment has been minimized.

Copy link
Member

commented Mar 11, 2016

The SSL is no longer added and removed there.

I'm going to close this. You might be legitimately concerned, but we're not going to change our process here at the current time. Even if we did sign our binaries with GPG, you'd then be concerned and filing more bugs asking whether our public key is really our public key and its heritage and how you know it's ours. I promise Google's network and CA infrastructure is leaps and bounds tighter than anybody on the Go team alone would be able to do. The Go team isn't going to manage our own keys. If we did, it might make you happy for a bit, but it wouldn't be more secure.

@bradfitz bradfitz closed this Mar 11, 2016

@brianredbeard

This comment has been minimized.

Copy link
Author

commented Mar 11, 2016

Seriously though, while I have a strong faith in the ability of the Google cloud team to properly secure, update, and maintain their SSL certificates, most users have a number of arbitrary certs in their bundle. While it's good that Chrome specifically handles updates to this bundle in a timely manner this is not the case of many installs of curl. As this is a compiler and not simply a binary which will be executed by users, it is ripe to be the target of an attack. Imagine the value of compromising the golang compiler for a user like @cloudflare, @coreos, BBC, or basically anyone on this list.

@csirac2

This comment has been minimized.

Copy link

commented Mar 11, 2016

Speaking of trusting arbitrary CAs, I've encountered an environment that added the in-house/private root CA to the CA bundles on their build boxes in order to deal with various self-hosted internal services running over TLS. Gaining control of the local CA certainly would've been a lot easier than compromising an actual CA.

GPG keys let build maintainers (rather than whatever is in the ca cert bundle this week) have a say in the provenance of their sources. Sure, you've got bigger problems than wonky build farms if someone is rummaging around on your network with a level of access that could exploit the above scenario, but the point is that CA infrastructure is a very different animal to specific, hand-curated GPG trust relationships. And, yes: builds break when signing keys change. For some, this is an important signal, and a feature.

Having helped administer an open source project that offered them, I can confirm that the number of people that access checksums (let alone signatures) is a tiny fraction of 1% of downloads. For many projects, it's perfectly justifiable not to offer GPG sigs. Just, let's not pretend that https is equivalent.

@minux

This comment has been minimized.

Copy link
Member

commented Mar 11, 2016

even if the binaries were signed, how could you be sure the
gpg key belongs to the Go team?

If Go team publishes the public key on a web page, then again
you have to trust the CA infrastructure to be certain the key is
genuine.

If you don't trust the current PKI infrastructure, then I don't think
you can trust gpg signed binary distributions either.

Or do you suggest another more trusted way to distribute the
public key?

If you don't trust the CA, then just mirror the Go repository in a
trusted machine and always build from source to make your
own binary distribution. It's that simple.

@csirac2

This comment has been minimized.

Copy link

commented Mar 11, 2016

I'm not arguing that the Go project should do gpg signed binaries. Not doing so is perfectly defensible. As you say, we can build ourselves.

I am saying that nominating a specific signing key once (every 12-24 months) to explicitly trust is completely different to trusting that hundreds of standard CAs plus arbitrary ones unique to each system and whatever intermidiate CAs in between, will enable TLS to really do its job on every single connection.

I only wish to counter the perception here that GPG signatures bring zero benefit. It may be negligible benefit (depending on where you sit), but it isn't zero.

@broady

This comment has been minimized.

Copy link
Member

commented Mar 11, 2016

Could you not do the same thing with the certificate used to secure golang.org?

The cert's current sha1 signature is 250667e0d6707003a665439694301899d213dd26. It will rotate once per year.

@csirac2

This comment has been minimized.

Copy link

commented Mar 11, 2016

Almost: curl --pinnedpubkey is apparently a thing now, but it's new enough that most distros aren't shipping it yet. curl --cacert and wget --ca-certificate aren't quite the same, but probably close enough in practice. We could also use openssl to extract and verify the signature shasum, but this isn't as clean because once again, it makes no guarantees for the next TLS connection over which we presumably fetch something.

Traditionally though, it's considered best practice to sign an artefact where possible, not just the connection it arrives on (hence why most software distribution channels continue to bother signing things rather than just trusting TLS).

Apologies for generating issue tracker noise, this is admittedly a very minor nit-pick, I do appreciate all the great work done here.

@bradfitz

This comment has been minimized.

Copy link
Member

commented Mar 11, 2016

@broady, yup. And here's a program to do that, with some curl-like flags:

https://gist.github.com/bradfitz/5c17a11e1f631e81073c

@adg

This comment has been minimized.

Copy link
Contributor

commented Mar 11, 2016

Once we provide an official apt repository (#10965), users on Debian/Ubuntu will have signed binaries they can verify, just like OS X and Windows.

@brianredbeard

This comment has been minimized.

Copy link
Author

commented May 26, 2016

For archive purposes, more evidence of threat from BlueCoat -

https://crt.sh/?id=19538258

@adg

This comment has been minimized.

Copy link
Contributor

commented May 26, 2016

@brianredbeard I'm not sure what to derive from that link. Can you provide some context?

@csirac2

This comment has been minimized.

Copy link

commented May 27, 2016

It seems people are worried about the implications of Blue Coat Systems - who (among other things) sell TLS MiTM proxy solutions - being made an official intermediate CA on the open internet by Symantec: https://news.ycombinator.com/item?id=11781915

@gtank

This comment has been minimized.

Copy link

commented May 27, 2016

I'll chime in to mention that this policy also makes it difficult to mirror or cache the Go binaries securely. With Linux distros I can serve an ISO image and detached signature anywhere. To offer the same security guarantees with Go I have to maintain a chain of custody all the way to a trusted server that uses TLS as well as Google does, or re-sign with my own (not widely trusted) GPG keys.

The Go team itself doesn't have to be involved with the signing to improve the situation. Just being able to verify that a particular artifact came from within Google without consulting Google servers each time would help. And Google does have at least one existing mechanism for doing GPG signing: https://www.google.com/linuxrepositories/

@thomsh

This comment has been minimized.

Copy link

commented Mar 1, 2017

No update on this thread sice May 2016, @brianredbeard raise a good point.

@bradfitz

This comment has been minimized.

Copy link
Member

commented Mar 1, 2017

@theznx, there are no updates on this thread because it's closed.

But I have aspirations of redoing a few pieces of our release tooling for Go 1.9 anyway, so I'll reopen this to track.

If Google's signing infrastructure supports GPG, we could published a detached signature as well. @broady, do you know? If so, it wouldn't add much pain to our release, since we already do all the binary signings in one step today.

@bradfitz bradfitz reopened this Mar 1, 2017

@bradfitz bradfitz changed the title Proposal: Sign Go downloads x/build: publish GPG signatures of downloads for releases Mar 1, 2017

@bradfitz bradfitz added this to the Go1.9Maybe milestone Mar 1, 2017

@gopherbot gopherbot added the Builders label Mar 21, 2017

@bradfitz

This comment has been minimized.

Copy link
Member

commented Jun 28, 2017

@broady, see internal docs on how to do this. I created shortlink http://go/gpg-sign.

If you have time to do this for Go 1.9, that'd be cool. Otherwise move this to Go 1.10.

@bradfitz bradfitz removed their assignment Jun 28, 2017

@bradfitz

This comment has been minimized.

Copy link
Member

commented Jul 20, 2017

@broady, do you want to do this for this release?

@broady

This comment has been minimized.

Copy link
Member

commented Aug 1, 2017

Sorry for the inaction. Yes, will look this week, hopefully get it done before 1.9 RC2.

@gopherbot

This comment has been minimized.

Copy link

commented Aug 23, 2017

Change https://golang.org/cl/58270 mentions this issue: release: upload asc files

@broady

This comment has been minimized.

Copy link
Member

commented Aug 24, 2017

I've signed go1.9rc2 for linux-amd64 only. I assume this is the artifact people need signed the most. Is that right?

wget https://storage.googleapis.com/golang/go1.9rc2.linux-amd64.tar.gz
wget https://storage.googleapis.com/golang/go1.9rc2.linux-amd64.tar.gz.asc
gpg --verify *.asc *.tar.gz

I'm sorry, I don't know much about practical use of GPG. Apparently the key that's used for signing is rotated yearly, and the latest is available here:
https://dl.google.com/dl/linux/linux_signing_key.pub

Are there any questions or concerns?

If there's demand, I'll work on signing every binary, not just linux-amd64.tar.gz

edit: ah, I think the subkey (1397 BC53 640D B551) will rotate out, but not 7721 F63B D38B 4796?

gopherbot pushed a commit to golang/build that referenced this issue Aug 24, 2017

release: upload asc files
Updates golang/go#14739.

Change-Id: Idf023b208b3a0605cb8841f572984f1b3053e2d8
Reviewed-on: https://go-review.googlesource.com/58270
Reviewed-by: Andrew Bonventre <andybons@golang.org>
@broady

This comment has been minimized.

Copy link
Member

commented Aug 24, 2017

go1.9rc2 now has asc files for all the tar.gz files.

releases will be signed in this manner going forward. (msi and pkg installers continue to be signed)

@broady broady closed this Aug 24, 2017

@yogo1212

This comment has been minimized.

Copy link

commented Feb 19, 2018

@broady thank you very much!
is this documented anywhere but here?
maybe it could go here here?

@golang golang locked and limited conversation to collaborators Feb 19, 2019

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
You can’t perform that action at this time.