New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cryptographically strong code signing #2240

Open
ypid opened this Issue Aug 7, 2016 · 29 comments

Comments

Projects
None yet
8 participants
@ypid

ypid commented Aug 7, 2016

You are going to great extend to ensure source code authenticity already. 👍

However, it seems to me that the weakest link here is SHA1 used by git. Ref entry point: sign all git commits

As I am not sure when this problem will be fixed at it’s core (git), I would propose to include a cryptographically strong hash sum over the whole commit (commit, tree, and blobs it references and recursively over submodules) in git tags which are directly signed with GnuPG, also using a cryptographically strong hash sum.

I was quite happy to find git-evtag today which implements this. I have read the Python implementation and it looks good to me. But I am sure you guys can do a more careful review of what ever implementation you end up using 😉

The main advantage this will give us is that targeted attacks one someone doing a git clone and the adversary being able to perform a preimage attack on a SHA1 hashed file will be more difficult. But even with just SHA1, according to Mike Gerwitz, such an attack would be even harder when the target already has a authentic copy of a repo and an attack is performed on git pull.

@andrewdavidwong

This comment has been minimized.

Show comment
Hide comment
@andrewdavidwong

andrewdavidwong Aug 7, 2016

Member

@marmarek, what do you think?

Member

andrewdavidwong commented Aug 7, 2016

@marmarek, what do you think?

@ypid ypid referenced this issue in owncloud/core Aug 10, 2016

Closed

Verifiable releases based on strong cryptography #25710

0 of 3 tasks complete
@CodesInChaos

This comment has been minimized.

Show comment
Hide comment
@CodesInChaos

CodesInChaos Aug 11, 2016

The risk with SHA-1 is that its collision resistance is dangerously weak. Its second pre-image resistance is still very strong.

The risk with SHA-1 is that its collision resistance is dangerously weak. Its second pre-image resistance is still very strong.

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Aug 11, 2016

Member

Indeed git-evtag looks interesting. Will look at it more closely
later.
We use tags for all the source verification anyway (not only for release
tags, but also intermediate pushes), so it should be very easy to
integrate it into our workflow.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

Member

marmarek commented Aug 11, 2016

Indeed git-evtag looks interesting. Will look at it more closely
later.
We use tags for all the source verification anyway (not only for release
tags, but also intermediate pushes), so it should be very easy to
integrate it into our workflow.

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

@nbraud

This comment has been minimized.

Show comment
Hide comment
@nbraud

nbraud Aug 14, 2016

@ypid Note that OpenPGP signatures are also vulnerable to SHA1 second-preimage attacks (the adversary can forge keys with specific keyids at that point).
As @CodesInChaos pointed out, second preimages on SHA1 are not an immediate concern.

nbraud commented Aug 14, 2016

@ypid Note that OpenPGP signatures are also vulnerable to SHA1 second-preimage attacks (the adversary can forge keys with specific keyids at that point).
As @CodesInChaos pointed out, second preimages on SHA1 are not an immediate concern.

@ypid ypid referenced this issue in debops/ansible-core Oct 11, 2016

Open

Change keyserver to hkps://hkps.pool.sks-keyservers.net #21

0 of 2 tasks complete

@andrewdavidwong andrewdavidwong added this to the Far in the future milestone Dec 24, 2016

@ypid ypid referenced this issue in cr-marcstevens/sha1collisiondetection Feb 23, 2017

Closed

Workaround for git using SHA2-512 exists #3

@jpouellet

This comment has been minimized.

Show comment
Hide comment
@jpouellet

jpouellet Feb 23, 2017

Contributor

https://shattered.it/ !! We should really revisit this sooner than later.

Is git-evtag the best recommendation at this point?

If so, thoughts adding docs about it to: https://www.qubes-os.org/doc/code-signing/
And enforcing it in: https://github.com/marmarek/signature-checker ?

Contributor

jpouellet commented Feb 23, 2017

https://shattered.it/ !! We should really revisit this sooner than later.

Is git-evtag the best recommendation at this point?

If so, thoughts adding docs about it to: https://www.qubes-os.org/doc/code-signing/
And enforcing it in: https://github.com/marmarek/signature-checker ?

@ypid

This comment has been minimized.

Show comment
Hide comment
@ypid

ypid Feb 23, 2017

@jpouellet Agreed. But note that this is not a preimage attack. But I think another attack is now practically possible. Consider an adversary submits a PR including a file generated with a cryptanalytic SHA-1 collision attack. The commit together with the authentic file get reviewed and enter the repository. Then when building Qubes OS, GitHub returns the forged version of the file. The OpenPGP signature and git tree will still be valid and the file enters a Qubes OS build. Ref Security challenges for the Qubes build process which at least makes a targeted attack on this pretty difficult. Note that finding such a collision is one thing, making it actually plausible and compromise Qubes OS is a different story I would say.

At least for the presented attack, it is possible to check if a file is generated with a cryptanalytic SHA-1 collision attack using the released sha1dcsum tool. I just checked Qubes OS sources against this and did not find a single file like this (checkout and git history parts of git history (would need to unpack pack files first which I did not do, see comment below). find . -type f -print0 | xargs --null sha1dcsum | fgrep '*coll*' (run in a Disposable VM of course 😉 ).

Edit: Improved command used for checking.

ypid commented Feb 23, 2017

@jpouellet Agreed. But note that this is not a preimage attack. But I think another attack is now practically possible. Consider an adversary submits a PR including a file generated with a cryptanalytic SHA-1 collision attack. The commit together with the authentic file get reviewed and enter the repository. Then when building Qubes OS, GitHub returns the forged version of the file. The OpenPGP signature and git tree will still be valid and the file enters a Qubes OS build. Ref Security challenges for the Qubes build process which at least makes a targeted attack on this pretty difficult. Note that finding such a collision is one thing, making it actually plausible and compromise Qubes OS is a different story I would say.

At least for the presented attack, it is possible to check if a file is generated with a cryptanalytic SHA-1 collision attack using the released sha1dcsum tool. I just checked Qubes OS sources against this and did not find a single file like this (checkout and git history parts of git history (would need to unpack pack files first which I did not do, see comment below). find . -type f -print0 | xargs --null sha1dcsum | fgrep '*coll*' (run in a Disposable VM of course 😉 ).

Edit: Improved command used for checking.

@ypid

This comment has been minimized.

Show comment
Hide comment
@ypid

ypid Feb 23, 2017

After looking into it all day, it does not appear to impact git's security immediately, except for targeted attacks against specific projects by very wealthy attackers. But we're well past the time when it seemed ok that git uses SHA1. If this gets improved into a chosen-prefix collision attack, git will start to be rather insecure.

Ref: https://git-annex.branchable.com/devblog/day_449__SHA1_break_day/

ypid commented Feb 23, 2017

After looking into it all day, it does not appear to impact git's security immediately, except for targeted attacks against specific projects by very wealthy attackers. But we're well past the time when it seemed ok that git uses SHA1. If this gets improved into a chosen-prefix collision attack, git will start to be rather insecure.

Ref: https://git-annex.branchable.com/devblog/day_449__SHA1_break_day/

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Feb 23, 2017

Member

@ypid in case of git, you should not look at files but git objects (some are available directly in .git, but most of them are packed).

As for git-evtag, I've just given it quick look and besides crypto itself, git integration seems to be very incomplete. For example it support only creating/verifying tags from currently checked out sources, without local modifications. So no way to verify evtag before checkout (just after git fetch). Also many arguments of git tag are missing in git evtag - for example it's impossible to create tag non-interactively (-m for message).

Member

marmarek commented Feb 23, 2017

@ypid in case of git, you should not look at files but git objects (some are available directly in .git, but most of them are packed).

As for git-evtag, I've just given it quick look and besides crypto itself, git integration seems to be very incomplete. For example it support only creating/verifying tags from currently checked out sources, without local modifications. So no way to verify evtag before checkout (just after git fetch). Also many arguments of git tag are missing in git evtag - for example it's impossible to create tag non-interactively (-m for message).

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Feb 23, 2017

Member
$ git evtag verify evtag1
error: Target d2ef17888a18316baebf2c2b68f516c689bdadfb is not HEAD (267c20cf8fb60490dcd32a682c92ff7b514e4ca1); currently git-evtag can only tag or verify HEAD in a pristine checkout
Member

marmarek commented Feb 23, 2017

$ git evtag verify evtag1
error: Target d2ef17888a18316baebf2c2b68f516c689bdadfb is not HEAD (267c20cf8fb60490dcd32a682c92ff7b514e4ca1); currently git-evtag can only tag or verify HEAD in a pristine checkout
@rootkovska

This comment has been minimized.

Show comment
Hide comment
@rootkovska

rootkovska Feb 23, 2017

Member

As a temporary solution, perhaps we might consider integrating the "collision detector" [1], presented by the authors of the attack, into qubes-builder? It's only not clear to me if these tool can be easily used on actual hashes, rather on files which are to be hashed?

[1] https://github.com/cr-marcstevens/sha1collisiondetection

Member

rootkovska commented Feb 23, 2017

As a temporary solution, perhaps we might consider integrating the "collision detector" [1], presented by the authors of the attack, into qubes-builder? It's only not clear to me if these tool can be easily used on actual hashes, rather on files which are to be hashed?

[1] https://github.com/cr-marcstevens/sha1collisiondetection

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Feb 23, 2017

Member

@ypid Note that OpenPGP signatures are also vulnerable to SHA1 second-preimage attacks (the adversary can forge keys with specific keyids at that point).

I think it isn't a problem if you use actual key file (qubes-builder/qubes-developers-key.asc for example), not only key IDs. Also hash used in gpg signature is configurable - we use SHA2 already.

Member

marmarek commented Feb 23, 2017

@ypid Note that OpenPGP signatures are also vulnerable to SHA1 second-preimage attacks (the adversary can forge keys with specific keyids at that point).

I think it isn't a problem if you use actual key file (qubes-builder/qubes-developers-key.asc for example), not only key IDs. Also hash used in gpg signature is configurable - we use SHA2 already.

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Feb 23, 2017

Member

As a temporary solution, perhaps we might consider integrating the "collision detector" [1], presented by the authors of the attack, into qubes-builder? It's only not clear to me if these tool can be easily used on actual hashes, rather on files which are to be hashed?

Even if not, it shouldn't be hard to iterate over all git objects with git cat-file.

Member

marmarek commented Feb 23, 2017

As a temporary solution, perhaps we might consider integrating the "collision detector" [1], presented by the authors of the attack, into qubes-builder? It's only not clear to me if these tool can be easily used on actual hashes, rather on files which are to be hashed?

Even if not, it shouldn't be hard to iterate over all git objects with git cat-file.

@ypid

This comment has been minimized.

Show comment
Hide comment
@ypid

ypid Feb 23, 2017

I think it isn't a problem if you use actual key file (qubes-builder/qubes-developers-key.asc for example)

That should mitigate this.

Also hash used in gpg signature is configurable - we use SHA2 already.

Yes, for signing files, the hash is configurable. Not however for the OpenPGP key fingerprints as they use SHA1 as specified: RFC4880, 12.2. Key IDs and Fingerprints

ypid commented Feb 23, 2017

I think it isn't a problem if you use actual key file (qubes-builder/qubes-developers-key.asc for example)

That should mitigate this.

Also hash used in gpg signature is configurable - we use SHA2 already.

Yes, for signing files, the hash is configurable. Not however for the OpenPGP key fingerprints as they use SHA1 as specified: RFC4880, 12.2. Key IDs and Fingerprints

@andrewdavidwong

This comment has been minimized.

Show comment
Hide comment
@andrewdavidwong

andrewdavidwong Feb 24, 2017

Member

@ypid Note that OpenPGP signatures are also vulnerable to SHA1 second-preimage attacks (the adversary can forge keys with specific keyids at that point).

I think it isn't a problem if you use actual key file (qubes-builder/qubes-developers-key.asc for example)

That should mitigate this.

Also hash used in gpg signature is configurable - we use SHA2 already.

Yes, for signing files, the hash is configurable. Not however for the OpenPGP key fingerprints as they use SHA1 as specified: RFC4880, 12.2. Key IDs and Fingerprints

I don't see the mitigation in that scenario. As you point out, the real danger of a SHA-1 second-preimage attack is the ability to create a key with the same fingerprint (not just key ID) as an existing key. @marmarek's suggestion is to use the actual key file, e.g., qubes-developers-key.asc, but I don't see how that helps. Suppose I download that file. I now need to verify that the keys it contains are genuine. How can I do that? Presumably, by verifying that their fingerprints match the trusted fingerprints I've already obtained out-of-band. But this just reintroduces the original problem.

Member

andrewdavidwong commented Feb 24, 2017

@ypid Note that OpenPGP signatures are also vulnerable to SHA1 second-preimage attacks (the adversary can forge keys with specific keyids at that point).

I think it isn't a problem if you use actual key file (qubes-builder/qubes-developers-key.asc for example)

That should mitigate this.

Also hash used in gpg signature is configurable - we use SHA2 already.

Yes, for signing files, the hash is configurable. Not however for the OpenPGP key fingerprints as they use SHA1 as specified: RFC4880, 12.2. Key IDs and Fingerprints

I don't see the mitigation in that scenario. As you point out, the real danger of a SHA-1 second-preimage attack is the ability to create a key with the same fingerprint (not just key ID) as an existing key. @marmarek's suggestion is to use the actual key file, e.g., qubes-developers-key.asc, but I don't see how that helps. Suppose I download that file. I now need to verify that the keys it contains are genuine. How can I do that? Presumably, by verifying that their fingerprints match the trusted fingerprints I've already obtained out-of-band. But this just reintroduces the original problem.

@HW42

This comment has been minimized.

Show comment
Hide comment
@HW42

HW42 Feb 24, 2017

I integrated the sha1 collision check into git (experimental and hacky, but should work): https://github.com/HW42/git

It's only not clear to me if these tool can be easily used on actual hashes, rather on files which are to be hashed?

IIUC this is not possible since you need the intermediate hashing state which cannot be reconstructed from only the hash.

The above code simply replaces the git SHA1 function with the collision checking function and calls exit if suspicious data is hashed by git.

As a temporary solution, perhaps we might consider integrating the "collision detector" [1], presented by the authors of the attack, into qubes-builder?

Just using the "patched" git should do this. How to bootstrap is another question.

HW42 commented Feb 24, 2017

I integrated the sha1 collision check into git (experimental and hacky, but should work): https://github.com/HW42/git

It's only not clear to me if these tool can be easily used on actual hashes, rather on files which are to be hashed?

IIUC this is not possible since you need the intermediate hashing state which cannot be reconstructed from only the hash.

The above code simply replaces the git SHA1 function with the collision checking function and calls exit if suspicious data is hashed by git.

As a temporary solution, perhaps we might consider integrating the "collision detector" [1], presented by the authors of the attack, into qubes-builder?

Just using the "patched" git should do this. How to bootstrap is another question.

@HW42

This comment has been minimized.

Show comment
Hide comment
@HW42

HW42 Feb 24, 2017

Also hash used in gpg signature is configurable - we use SHA2 already.

Yes, for signing files, the hash is configurable. Not however for the OpenPGP key fingerprints as they use SHA1 as specified: RFC4880, 12.2. Key IDs and Fingerprints

I think for our use case of GPG, collision attacks on the keys are mostly useless. The only scenario of which I currently can think is that some new core developer who gets their UID signed by the master key could have a colliding key with another UID. So they could forge the origin of commits a little bit. Therefore I think currently we can wait here until some "standardized" solution exists.

HW42 commented Feb 24, 2017

Also hash used in gpg signature is configurable - we use SHA2 already.

Yes, for signing files, the hash is configurable. Not however for the OpenPGP key fingerprints as they use SHA1 as specified: RFC4880, 12.2. Key IDs and Fingerprints

I think for our use case of GPG, collision attacks on the keys are mostly useless. The only scenario of which I currently can think is that some new core developer who gets their UID signed by the master key could have a colliding key with another UID. So they could forge the origin of commits a little bit. Therefore I think currently we can wait here until some "standardized" solution exists.

@rootkovska

This comment has been minimized.

Show comment
Hide comment
@rootkovska

rootkovska Feb 24, 2017

Member

I think for our use case of GPG, collision attacks on the keys are mostly useless.

Exactly.

I integrated the sha1 collision check into git (experimental and hacky, but should work):

Great! Did you check it with all our repos? (in the fsck mode)? How long it takes?

Member

rootkovska commented Feb 24, 2017

I think for our use case of GPG, collision attacks on the keys are mostly useless.

Exactly.

I integrated the sha1 collision check into git (experimental and hacky, but should work):

Great! Did you check it with all our repos? (in the fsck mode)? How long it takes?

@ypid

This comment has been minimized.

Show comment
Hide comment
@ypid

ypid Feb 24, 2017

@HW42 Nice work! In cr-marcstevens/sha1collisiondetection#3 I just saw that Jeff King also works on this: https://github.com/peff/git/tree/jk/sha1dc

It seems Jeff King is making an attempt at using sha1collisiondetection in git, which would also resolve this issue:
http://marc.info/?l=git&m=148789111730142&w=2

ypid commented Feb 24, 2017

@HW42 Nice work! In cr-marcstevens/sha1collisiondetection#3 I just saw that Jeff King also works on this: https://github.com/peff/git/tree/jk/sha1dc

It seems Jeff King is making an attempt at using sha1collisiondetection in git, which would also resolve this issue:
http://marc.info/?l=git&m=148789111730142&w=2

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Feb 24, 2017

Member

git-evtag project contains also python script to calculate evtag (you need to manually add/compare it to tag message). But this script seems to not have limitation on handling only HEAD. In only needs that submodules are fetched beforehand.
Maybe this is the way to integrate it into qubes-builder?

Member

marmarek commented Feb 24, 2017

git-evtag project contains also python script to calculate evtag (you need to manually add/compare it to tag message). But this script seems to not have limitation on handling only HEAD. In only needs that submodules are fetched beforehand.
Maybe this is the way to integrate it into qubes-builder?

@ypid

This comment has been minimized.

Show comment
Hide comment
@ypid

ypid Feb 24, 2017

@marmarek's suggestion is to use the actual key file, e.g., qubes-developers-key.asc, but I don't see how that helps. Suppose I download that file. I now need to verify that the keys it contains are genuine. How can I do that? Presumably, by verifying that their fingerprints match the trusted fingerprints I've already obtained out-of-band. But this just reintroduces the original problem.

It helps in that one does not need to rely on the SHA1 fingerprint alone in case additional strong checksums are provided.

Also note that when a OpenPGP key gets signed by other OpenPGP key the signature is made over the primary public key packet and not its SHA1 fingerprint. Ref Anatomy of a GPG Key which I found to be an excellent read on how GnuPG and OpenPGP work in detail.

ypid commented Feb 24, 2017

@marmarek's suggestion is to use the actual key file, e.g., qubes-developers-key.asc, but I don't see how that helps. Suppose I download that file. I now need to verify that the keys it contains are genuine. How can I do that? Presumably, by verifying that their fingerprints match the trusted fingerprints I've already obtained out-of-band. But this just reintroduces the original problem.

It helps in that one does not need to rely on the SHA1 fingerprint alone in case additional strong checksums are provided.

Also note that when a OpenPGP key gets signed by other OpenPGP key the signature is made over the primary public key packet and not its SHA1 fingerprint. Ref Anatomy of a GPG Key which I found to be an excellent read on how GnuPG and OpenPGP work in detail.

@marmarek

This comment has been minimized.

Show comment
Hide comment
@marmarek

marmarek Feb 24, 2017

Member

Ah, it do need to checkout submodules first - at least initialize appropriate directories with .git directory. Generally git submodule works on currently checked out version, not arbitrary revision. And also git clone --recursive is ignored when git clone -n is used. This is mostly a problem when cloning new repository, or introducing new submodule. Otherwise it should be enough to run git submodule foreach git fetch before git-evtag.
I think the best we can achieve with git-evtag (if going this way in the first place) is:

  1. For fresh clone:
  • execute git clone -n
  • verify git tag signature
  • checkout it (including submodules)
  • verify git-evtag - if it fails, rollback the operation - simply remove the directory (it's already how qubes-builder behaves)
  1. For subsequent git fetch (make prepare-merge, make get-sources etc)
  • execute git fetch
  • verify git tag signature
  • execute git submodule foreach git fetch to fetch new commits in submodules
  • try to verify git-evtag - if it fails, nothing needs to be rolled back, because we haven't checkout the new commits

One problem with this approach is that, in the second case, if new submodule got introduced, git-evtag calculation would fail (because those commits weren't included in git submodule foreach git fetch). This is rather rare situation and I think we can handle it manually (doing that checkout and if git-evtag verification still fails, rolling back changes manually).

The other problem is that, in the first case, we have checked out sources before git-evtag verification. If verification fails, that repository is removed from disk, but there is short time span when it is there (**). And also some more git parts are exposed before verification of git objects referenced by fetched tag. In short: 1) fetch objects 2) verify tag, 3) checkout objects 4) verify checked out objects.

(*) When I write git-evtag, I mean that python implementation, which can operate on arbitrary revision, not HEAD.
(**) This potential race condition isn't a problem in upcoming github integration (QubesOS/qubes-builder-github#7) because we use builder-wide locks for all operations.

Member

marmarek commented Feb 24, 2017

Ah, it do need to checkout submodules first - at least initialize appropriate directories with .git directory. Generally git submodule works on currently checked out version, not arbitrary revision. And also git clone --recursive is ignored when git clone -n is used. This is mostly a problem when cloning new repository, or introducing new submodule. Otherwise it should be enough to run git submodule foreach git fetch before git-evtag.
I think the best we can achieve with git-evtag (if going this way in the first place) is:

  1. For fresh clone:
  • execute git clone -n
  • verify git tag signature
  • checkout it (including submodules)
  • verify git-evtag - if it fails, rollback the operation - simply remove the directory (it's already how qubes-builder behaves)
  1. For subsequent git fetch (make prepare-merge, make get-sources etc)
  • execute git fetch
  • verify git tag signature
  • execute git submodule foreach git fetch to fetch new commits in submodules
  • try to verify git-evtag - if it fails, nothing needs to be rolled back, because we haven't checkout the new commits

One problem with this approach is that, in the second case, if new submodule got introduced, git-evtag calculation would fail (because those commits weren't included in git submodule foreach git fetch). This is rather rare situation and I think we can handle it manually (doing that checkout and if git-evtag verification still fails, rolling back changes manually).

The other problem is that, in the first case, we have checked out sources before git-evtag verification. If verification fails, that repository is removed from disk, but there is short time span when it is there (**). And also some more git parts are exposed before verification of git objects referenced by fetched tag. In short: 1) fetch objects 2) verify tag, 3) checkout objects 4) verify checked out objects.

(*) When I write git-evtag, I mean that python implementation, which can operate on arbitrary revision, not HEAD.
(**) This potential race condition isn't a problem in upcoming github integration (QubesOS/qubes-builder-github#7) because we use builder-wide locks for all operations.

@HW42

This comment has been minimized.

Show comment
Hide comment
@HW42

HW42 Feb 24, 2017

Great! Did you check it with all our repos?

I did already checked those I had checked out. I now have run it over all the 69 repos GitHub listed. Nothing found (also no other 'fsck' warnings :]).

(In theory GitHub could have hidden bad objects from me.)

How long it takes?

This is normally not noticeable slower than git. Somebody made some measurements against OpenSSL for upstream (https://public-inbox.org/git/20170223230621.43anex65ndoqbgnf@sigill.intra.peff.net/, That's not my patch but it does the same thing and uses the same lib).

It looks like upstream will incorporate a very similar patch. So probably we can just wait until it's packaged for the common distros.

HW42 commented Feb 24, 2017

Great! Did you check it with all our repos?

I did already checked those I had checked out. I now have run it over all the 69 repos GitHub listed. Nothing found (also no other 'fsck' warnings :]).

(In theory GitHub could have hidden bad objects from me.)

How long it takes?

This is normally not noticeable slower than git. Somebody made some measurements against OpenSSL for upstream (https://public-inbox.org/git/20170223230621.43anex65ndoqbgnf@sigill.intra.peff.net/, That's not my patch but it does the same thing and uses the same lib).

It looks like upstream will incorporate a very similar patch. So probably we can just wait until it's packaged for the common distros.

@HW42

This comment has been minimized.

Show comment
Hide comment
@HW42

HW42 Feb 24, 2017

But this script seems to not have limitation on handling only HEAD. In only needs that submodules are fetched beforehand.

IIUC the C/rust version can be easily changed to do the same. The real problem is the handling of submodules as you already noticed.

The other problem is that, in the first case, we have checked out sources before git-evtag verification. If verification fails, that repository is removed from disk, but there is short time span when it is there (**).

We could use a temporary directory such that at least nothing accesses the files accidentally (I'm not sure if it's easy to convince git to do that in this case).

And also some more git parts are exposed before verification of git objects referenced by fetched tag. In short: 1) fetch objects 2) verify tag, 3) checkout objects 4) verify checked out objects.

I think that you need to process/parse a lot data before verifying it is in general a big limitation of git. The only thing you can skip (without submodules) is the checkout part AFAIK.

One option would be to mirror the repository locally and then temporarily clone+checkout them for verification. But I don't see how this could be done without a lot of "hackery" which we a) don't want to maintain and b) don't want to have in verification code.

HW42 commented Feb 24, 2017

But this script seems to not have limitation on handling only HEAD. In only needs that submodules are fetched beforehand.

IIUC the C/rust version can be easily changed to do the same. The real problem is the handling of submodules as you already noticed.

The other problem is that, in the first case, we have checked out sources before git-evtag verification. If verification fails, that repository is removed from disk, but there is short time span when it is there (**).

We could use a temporary directory such that at least nothing accesses the files accidentally (I'm not sure if it's easy to convince git to do that in this case).

And also some more git parts are exposed before verification of git objects referenced by fetched tag. In short: 1) fetch objects 2) verify tag, 3) checkout objects 4) verify checked out objects.

I think that you need to process/parse a lot data before verifying it is in general a big limitation of git. The only thing you can skip (without submodules) is the checkout part AFAIK.

One option would be to mirror the repository locally and then temporarily clone+checkout them for verification. But I don't see how this could be done without a lot of "hackery" which we a) don't want to maintain and b) don't want to have in verification code.

@andrewdavidwong

This comment has been minimized.

Show comment
Hide comment
@andrewdavidwong

andrewdavidwong Feb 25, 2017

Member

[Admittedly getting off-topic]

It helps in that one does not need to rely on the SHA1 fingerprint alone in case additional strong checksums are provided.

Ok, but in what case would a strong checksum be provided for a key file, and how would that checksum be made trustworthy? Would it be... PGP-signed? If so, we have a regress problem.

Also note that when a OpenPGP key gets signed by other OpenPGP key the signature is made over the primary public key packet and not its SHA1 fingerprint. Ref Anatomy of a GPG Key which I found to be an excellent read on how GnuPG and OpenPGP work in detail.

Ok, but that works better in situations where key trust is conveyed via a Web of Trust, which tends not to be the case in the Qubes community. For a variety of reasons, we rely far more on fingerprints. (One reason, I suspect, is that there doesn't yet exist a fully secure way to sign other people's keys without exposing the private master key doing the signing, even with Split GPG.)

Member

andrewdavidwong commented Feb 25, 2017

[Admittedly getting off-topic]

It helps in that one does not need to rely on the SHA1 fingerprint alone in case additional strong checksums are provided.

Ok, but in what case would a strong checksum be provided for a key file, and how would that checksum be made trustworthy? Would it be... PGP-signed? If so, we have a regress problem.

Also note that when a OpenPGP key gets signed by other OpenPGP key the signature is made over the primary public key packet and not its SHA1 fingerprint. Ref Anatomy of a GPG Key which I found to be an excellent read on how GnuPG and OpenPGP work in detail.

Ok, but that works better in situations where key trust is conveyed via a Web of Trust, which tends not to be the case in the Qubes community. For a variety of reasons, we rely far more on fingerprints. (One reason, I suspect, is that there doesn't yet exist a fully secure way to sign other people's keys without exposing the private master key doing the signing, even with Split GPG.)

@andrewdavidwong

This comment has been minimized.

Show comment
Hide comment
@andrewdavidwong

andrewdavidwong Feb 25, 2017

Member

Yes, for signing files, the hash is configurable.

Not however for the OpenPGP key fingerprints as they use SHA1 as specified: RFC4880, 12.2. Key IDs and Fingerprints

I think for our use case of GPG, collision attacks on the keys are mostly useless. [...]

You're right about collision attacks being mostly useless for our use case, but they were talking about second preimage attacks (which, we agree, are not an immediate concern right now).

Member

andrewdavidwong commented Feb 25, 2017

Yes, for signing files, the hash is configurable.

Not however for the OpenPGP key fingerprints as they use SHA1 as specified: RFC4880, 12.2. Key IDs and Fingerprints

I think for our use case of GPG, collision attacks on the keys are mostly useless. [...]

You're right about collision attacks being mostly useless for our use case, but they were talking about second preimage attacks (which, we agree, are not an immediate concern right now).

@ypid

This comment has been minimized.

Show comment
Hide comment
@ypid

ypid Feb 25, 2017

Ok, but in what case would a strong checksum be provided for a key file, and how would that checksum be made trustworthy? Would it be... PGP-signed? If so, we have a regress problem.

The strong checksum would be made trustworthy the same way any other OpenPGP fingerprint would be made trustworthy (print out, website, phone, t-shirt worn by core dev ;-) ). Just the instructions on how to check authenticity of the key would need to be extended.

But just hashing gpg -a --export is not very robust. One would need to only hash the primary public key packet. The following handy oneliner should do just that:

$ tmp_dir="$(mktemp -d)"; pushd "$tmp_dir" >/dev/null && gpg --export "427F 11FD 0FAA 4B08 0123  F01C DDFA 1A3E 3687 9494" > pub.key && gpgsplit pub.key && sha256sum ./*.public_key; popd >/dev/null && rm -r "$tmp_dir"
125d9b76ae515b819e882822fb97f531aa3a4d64791e1f244c89b2df23867c35  ./000001-006.public_key

gpgsha2fingerprint

I guess I will include such a hash when I next make a print out of my pubic OpenPGP key 😉

Ok, but that works better in situations where key trust is conveyed via a Web of Trust, which tends not to be the case in the Qubes community.

True. But at least all important Qubes OS keys are signed by the Qubes Master Signing Key.

ypid commented Feb 25, 2017

Ok, but in what case would a strong checksum be provided for a key file, and how would that checksum be made trustworthy? Would it be... PGP-signed? If so, we have a regress problem.

The strong checksum would be made trustworthy the same way any other OpenPGP fingerprint would be made trustworthy (print out, website, phone, t-shirt worn by core dev ;-) ). Just the instructions on how to check authenticity of the key would need to be extended.

But just hashing gpg -a --export is not very robust. One would need to only hash the primary public key packet. The following handy oneliner should do just that:

$ tmp_dir="$(mktemp -d)"; pushd "$tmp_dir" >/dev/null && gpg --export "427F 11FD 0FAA 4B08 0123  F01C DDFA 1A3E 3687 9494" > pub.key && gpgsplit pub.key && sha256sum ./*.public_key; popd >/dev/null && rm -r "$tmp_dir"
125d9b76ae515b819e882822fb97f531aa3a4d64791e1f244c89b2df23867c35  ./000001-006.public_key

gpgsha2fingerprint

I guess I will include such a hash when I next make a print out of my pubic OpenPGP key 😉

Ok, but that works better in situations where key trust is conveyed via a Web of Trust, which tends not to be the case in the Qubes community.

True. But at least all important Qubes OS keys are signed by the Qubes Master Signing Key.

@andrewdavidwong

This comment has been minimized.

Show comment
Hide comment
@andrewdavidwong

andrewdavidwong Feb 26, 2017

Member

Ah, I understand what you have in mind now. So, basically distributing a strong hash of (the appropriate content of) the Qubes Master Signing Key instead of directly distributing the latter's fingerprint? Yeah, I could see that working. Anyway, let's hope this part remains theoretical.

Member

andrewdavidwong commented Feb 26, 2017

Ah, I understand what you have in mind now. So, basically distributing a strong hash of (the appropriate content of) the Qubes Master Signing Key instead of directly distributing the latter's fingerprint? Yeah, I could see that working. Anyway, let's hope this part remains theoretical.

@ypid

This comment has been minimized.

Show comment
Hide comment
@ypid

ypid Feb 26, 2017

I guess if one decides to do this the strong hash should be provided in addition because SHA1 fingerprints are the default and the way I showed is not that common (and currently not needed).
</offtopic> :)

ypid commented Feb 26, 2017

I guess if one decides to do this the strong hash should be provided in addition because SHA1 fingerprints are the default and the way I showed is not that common (and currently not needed).
</offtopic> :)

@ypid ypid referenced this issue in kylemanna/docker-bitcoind May 15, 2017

Merged

Enforce the full OpenPGP fingerprint of ppa:bitcoin/bitcoin #43

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment