Release signed checksums along with directly signed ISOs

[ypid]

Hi

I think it would be a good idea to also release cryptographically strong checksums like SHA256 for ISOs along with signature files for the ISOs. Those checksums should be signed as well of course. The main reason for this is to have a second way to verify the Qubes OS installation media. You described it already quite good here: https://www.qubes-os.org/doc/InstallSecurity/

How can you be sure that the system the Installation media (Optical Discs seem to be the best option here) was created on to be trustworthy. You can’t. So one option is to verify the read-only media on other systems and see if it matches.

Verifying on Linux

Successfully tested:

• Checksum: sha256sum /dev/cdrom
• Signature: gpg --verify Qubes-R3.0-rc1-x86_64-DVD.iso.asc /dev/cdrom

Verifying on Windows

On Windows it is harder as it does not have GPG installed by default and I did not find a way to read directly from DVD. But it does include a checksum tool by now so the easier way if you just want to check the integrity of the Installation media is to calculate the checksum and compare it to the (hand-)written checksum. The checksum can then be verified by other means on multiple other systems via GPG and visiting https://www.qubes-os.org and check the checksum.

• Preparation: Create an image of the installation media: I tried it via CDBurnerXP and ImgBurn. Only ImgBurn created an correct image (file size matched). dd for Windows is probably also an option.
• Checksum: certutil -hashfile Qubes-R3.0-rc1-x86_64-DVD.iso SHA256
• Signature: gpg --verify Qubes-R3.0-rc1-x86_64-DVD.iso.asc /dev/cdrom, untested because my GPG 4 Windows could somehow not import the public key Qubes OS Release 3 Signing Key, not sure why. I downloaded it from https://github.com/QubesOS/qubes-secpack/tree/master/keys/release-keys and imported all three of them, R3 was the only key not appearing in the public key ring but ok.

BTW: The checksums for Qubes-R3.0-rc1-x86_64-DVD.iso seems to be:

• SHA256: 29f6cb815ddb7f5351f65356f167eebebbe066134538f89e4a97b5346920c085
• MD5: 11d3085370e8d1a752f8490952249938 (as posted by 107cwk)

Proposal

For each release, include a checksum file containing checksums of the ISO using multiple hash algorithms and sign it.

BTW: Any reason to sign the ISO directly instead of just signed the checksum file as for example Debian does it?

[adrelanos]

Does Qubes have existing scripts for OpenPGP signatures creation or do you do this by hand?

Is the output of sha256sum /dev/cdrom deterministic? If you burn various dvds using various dvd burners, will the checksum always be the same or vary?

At Whonix we have it all. Signed files, sha256 files, sha512 files, sha256 signature, sha512 signature, torrent files and torrent signatures. Check out the download table or folder. As well as sophisticated OpenPGP verification instructions. Those contain validating file names (OpenPGP signs blobs, not names of files) and checking signature creation time. As well as scripts to automate most of the release maintenance work.

(Why mention Whonix? Because I'll be concentrating on Qubes now. And this is stuff I did for Whonix, understand well, that works well. So I can potentially help with this.)

[adrelanos]

Does Qubes have existing scripts for OpenPGP signatures creation or do you do this by hand?

Is the output of sha256sum /dev/cdrom deterministic? If you burn various dvds using various dvd burners, will the checksum always be the same or vary?

At Whonix we have it all. Signed files, sha256 files, sha512 files, sha256 signature, sha512 signature, torrent files and torrent signatures. Check out the download table or folder. As well as sophisticated OpenPGP verification instructions. Those contain validating file names (OpenPGP signs blobs, not names of files) and checking signature creation time. As well as scripts to automate most of the release maintenance work.

(Why mention Whonix? Because I'll be concentrating on Qubes now. And this is stuff I did for Whonix, understand well, that works well. So I can potentially help with this.)

[ypid]

Well done 👍

sha256sum /dev/cdrom is probably not deterministic but I burned two DVDs of Qubes-R3.0-rc1-x86_64-DVD.iso and it works for both on various systems except the problem with CDBurnerXP, but those kinds of things can happen and there are solutions for them (http://unix.stackexchange.com/questions/84469/how-to-take-sha-1-sha-256-or-md5-of-cds-dvds).
(Edit: Also successfully tested it with an Debian jessie iso.)

BTW: Tails also releases a checksum of there ISO: https://tails.boum.org/download/index.en.html#index2h1

[ypid]

Well done 👍

sha256sum /dev/cdrom is probably not deterministic but I burned two DVDs of Qubes-R3.0-rc1-x86_64-DVD.iso and it works for both on various systems except the problem with CDBurnerXP, but those kinds of things can happen and there are solutions for them (http://unix.stackexchange.com/questions/84469/how-to-take-sha-1-sha-256-or-md5-of-cds-dvds).
(Edit: Also successfully tested it with an Debian jessie iso.)

BTW: Tails also releases a checksum of there ISO: https://tails.boum.org/download/index.en.html#index2h1

[ypid]

To make the checksum approach more robust I would propose to use many different algorithms (not only SHA2). Here is an example implementation:

#!/bin/sh

RELEASE="$1"

if [ -z "$RELEASE" ]
then
    echo "Base ISO file name as first parameter expected." 1>&2
    exit 1
fi
shift

if [ -z "$1" ]
then
    echo "Files for processing needed." 1>&2
    exit 1
fi

echo "# Checksum file for ${RELEASE}.
# For verification see https://www.qubes-os.org/doc/VerifyingSignatures/
# Short:
# * gpg --verify ${RELEASE}.checksums.asc
# * sha512sum --check ${RELEASE}.checksums
# Alternatively you can also run GPG directly against the ISO file:
# * gpg --verify ${RELEASE}.asc
#
# Format of this file: The first line is the sha512 checksum of the ISO. The following lines are checksums using a variety of different hash algorithms formated as BSD checksum file.
"
for file in "$@"
do
    sha512sum "$file"
    rhash --all --bsd "$file"
done

[ypid]

To make the checksum approach more robust I would propose to use many different algorithms (not only SHA2). Here is an example implementation:

#!/bin/sh

RELEASE="$1"

if [ -z "$RELEASE" ]
then
    echo "Base ISO file name as first parameter expected." 1>&2
    exit 1
fi
shift

if [ -z "$1" ]
then
    echo "Files for processing needed." 1>&2
    exit 1
fi

echo "# Checksum file for ${RELEASE}.
# For verification see https://www.qubes-os.org/doc/VerifyingSignatures/
# Short:
# * gpg --verify ${RELEASE}.checksums.asc
# * sha512sum --check ${RELEASE}.checksums
# Alternatively you can also run GPG directly against the ISO file:
# * gpg --verify ${RELEASE}.asc
#
# Format of this file: The first line is the sha512 checksum of the ISO. The following lines are checksums using a variety of different hash algorithms formated as BSD checksum file.
"
for file in "$@"
do
    sha512sum "$file"
    rhash --all --bsd "$file"
done

[marmarek]

Checksums uploaded:
http://ftp.qubes-os.org/iso/

But this isn't fully done, because needs some documentation (those files aren't linked anywhere), including how to verify them

[marmarek]

Checksums uploaded:
http://ftp.qubes-os.org/iso/

But this isn't fully done, because needs some documentation (those files aren't linked anywhere), including how to verify them

[unman]

I've been looking at rewriting the Downloads and Verifying Signatures pages in response to #1114 .
I'll take a stab at including this as well if you like.

[unman]

I've been looking at rewriting the Downloads and Verifying Signatures pages in response to #1114 .
I'll take a stab at including this as well if you like.

[ypid]

Sure, sounds good 👍

[ypid]

Sure, sounds good 👍

[andrewdavidwong]

The digests should probably be labeled. For example, the content of Qubes-R3.0-rc3-x86_64-DVD.iso.DIGESTS is:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

9b866b55cb586421f329e513234f7b06 *Qubes-R3.0-rc3-x86_64-DVD.iso
438f619adde30eb334c3ae3cce3a79ed14225ea4 *Qubes-R3.0-rc3-x86_64-DVD.iso
e04020ba4d1651c289b35723a799ac9dc7248cb26a63d4db7b47baf70648b750 *Qubes-R3.0-rc3-x86_64-DVD.iso
668e1cfab5d645c5dd6db12225094b898a815e2e6242e181ac43e4df3a0e620a5c62ff642ac27aa4899e784baff69ff12f16e9ea6f2ad1d68d79914e36a63239 *Qubes-R3.0-rc3-x86_64-DVD.iso
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCAAGBQJV+efbAAoJEMsRyh0D+lCCUyMP/j8JCijpqygUIDWBG4i2j5i8
u4AABDbwsmFZaKlnoYm4lFozNpsa3SDJkUtLCf2Of77B1EEAf93s/ee3j3T8HGL0
CIc4uLOmB+QBvc4PpduPCFhxpInzczTVUJNhcRnDTw65ZrUIvazMm1HXxifqsmJO
oCeMWo5jlMYJY8j+PnTZmJdZSlmq1icd6FdIbrAq48I3BOUngvbxi25XvWpXzpBO
WYYZDd/qjfMHXhyHG9AWbdgbIBFIZNsFmYtE3ID97NOp7p5oOxSXYmGHsZxWEDPf
7eE5TK4QohJrYkem9HlCn48tBmfbzeKDDIfXG5wPRe8kTZ2DoYkNiY353HBQVLsr
qoaWlAtP0PtfdE4KSbE56g2rK+X5Lp2OdAYIM6WRTsrByPai2SsJ8WYpSkm1hhlx
0d/ZiK6+OURzaQjCshJ1i2q4vrBNE7QLv3ExL5c1noipH1WBhpja6+G6zvScIl7P
2sVk+Lh++AZgAACBOk0Gb78EVKGc2f+oiBiSvlAyfv0i64hUtqZg2QN69A1UqEDl
j7rJpypPwin9RURX7tEgDHdGN87SwPD15rB9Lv5FRP/Arv/CXJUNMDtRdY1so8fx
8v64wL8PhSpN9QUYHp3PO/0C4y3ECB70TpkKpcTwBJSiFzPchmNDP18VDZOsfXoY
QHewixxdvfqiaLlT+cn2
=JX6X
-----END PGP SIGNATURE-----

But it's not entirely clear which hash functions were used to generate each of those four outputs.

[andrewdavidwong]

The digests should probably be labeled. For example, the content of Qubes-R3.0-rc3-x86_64-DVD.iso.DIGESTS is:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256

9b866b55cb586421f329e513234f7b06 *Qubes-R3.0-rc3-x86_64-DVD.iso
438f619adde30eb334c3ae3cce3a79ed14225ea4 *Qubes-R3.0-rc3-x86_64-DVD.iso
e04020ba4d1651c289b35723a799ac9dc7248cb26a63d4db7b47baf70648b750 *Qubes-R3.0-rc3-x86_64-DVD.iso
668e1cfab5d645c5dd6db12225094b898a815e2e6242e181ac43e4df3a0e620a5c62ff642ac27aa4899e784baff69ff12f16e9ea6f2ad1d68d79914e36a63239 *Qubes-R3.0-rc3-x86_64-DVD.iso
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQIcBAEBCAAGBQJV+efbAAoJEMsRyh0D+lCCUyMP/j8JCijpqygUIDWBG4i2j5i8
u4AABDbwsmFZaKlnoYm4lFozNpsa3SDJkUtLCf2Of77B1EEAf93s/ee3j3T8HGL0
CIc4uLOmB+QBvc4PpduPCFhxpInzczTVUJNhcRnDTw65ZrUIvazMm1HXxifqsmJO
oCeMWo5jlMYJY8j+PnTZmJdZSlmq1icd6FdIbrAq48I3BOUngvbxi25XvWpXzpBO
WYYZDd/qjfMHXhyHG9AWbdgbIBFIZNsFmYtE3ID97NOp7p5oOxSXYmGHsZxWEDPf
7eE5TK4QohJrYkem9HlCn48tBmfbzeKDDIfXG5wPRe8kTZ2DoYkNiY353HBQVLsr
qoaWlAtP0PtfdE4KSbE56g2rK+X5Lp2OdAYIM6WRTsrByPai2SsJ8WYpSkm1hhlx
0d/ZiK6+OURzaQjCshJ1i2q4vrBNE7QLv3ExL5c1noipH1WBhpja6+G6zvScIl7P
2sVk+Lh++AZgAACBOk0Gb78EVKGc2f+oiBiSvlAyfv0i64hUtqZg2QN69A1UqEDl
j7rJpypPwin9RURX7tEgDHdGN87SwPD15rB9Lv5FRP/Arv/CXJUNMDtRdY1so8fx
8v64wL8PhSpN9QUYHp3PO/0C4y3ECB70TpkKpcTwBJSiFzPchmNDP18VDZOsfXoY
QHewixxdvfqiaLlT+cn2
=JX6X
-----END PGP SIGNATURE-----

But it's not entirely clear which hash functions were used to generate each of those four outputs.

[ypid]

Hash: SHA256

? Works for me. I proposed an alternative (BSD) Format above but this is less common. My script would generate a "combined" checksum file which is understood (with warnings) as both formats (sha256sum and rhash).

[ypid]

Hash: SHA256

? Works for me. I proposed an alternative (BSD) Format above but this is less common. My script would generate a "combined" checksum file which is understood (with warnings) as both formats (sha256sum and rhash).

[andrewdavidwong]

Hash: SHA256

? Works for me.

No, that's the hash function used for the PGP signature. It's not part of the content of the message.

I'm suggesting that instead of this:

9b866b55cb586421f329e513234f7b06 *Qubes-R3.0-rc3-x86_64-DVD.iso
438f619adde30eb334c3ae3cce3a79ed14225ea4 *Qubes-R3.0-rc3-x86_64-DVD.iso
e04020ba4d1651c289b35723a799ac9dc7248cb26a63d4db7b47baf70648b750 *Qubes-R3.0-rc3-x86_64-DVD.iso
668e1cfab5d645c5dd6db12225094b898a815e2e6242e181ac43e4df3a0e620a5c62ff642ac27aa4899e784baff69ff12f16e9ea6f2ad1d68d79914e36a63239 *Qubes-R3.0-rc3-x86_64-DVD.iso

We should do this:

MD5 9b866b55cb586421f329e513234f7b06 *Qubes-R3.0-rc3-x86_64-DVD.iso
SHA1 438f619adde30eb334c3ae3cce3a79ed14225ea4 *Qubes-R3.0-rc3-x86_64-DVD.iso
SHA256 e04020ba4d1651c289b35723a799ac9dc7248cb26a63d4db7b47baf70648b750 *Qubes-R3.0-rc3-x86_64-DVD.iso
SHA512 668e1cfab5d645c5dd6db12225094b898a815e2e6242e181ac43e4df3a0e620a5c62ff642ac27aa4899e784baff69ff12f16e9ea6f2ad1d68d79914e36a63239 *Qubes-R3.0-rc3-x86_64-DVD.iso

[andrewdavidwong]

Hash: SHA256

? Works for me.

No, that's the hash function used for the PGP signature. It's not part of the content of the message.

I'm suggesting that instead of this:

9b866b55cb586421f329e513234f7b06 *Qubes-R3.0-rc3-x86_64-DVD.iso
438f619adde30eb334c3ae3cce3a79ed14225ea4 *Qubes-R3.0-rc3-x86_64-DVD.iso
e04020ba4d1651c289b35723a799ac9dc7248cb26a63d4db7b47baf70648b750 *Qubes-R3.0-rc3-x86_64-DVD.iso
668e1cfab5d645c5dd6db12225094b898a815e2e6242e181ac43e4df3a0e620a5c62ff642ac27aa4899e784baff69ff12f16e9ea6f2ad1d68d79914e36a63239 *Qubes-R3.0-rc3-x86_64-DVD.iso

We should do this:

MD5 9b866b55cb586421f329e513234f7b06 *Qubes-R3.0-rc3-x86_64-DVD.iso
SHA1 438f619adde30eb334c3ae3cce3a79ed14225ea4 *Qubes-R3.0-rc3-x86_64-DVD.iso
SHA256 e04020ba4d1651c289b35723a799ac9dc7248cb26a63d4db7b47baf70648b750 *Qubes-R3.0-rc3-x86_64-DVD.iso
SHA512 668e1cfab5d645c5dd6db12225094b898a815e2e6242e181ac43e4df3a0e620a5c62ff642ac27aa4899e784baff69ff12f16e9ea6f2ad1d68d79914e36a63239 *Qubes-R3.0-rc3-x86_64-DVD.iso

[marmarek]

The file is formate that way to work directly with tool like sha256sum -c etc. Adding some prefix would break that. Maybe we should simply add
a README.txt in that directory?

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[marmarek]

The file is formate that way to work directly with tool like sha256sum -c etc. Adding some prefix would break that. Maybe we should simply add
a README.txt in that directory?

Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?

[andrewdavidwong]

I suppose we could just explain it in the documentation.

[andrewdavidwong]

I suppose we could just explain it in the documentation.

[andrewdavidwong]

OK, I created a new section which explains the digests. I included instructions for generating the hash values using openssl and verifying the clearsigned file itself using gpg, but I didn't mention sha256sum or anything else. Instructions regarding those (and any other good alternative methods, especially for Windows and Mac users) would be welcome additions to the documentation.

[andrewdavidwong]

OK, I created a new section which explains the digests. I included instructions for generating the hash values using openssl and verifying the clearsigned file itself using gpg, but I didn't mention sha256sum or anything else. Instructions regarding those (and any other good alternative methods, especially for Windows and Mac users) would be welcome additions to the documentation.

[marmarek]

Thanks @axon-qubes !

[marmarek]

Thanks @axon-qubes !