Generate checksums of downloadable files

[andreastt]

For security and auditing reasons we should generate a list of checksums of the downloadable files we offer from the geckodriver project.

[bitmvr]

Couldn't agree more. I rely heavily on checksums while troubleshooting. Thanks for creating the issue!

[shwoodard]

Please?! You're shipping binaries here that a lot of people rely on in secure environments. Would be nice to know we got what you wanted us to have.

[andreastt]

Patches accepted (-:

[shwoodard]

Hmm... I think it's just a matter of computing the sha before you upload~ and adding it as a comment when creating the release.

[jgraham]

tbh the threat model here seems very limited. Does publishing a hash of the file defend against anything except accidentally corrupt downloads? The files are served over https from a well-known location and any attacker with access to the server could equally replace the files containing the hashes.

I'm happy to publish the hashes, but if there's some big security win it isn't obvious to me.

[andreastt]

I’m somewhat concerned by the fact that geckodriver gets re-distributed through 3rd party sources such as the NuGet .NET package manager. There also exists a Node.js NPM module that automatically downloads geckodriver. In both these cases it would be good to test the downloads against the known-good checksums, but in principle I agree with you: the actual attack surface here is small. The benefit mostly lies in auditing.

[jgraham]

I think that scenario only makes sense if there exists a user who

• Installs geckodriver using some third party tool
• Has access to the zip/tar file downloaded by that tool
• Has the knowledge and ability to generate a checksum of that download
• Can download the checksum file from our releases page
• Cannot simply download the package file from our releases page (e.g. because the download is too large) to verify identity directly.

I doubt such a user exists, particularly because geckodriver is so small (by modern standards). There might be a stronger argument relating to a 4Gb ISO file or similar. If you don't trust the third party tool to do the right thing here then why do you trust it not to execute malicious code itself?

[bitmvr]

I see this as an opportunity to help standardize the driver efforts within the testing community.

Things that would help with that is a parseable file, preferably JSON. This JSON file would contain a history of all releases. You would have an object for LATEST plus other objects by version VERSION. That object would contain the path to the driver and its checksum. This would allow those of us in the community to develop efficient driver managers that don't rely on heavy and often times annoying HTML parsers with the ability to confidently alert the users/admins that the drivers were installed/updated successfully. Such tools would then help those who run internal grids as well as companies as SauceLabs or BrowserStack.

[andreastt]

Common practice in this area is to provide, per release, one PGP-signed file per archive or one file listing the checksums for all archives. We should try very hard not to invent a new standard. One PGP-signed .asc file per archive we publish will ensure existing tooling in this area can be used.

Because third party resources may start distributing geckodriver at any point without our approval (they don’t need it), there should be a way to verify that the distributed archives aren’t corrupted in the process. When an archive is redistributed, most serious package managers will do implicit checksum verification on behalf of the user. By PGP-signing the checksums, it is possible to verify that the checksum file too was not maliciously manipulated (web of trust, &c.).

[bloudraak]

This is in response to #1116 being closed.

Checksums offer no guarantees in terms of security and auditing. In 2016 a hacker was able to gain access to a Linux Distro, replace the ISO images and the checksums.

Within regulated industries, we need the ability to offer strong guarantees of integrity (the file was not modified after it was officially built), and its reputation (the file originated from an official source, e.g. Mozilla Build Servers). The gecko driver is used to run browser tests against production systems. Should the binary be compromised (often months after it was downloaded) and a backdoor is added to send all "activities" to an untrusted source, we may just have increased the attack surface by giving potential attackers to authenticated pages (which in many cases increases the attack surface a thousand fold).

Here's how it works in practice in most organizations I worked at. A binary resides on a system, often with origins unknown. Some third party or homegrown software will check its digital signature and validate whether the vendor can be trusted. Depending on the data classification of the system, this may flag a security incident, the file is quarantined, or a friendly reminder to the user about unauthorized software. This isn't just any signature. On Windows, you'll need to look at Authenticode Code Signing. On Linux, look at signing kernel modules.

The signature is embedded into the binary itself (and not just the installer, or archived used to obtain it).
Other systems can then check the integrity and validate the reputation of the binary using the signature stored therein. Each platform implements this differently and thus, there is no one solution fits all.

My concern is mostly Microsoft Windows since most of the industries I work in use Firefox in that environment.

Naturally, I'm assuming that Mozilla is building the gecko driver using the same infrastructure to build Firefox. If that is the case, why not just sign the gecko driver the same way as with Firefox.

That said, having a checksum (SHA256 or MD5) that accompanies the download can facilitate better automation in that the binary can be checked for corruption.

[whimboo]

For reference issue #1116 has been reopened, and will be handle differently.

[whimboo]

@andreastt shall we close this issue in favor of #1116?

[andreastt]

@whimboo We still need checksums for the downloadable .zips?

[whimboo]

We should get those mostly for free I think when building via taskcluster. But sure, lets leave this issue open until https://bugzilla.mozilla.org/show_bug.cgi?id=1372587 has been fixed.

[andreastt]

Another thing that was discussed earlier in this thread was PGP signing the checksums so users know they are correct and from an authenticated source, but we can split that out to a separate task after we have checksums accompanying the release notes.

[whimboo]

Also this we should have automatically or with very minimal work. For example see: https://archive.mozilla.org/pub/firefox/nightly/latest-mozilla-central/. Lookout for both the checksum files of a platform.

[whimboo]

With https://bugzilla.mozilla.org/show_bug.cgi?id=1577110 fixed on May 8th we have signed geckodriver binaries for Mac and Windows in our CI. For Linux we have checksum files. That means we will make use of these builds starting with the next 0.27.0 release.

For the latest nightly build see:
https://firefox-ci-tc.services.mozilla.com/tasks/index/gecko.v2.mozilla-central.latest.geckodriver

[gtkpr]

tbh the threat model here seems very limited. Does publishing a hash of the file defend against anything except accidentally corrupt downloads? The files are served over https from a well-known location and any attacker with access to the server could equally replace the files containing the hashes.

I'm happy to publish the hashes, but if there's some big security win it isn't obvious to me.

Couldn't you just post the hash in multiple places (i.e. at least one other place other than github) as an extra line of defense? Creates at least twice as much work for the malicious actor. If the checksums that are posted publicly in two (or, ideally, multiple) locations do not correspond, then something is amiss.

[gtkpr]

With https://bugzilla.mozilla.org/show_bug.cgi?id=1577110 fixed on May 8th we have signed geckodriver binaries for Mac and Windows in our CI. For Linux we have checksum files. That means we will make use of these builds starting with the next 0.27.0 release.

For the latest nightly build see:
https://firefox-ci-tc.services.mozilla.com/tasks/index/gecko.v2.mozilla-central.latest.geckodriver

Where do I find the checksum for any given downloadable geckodriver?

[whimboo]

Given that builds for Windows and MacOS are all signed with a certificate from Mozilla we do not supply checksum files for these builds. Only on Linux were no signing takes place you can find the checksum files (.asc extension) beside the builds. This applies to the above Nightly and also release builds.