[Feature]: add checksum validation to browser installation

[Eric-Arellano]

🚀 Feature Request

Add to playwright-core a file with the expected SHA256 checksums for the browser binaries installed by browsers.json. playwright install will validate that the checksum matches what is expected after downloading binaries. If there is a mismatch, the install process will fail.

The script roll_browser.js would be updated to embed the checksums into playwright-core in the same PR that bumps the revision number. It would generate the SHA256 checksum based on the downloaded file.

Example

No response

Motivation

This proposal is to improve supply chain security for all Playwright browser users. Checksum validation substantially reduces the risk of a bad actor overwriting the binaries hosted in Azure with malicious binaries.

Why do I propose storing the checksums in the source code itself, rather than pulling it from the server? Pulling the checksums from the same server where the binaries are downloaded would not give much security improvement because a bad actor could simply rewrite the checksums to match their malicious binary.

Given that npmjs.com dependencies are immutable, once a playwright-core version is released, the embedded checksums cannot be compromised. This means that an attacker who compromises Azure cannot inject malicious binaries for existing releases.

My proposal does still have a limitation: if Azure was compromised during the specific roll_browser.js execution, and that compromise goes undetected until after the PR merges and the version ships to NPM, then the new version would have malicious checksums. (The bad version could be yanked once detected.)

However, despite this limitation, this feature still substantially improves the security posture:

• Current state: Azure compromises at any time can retroactively compromise all historical Playwright releases
• Proposed state: Azure compromise only affects versions rolled with roll_browser.js during the narrow compromise window; existing releases remain protected. There is a chance to detect the compromise before the bad roll_browser.js result is merged and released.

I would be interested in contributing this feature on behalf of IBM.

[Skn0tt]

Hi Eric! Thanks for opening the feature request, this makes sense. In your view, how would an effort like this relate to signing binaries? (see #35159)

[Eric-Arellano]

Ah, cool, I hadn't seen #35159.

Signatures are superior to checksum validation: they verify both integrity and authenticity, whereas checksums only validate integrity. With signatures, an attacker would need to both compromise Azure CDN to upload the malicious binaries and compromise the private key to sign them. Whereas with checksums, an attacker only needs to compromise Azure CDN to upload the malicious binaries. A good signature implementation will make it hard to compromise the private key, so it's an extra layer of defense.

(With signatures, it is important that the public key can be trusted. Otherwise, someone could change the public key to a malicious key. That is solved by publishing the public key with something like https://www.sigstore.dev.)

--

However, it wouldn't be wrong to keep checksum validation at the same time as signatures. Only, it'd be somewhat redundant.

If #35159 is going to take more time to get working, then I would encourage doing checksum validation is an immediate substantial win. While it's not as robust as signatures, it does substantially improve the security posture.

Adding checksums would hopefully be little cost to the project because it integrates nicely into the roll_browser.js workflow. I'd also be willing to contribute the patch, including following any practices you want like adding tests.

[Skn0tt]

Let's wait for signatures then. This build infra work always takes a little longer than expected.