sign binaries

[ry]

No description provided.

[jornfranke]

Is there a plan when the binaries will be signed?

[ghost]

Found on HN today: https://github.com/indygreg/PyOxidizer/tree/main/apple-codesign

apple-codesign is a crate implementing functionality related to code signing
on Apple platforms.

All functionality is implemented in pure Rust and doesn't require any 3rd party
or proprietary software nor do we require running on Apple platforms.

I wonder if this can be used for Deno binaries?

[jornfranke]

Alternatively, there is also cosign: https://github.com/SigStore/cosign
It has also hardware support for signing including hardware used in major cloud vendors (if CI pipeline runs in one of those)

[cowboyd]

@waynebloss Their rcodesign binary does appear to work on Deno compiled binaries, so it could be an option. I have not tried notarizing and / or stapling

[cowboyd]

UPDATE:

We tried to codesign using the apple-codesign project linked above, but it corrupted the resulting executable and trying to run it resulted in the following error:

#
# Fatal process OOM in Failed to reserve virtual memory for CodeRange
#

We tried to shave this yak from all manner of different positions, but in the end, we couldn't get it to work. For the time being, we're just going to follow the advice here https://users.rust-lang.org/t/distributing-cli-apps-on-macos/70223 and have users install via curl. Since our use-case is a developer tool this is acceptable for now.

[jornfranke]

Probably apple-codesign is not the right tool for this. Why not https://github.com/SigStore/cosign?

[jornfranke]

or if you want to be more traditional one can sign the binaries with GnuPG: https://www.gnupg.org/gph/en/manual/x135.html

[cowboyd]

@jornfranke We did look into cosign, but it was not clear from the website or the documentation that it would solve the specific problem at hand which is to distribute a single executable binary to users on macOS. In the end, we decided not to sign at all for two reasons. First, whether we used apple-codesign, or Apple's own codesign tool, doing so corrupted the executable created by deno compile. It seemed likely that we would achieve a similar result no matter which tool we used, so we stopped our exploration there because of the second point which is that code signing alone is not enough.

In order to run an executable downloaded via your browser, you have to have it notarized by Apple's Gatekeeper service. However, Gatekeeper only accepts .zip, .pkg, or .app archives which introduces its own deployment headaches beyond "download and go"

That is, until we found out that if you installed via curl like the rest of humanity, everything was "fine." And for those coming to this thread who were ignorant like we were let me re-iterate. You do not need to sign your binaries in order to distribute them on macOS. Note that the deno executable itself is not signed:

$ codesign -dv  `which deno`
Executable=/opt/homebrew/Cellar/deno/1.25.1/bin/deno
Identifier=deno-4aaad70ccec2ac75
Format=Mach-O thin (arm64)
CodeDirectory v=20400 size=628120 flags=0x20002(adhoc,linker-signed) hashes=19625+0 location=embedded
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements=none

In an ideal world, we would sign our CLI, but since it is for developers, and installation via curl is acceptable, we decided to punt on it for now.

To get code signing to work reliably, deno compile either needs to emit strictly valid Mach-O binaries, or there needs to be a consensus solution on how to distribute CLIs as .pkg archives; perhaps both. For our part, we'll likely revisit this once our requirements change, or the cost of making it work is low enough, but until then it's unsigned binaries for us. The fact that deno compile even made this conversation possible is enough.

[jornfranke]

Thanks for the detailed response. Then the non-existing description of the issue is misleading. I do not care about Apple signing, but in general all binaries (linux, windows etc.) should be signed so I can verify myself that they have not been tampered with. Maybe then I should create a dedicated issue for this?

[cowboyd]

@jornfranke This could be what you're looking for #11154

There, @AlexanderOMara lays out a method to make it work for Mach-O, but from my (limited) understanding of the subject, it would require a separate strategy for assembling the standalone binary for each platform executable format (Mach-O, ELF, PE) rather than the magical trailer strategy used now.

[AlexanderOMara]

@jornfranke This could be what you're looking for #11154

There, @AlexanderOMara lays out a method to make it work for Mach-O, but from my (limited) understanding of the subject, it would require a separate strategy for assembling the standalone binary for each platform executable format (Mach-O, ELF, PE) rather than the magical trailer strategy used now.

For Windows you could either add a new data section or tuck the data into the PE resources and read that.

For Linux I'm not sure how common code signing is, but you can quite-easily add a new unloaded section to an ELF binary which can easily be read from disk.

[jornfranke]

the signing does not need to be in the binary. It can be perfectly a dedicated file with the signature.
I mean sth. like for instance in Apache Maven:

• Binary: https://dlcdn.apache.org/maven/maven-3/3.8.6/binaries/apache-maven-3.8.6-bin.tar.gz
• Signature: https://downloads.apache.org/maven/maven-3/3.8.6/binaries/apache-maven-3.8.6-bin.tar.gz.asc
  And I can verify using the public key of the Apache Foundation (different host etc.): https://downloads.apache.org/maven/KEYS

The process is described here: https://infra.apache.org/release-signing#verifying-signature

[tolmasky]

Any reason not to use the same approach as node, and simply provide a clearsigned shasum file that can be (optionally) downloaded alongside any given release. This is similar to what @jornfranke is suggesting, and also allows existing Dockerfiles/scripts to use the same boilerplate they already use to verify node releases as with deno.

Basically, you have:

1. The actual binary, unchanged: https://github.com/denoland/deno/releases/download/1.29.2/deno-x86_64-apple-darwin.zip
2. A SHASUMS256.txt.asc file, which contains the sha-256 checksums of each platform release, in cleartext, with the signature attached at the end.

You can thus verify the SHASUMS256.txt.asc file just by running gpg --batch --decrypt --output SHASUMS.txt SHASUMS256.txt.asc

Once that is confirmed, you can then check that the checksum for deno-x86_64-apple-darwin.zip matches the one in the text file. You can look at the node SHASUMS256.txt.asc file here: https://nodejs.org/dist/v0.10.48/SHASUMS256.txt.asc (this will load in your browser and you can read it directly since this method does not compress anything).

[coreybutler]

While there is benefit to verifying the integrity of executables with signing/verification, it doesn't address a core problem. Most consumers don't verify downloads. This is the primary reason why Microsoft SmartScreen exists in Windows environments.

If the binary cannot be signed with an EV certificate, Windows won't trust it. In most company environments, this will prevent programs from running or being installed.

[crowlKats]

@tolmasky for that there is #7253

[tolmasky]

@crowlKats Excellent -- as far as I can tell this doesn't cover signature though right? It seems like what I am describing is to accomplish this bug through that one. Is it worth referencing this bug there too?

[lucasfernog-crabnebula]

UPDATE:

We tried to codesign using the apple-codesign project linked above, but it corrupted the resulting executable and trying to run it resulted in the following error:

#
# Fatal process OOM in Failed to reserve virtual memory for CodeRange
#

We tried to shave this yak from all manner of different positions, but in the end, we couldn't get it to work. For the time being, we're just going to follow the advice here https://users.rust-lang.org/t/distributing-cli-apps-on-macos/70223 and have users install via curl. Since our use-case is a developer tool this is acceptable for now.

in case anyone sees this, I've also faced this issue, and it seems like it only happens when you code sign a macOS binary with the hardened runtime flag. The solution is to also include this entitlement:

<?xml version="1.0" encoding="UTF-8"?>
                <!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
                <plist version="1.0">
                <dict>
                    <key>com.apple.security.cs.allow-unsigned-executable-memory</key>
                    <true />
                </dict>
                </plist>