cmd/go: macOS on arm64 requires codesigning

[FiloSottile]

On the production Apple Silicon machines, Go binaries are killed at start. #38485 (comment)

It looks like all binaries need to be codesigned now, and indeed running codesign -s - on them lets them run correctly.

This stops go test and go run from working, and requires an extra step after go build to get a functional binary.

This also affects the bootstrapped compiler itself.

[FiloSottile]

@cherrymui indeed, binaries produced by the machine's clang are codesigned, in what looks like the same exact way as the output of codesign -s -.

filippo@Filippos-MacBook-Pro tmp % clang hello.c
filippo@Filippos-MacBook-Pro tmp % ./a.out
Hello M1!
filippo@Filippos-MacBook-Pro tmp % codesign -d -v a.out                                 
Executable=/Users/filippo/tmp/a.out
Identifier=a.out
Format=Mach-O thin (arm64)
CodeDirectory v=20400 size=510 flags=0x20002(adhoc,linker-signed) hashes=13+0 location=embedded
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements=none

[FiloSottile]

Actually, codesign outputs don't have the linker-signed flag.

[cherrymui]

Thanks, @FiloSottile !

Interesting. The C compiler on the DTK doesn't do that...

Could you try one more thing: run clang -v hello.c to see if the C compiler invokes codesign or how it signs the binary? Thanks!

[FiloSottile]

It doesn't look like the linker invocation has anything special, so I assume ld just does that (and @jedisct1 said so on Twitter as well https://twitter.com/jedisct1/status/1328862207715794946).

"/Library/Developer/CommandLineTools/usr/bin/ld" -demangle -lto_library /Library/Developer/CommandLineTools/usr/lib/libLTO.dylib -no_deduplicate -dynamic -arch arm64 -platform_version macos 11.0.0 11.0 -syslibroot /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk -o a.out -L/usr/local/lib /var/folders/jh/3ydm4lxd71s2__g_x4hny6r00000gn/T/hello-580ca7.o -lSystem /Library/Developer/CommandLineTools/usr/lib/clang/12.0.0/lib/darwin/libclang_rt.osx.a

I expect external linking should generate codesigned binaries, but trying `go build -ldflags="-linkmode=external" fails immediately with

# tmp
loadinternal: cannot find runtime/cgo

[cherrymui]

loadinternal: cannot find runtime/cgo

This is not a failure. It prints a message but it should successfully link the binary.

[FiloSottile]

You're right, I do get the binary. Interestingly, it comes out linker-signed but doesn't run, and codesign doesn't work on it.

filippo@Filippos-MacBook-Pro tmp % codesign -d -v tmp                                                       
Executable=/Users/filippo/tmp/tmp
Identifier=a.out
Format=Mach-O thin (arm64)
CodeDirectory v=20400 size=7038 flags=0x20002(adhoc,linker-signed) hashes=217+0 location=embedded
Signature=adhoc
Info.plist=not bound
TeamIdentifier=not set
Sealed Resources=none
Internal requirements=none
filippo@Filippos-MacBook-Pro tmp % ./tmp             
zsh: killed     ./tmp
filippo@Filippos-MacBook-Pro tmp % codesign -s - tmp 
tmp: the codesign_allocate helper tool cannot be found or used

[cherrymui]

Thanks!

Interesting... I'll see if it is possible to generate LC_CODE_SIGNATURE in the linker. If that's not possible, maybe shell out codesign.

Any down side for that? If the user wants to sign with a different identity, would that still be possible?

[ianlancetaylor]

I don't understand this. What is the point of code signing, if every execution of clang or the Go compiler produces a code signed binary?

[FiloSottile]

Any down side for that?

I guess the main problem would be with cross-compilation, but anyway I guess adhoc signatures only work on the same machine where they are generated.

This would be a major pain for projects distributing binaries. I wonder how Homebrew deals with it, since by default they install pre-built "Bottles".

What is the point of code signing, if every execution of clang or the Go compiler produces a code signed binary?

I don't know, but I can imagine 1) to normalize the execution path such that it always check signatures unconditionally, simplifying it and 2) to block binaries generated on other machines (and not properly signed by a trusted Developer ID) which before was sort of the role of the quarantine attributes.

[terinjokes]

You're right, I do get the binary. Interestingly, it comes out linker-signed but doesn't run, and codesign doesn't work on it.

My understanding is this is a bug already reported to Apple, see Homebrew/brew#9082 (comment)

I wonder how Homebrew deals with it, since by default they install pre-built "Bottles".

Per Homebrew/brew#7857 (comment) I don't think they're close to code signing bottles on ARM yet.

[rolandshoemaker]

2. to block binaries generated on other machines (and not properly signed by a trusted Developer ID) which before was sort of the role of the quarantine attributes.

Looks like binaries aren't required to have a signature linked to a Developer ID if they were signed elsewhere. I was able to apply an ad-hoc signature to a binary on my Intel mac and transfer it to my M1 mac and it ran just fine. Appears like binaries just need a signature, doesn't really seem to matter where it came from.