Track tasks and feature requests
Join 40 million developers who use GitHub issues to help identify, assign, and keep track of the features and bug fixes your projects need.
Sign up for free See pricing for teams and enterprisesall: ensure that Go toolchain meets Apple’s notarization requirements #34986
Comments
This comment has been minimized.
This comment has been minimized.
To be able to run updated Go 1.13.3 and compile my project with it, I had to whitelist so far these binaries:
Whitelisting each takes the following:
|
This comment has been minimized.
This comment has been minimized.
Good times. /cc @golang/osp-team |
This comment has been minimized.
This comment has been minimized.
Related: #34748 (we need a Catalina builder) |
This comment has been minimized.
This comment has been minimized.
Discussion on Hacker News: https://news.ycombinator.com/item?id=21179970 Also, Catalina is reported to be a disastrous upgrade. |
This comment has been minimized.
This comment has been minimized.
I’ll take a look at this. |
This comment has been minimized.
This comment has been minimized.
I can reproduce on my personal MBP with macOS 10.15 (19A602) when installing from the .pkg installer too, however it's a different (less intrusive) experience. The "macOS cannot verify that this app is free from malware" prompt appears when first running the installer. After accepting and installing, Go binaries such as |
This comment has been minimized.
This comment has been minimized.
In the process of submitting the installer to Apple's notary service, but the process is very slow due to a service issue on Apple's side and our internal tooling keeps timing out. Diagnosing this has been difficult. It may be that we will need to enable the Hardened Runtime which likely requires a newer version of Xcode that we're using to build releases and an audit of whatever entitlements are required for our binaries to run. @aclements can you take a look at the link above and let me know which you think we'll need (if any)? |
This comment has been minimized.
This comment has been minimized.
@andybons, I don't think we need any of the runtime exceptions from that page. The only one that we may even potentially want is "Disable Library Validation Entitlement", which would let users use the the |
This comment has been minimized.
This comment has been minimized.
The ( "go1.13.3.darwin-amd64.pkg can't be opened because Apple cannot check it for malicious software." Meanwhile, the older Go version I had installed from before the OS X Catalina update, continues to work fine, no warning dialogs. |
This comment has been minimized.
This comment has been minimized.
@vp2177 it’s the same issue. |
This comment has been minimized.
This comment has been minimized.
Good news: according to Apple, we don’t have to enable the hardened runtime until January 2020. That said, we’ll likely have to start signing the binaries themselves (not just the installer as we do now), so this issue should stay open even if we fix the problem in the short term via notarization of the installer. |
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
This comment has been minimized.
@andybons Yes.
|
This comment has been minimized.
This comment has been minimized.
@andybons yes, it's enabled:
|
This comment has been minimized.
This comment has been minimized.
Our installer is getting stuck in limbo with the Notarization service. We’ve contacted Apple and are awaiting a response. It’s unclear what the ETA on that will be, though. |
This comment has been minimized.
This comment has been minimized.
https://developer.apple.com/system-status/ now shows an issue with the Notary service:
|
This comment has been minimized.
This comment has been minimized.
Notarization of the installer package has completed successfully. We are testing locally to ensure no scary popups are shown. |
This comment has been minimized.
This comment has been minimized.
While we believe we have the installer package issue resolved, we have to deal with the |
This comment has been minimized.
This comment has been minimized.
As an update, this is what we are going to do:
Since the notarization process is recursive (and the bits being distributed in the I will update this thread once this work is complete. |
This comment has been minimized.
This comment has been minimized.
The initial work mentioned in #34986 (comment) is now complete. We will likely issue a point release this week with the changes after some further testing. |
This comment has been minimized.
This comment has been minimized.
Below is derived from the log returned by Apple’s notarization service. I expect any warning to turn into an error come January 2020: Warnings we intend to fix by enabling the Hardened Runtime on the listed binaries:
Warnings we’ll have to fix another way:
De-duping from the above, the following binaries need to be dealt with (through obfuscation or some other means):
|
This comment has been minimized.
This comment has been minimized.
Is it saying the binaries need any particular hardened runtime entitlements, or just that all binaries need to be signed as having a hardened runtime? |
This comment has been minimized.
This comment has been minimized.
@aclements just that the binaries must have the hardened runtime enabled. It doesn’t (and likely won‘t/can’t?) say what entitlements a binary needs. |
This comment has been minimized.
This comment has been minimized.
Change https://golang.org/cl/208118 mentions this issue: |
Starting with macOS 10.15 (Catalina), Apple now requires all software distributed outside of the App Store to be notarized. Any binaries we distribute must abide by a strict set of requirements like code-signing and having a minimum target SDK of 10.9 (amongst others). Apple’s notarization service will recursively inspect archives looking to find notarization candidate binaries. If it finds a binary that does not meet the requirements or is unable to decompress an archive, it will reject the entire distribution. From cursory testing, it seems that the service uses content sniffing to determine file types, so changing the file extension will not work. There are some binaries and archives included in our distribution that are being detected by Apple’s service as potential candidates for notarization or decompression. As these are files used by tests and some are intentionally invalid, we don’t intend to ever make them compliant. As a workaround for this, we base64-encode any binaries or archives that Apple’s notarization service issues a warning for, as these warnings will become errors in January 2020. Updates #34986 Change-Id: I106fbb6227b61eb221755568f047ee11103c1680 Reviewed-on: https://go-review.googlesource.com/c/go/+/208118 Run-TryBot: Andrew Bonventre <andybons@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
This comment has been minimized.
This comment has been minimized.
Change https://golang.org/cl/208219 mentions this issue: |
This comment has been minimized.
This comment has been minimized.
Change https://golang.org/cl/208220 mentions this issue: |
This comment has been minimized.
This comment has been minimized.
@gopherbot please open a backport for 1.12 and 1.13 since otherwise Apple will reject future point releases of those versions |
This comment has been minimized.
This comment has been minimized.
Backport issue(s) opened: #35747 (for 1.12), #35748 (for 1.13). Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases. |
1 similar comment
This comment has been minimized.
This comment has been minimized.
Backport issue(s) opened: #35747 (for 1.12), #35748 (for 1.13). Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases. |
This comment has been minimized.
This comment has been minimized.
I wonder why tests and test data need to be shipped in releases. People interested in that can always retrieve it from the source repository. Has that been considered? |
This comment has been minimized.
This comment has been minimized.
@nightlyone, it has, several times. The main reason we continue to include them is that when a user reports a problem, it's nice to be able to ask them "What does |
…pple notarization to fail Starting with macOS 10.15 (Catalina), Apple now requires all software distributed outside of the App Store to be notarized. Any binaries we distribute must abide by a strict set of requirements like code-signing and having a minimum target SDK of 10.9 (amongst others). Apple’s notarization service will recursively inspect archives looking to find notarization candidate binaries. If it finds a binary that does not meet the requirements or is unable to decompress an archive, it will reject the entire distribution. From cursory testing, it seems that the service uses content sniffing to determine file types, so changing the file extension will not work. There are some binaries and archives included in our distribution that are being detected by Apple’s service as potential candidates for notarization or decompression. As these are files used by tests and some are intentionally invalid, we don’t intend to ever make them compliant. As a workaround for this, we base64-encode any binaries or archives that Apple’s notarization service issues a warning for, as these warnings will become errors in January 2020. Updates #34986 Fixes #35748 Change-Id: I106fbb6227b61eb221755568f047ee11103c1680 Reviewed-on: https://go-review.googlesource.com/c/go/+/208118 Run-TryBot: Andrew Bonventre <andybons@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> (cherry picked from commit 8bbfc51) Reviewed-on: https://go-review.googlesource.com/c/go/+/208219 Reviewed-by: Alexander Rakoczy <alex@golang.org>
…pple notarization to fail Starting with macOS 10.15 (Catalina), Apple now requires all software distributed outside of the App Store to be notarized. Any binaries we distribute must abide by a strict set of requirements like code-signing and having a minimum target SDK of 10.9 (amongst others). Apple’s notarization service will recursively inspect archives looking to find notarization candidate binaries. If it finds a binary that does not meet the requirements or is unable to decompress an archive, it will reject the entire distribution. From cursory testing, it seems that the service uses content sniffing to determine file types, so changing the file extension will not work. There are some binaries and archives included in our distribution that are being detected by Apple’s service as potential candidates for notarization or decompression. As these are files used by tests and some are intentionally invalid, we don’t intend to ever make them compliant. As a workaround for this, we base64-encode any binaries or archives that Apple’s notarization service issues a warning for, as these warnings will become errors in January 2020. Updates #34986 Updates #35747 Change-Id: I106fbb6227b61eb221755568f047ee11103c1680 Reviewed-on: https://go-review.googlesource.com/c/go/+/208118 Run-TryBot: Andrew Bonventre <andybons@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org> (cherry picked from commit 8bbfc51) Reviewed-on: https://go-review.googlesource.com/c/go/+/208220 Reviewed-by: Alexander Rakoczy <alex@golang.org>
This comment has been minimized.
This comment has been minimized.
It also seems desirable for |
This comment has been minimized.
This comment has been minimized.
Change https://golang.org/cl/208227 mentions this issue: |
This comment has been minimized.
This comment has been minimized.
Change https://golang.org/cl/208266 mentions this issue: |
+ The .pkg installer is now constructed by an internal process, since the binaries within the installer need to be signed using internal-only certs + 1.11 is no longer supported, so the tour is no longer shipped with the release Updates golang/go#34986 Change-Id: Ic05198dec2fdbfb26d9011944051a97c777e3898 Reviewed-on: https://go-review.googlesource.com/c/build/+/208266 Run-TryBot: Andrew Bonventre <andybons@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Binary files included in testdata directories can cause Apple’s notarization service to reject us since they don’t abide by their strict requirements. To emulate go mod vendor, remove all _test.go and testdata files from the vendor directory and update the instructions. Updates #34986 Fixes #35747 Change-Id: I5cde905fc78838d2e3b1519dab4aeee13d8d5356 Reviewed-on: https://go-review.googlesource.com/c/go/+/208227 Run-TryBot: Andrew Bonventre <andybons@golang.org> TryBot-Result: Gobot Gobot <gobot@golang.org> Reviewed-by: Alexander Rakoczy <alex@golang.org>
This comment has been minimized.
This comment has been minimized.
I've run an altered We plan to enable the hardened runtime for the 1.14 beta (scheduled for early December) and see if anything breaks. If everything looks good, we'll enable it for all releases moving forward. |
What version of Go are you using (
go version
)?Does this issue reproduce with the latest release?
Yes
What operating system and processor architecture are you using (
go env
)?go env
OutputWhat did you do?
Installed go from https://dl.google.com/go/go1.13.3.darwin-amd64.tar.gz and then tried to use it to build project. This is on macOS Catalina 10.15:
What did you expect to see?
Go works as usual.
What did you see instead?
On first
go
call I'm presented with OS pop-up that "macOS cannot verify that this app is free from malware", with options either to discard this window (the process is killed then) or move the binary to trash. After whitelisting this binary via Preferences → Security & Privacy → General → Allow Anyway,go
can be called fine.Next I tried to build project with
go install
and got the came pop-up for "compile" tool.