Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime: GOOS=ios fails Apple's app validation due to use of private API #58323

Closed
danderson opened this issue Feb 4, 2023 · 17 comments
Closed
Assignees
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge mobile Android, iOS, and x/mobile NeedsFix The path to resolution is known, but the work has not been done. OS-Darwin
Milestone

Comments

@danderson
Copy link
Contributor

What version of Go are you using (go version)?

$ go version
go version go1.20rc3-ts178d6bc darwin/amd64

This is Tailscale's Go toolchain from https://github.com/tailscale/go. It has some minor changes, but for the purposes of this bug it's identical to the final Go 1.20 release. You can see our very minor additional commits at https://github.com/tailscale/go/commits/tailscale.go1.20 , prefixed [tailscale1.20].

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/tsbuild/Library/Caches/go-build"
GOENV="/Users/tsbuild/Library/Application Support/go/env"
GOEXE=""
GOEXPERIMENT=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/tsbuild/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/tsbuild/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/tsbuild/.cache/tailscale-go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/tsbuild/.cache/tailscale-go/pkg/tool/darwin_amd64"
GOVCS=""
GOVERSION="go1.20rc3-ts178d6bc"
GCCGO="gccgo"
GOAMD64="v1"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/dev/null"
GOWORK=""
CGO_CFLAGS="-O2 -g"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-O2 -g"
CGO_FFLAGS="-O2 -g"
CGO_LDFLAGS="-O2 -g"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -arch x86_64 -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/h6/n1fmcdqs5z765682qxt8sr7w0000gn/T/go-build460345883=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Built a test release of Tailscale's iOS app, which unfortunately is not open-source. It's a mixed swift+Go application built through xcode.

What did you expect to see?

A working app that passes Apple's app store validation.

What did you see instead?

Apple rejected the app bundle, on the grounds that it failed "SPI validation". This is apparently the process where Apple checks for unauthorized use of private APIs by the app.

*** Error: ERROR: Asset validation failed (11) The app references non-public symbols in Payload/Tailscale.app/PlugIns/IPNExtension.appex/IPNExtension: _xpc_date_create_from_current (ID: e6cd0a76-b3c5-4ce4-8a35-600308487c8c)

AFAICT, this is due to 76d39ae , which added a call to xpc_date_create_from_current in an attempt to fix atfork problems on darwin.

The same app builds and validates successfully when targeting macOS, so it appears that xpc_date_create_from_current is marked private only on iOS, not macOS.

@gopherbot gopherbot added the compiler/runtime Issues related to the Go compiler and/or runtime. label Feb 4, 2023
@danderson
Copy link
Contributor Author

cc @rsc, who appears to have all the state on this unfortunate set of fork issues. Sorry to be the bearer of annoying news :(

bradfitz added a commit to tailscale/go that referenced this issue Feb 4, 2023
Updates tailscale/corp#9061
Updates golang#58323

Signed-off-by: Brad Fitzpatrick <brad@danga.com>
danderson pushed a commit to tailscale/go that referenced this issue Feb 4, 2023
Updates tailscale/corp#9061
Updates golang#58323

Signed-off-by: Brad Fitzpatrick <brad@danga.com>
@danderson
Copy link
Contributor Author

@bradfitz suggested just removing the osinit_hack on iOS as a test, on the principle that apparently fork+exec is forbidden in non-jailbroken iOS anyway.

Patch is at tailscale@d0f872e, which through dead code elimination removes the offending call from the binary. With this patch applied to the toolchain, Apple is once again happy with the Tailscale iOS app.

We haven't dug very deeply into whether disabling osinit_hack is safe, we're just going off internet claims that fork+exec is not permitted in the iOS runtime environment. If y'all have a direct line to Apple, it'd be good to get verification from them.

@bcmills bcmills added OS-Darwin NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. mobile Android, iOS, and x/mobile labels Feb 6, 2023
@bcmills bcmills added this to the Go1.21 milestone Feb 6, 2023
@bradfitz
Copy link
Contributor

bradfitz commented Feb 7, 2023

@bcmills, you targeted this at the Go1.21 milestone, but this is really a regression in Go 1.20.

@bcmills
Copy link
Contributor

bcmills commented Feb 7, 2023

Yes. Presumably it should be addressed at HEAD and backported.

@rsc
Copy link
Contributor

rsc commented Feb 8, 2023

Yes, this should be fixed at HEAD and backported. Simply skipping the calls on ios makes sense to me: if you can't call exec, you don't need to work around the bugs introduced by calling exec.

@rsc
Copy link
Contributor

rsc commented Feb 8, 2023

Ironically, an Apple engineer suggested using xpc_date_create_from_current specifically because it was a public function. But maybe that's only true on macOS.

@rsc
Copy link
Contributor

rsc commented Feb 8, 2023

@danderson can you please check whether https://go-review.googlesource.com/c/go/+/466516 also signs properly? It's the same as your patch except the if statement is in a darwin-specific file to try to keep the ios knowledge more localized. If that works too, we will commit it and backport the fix. Thanks!

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/466516 mentions this issue: runtime: skip darwin osinit_hack on ios

@rsc
Copy link
Contributor

rsc commented Feb 8, 2023

@gopherbot please backport go1.20

@gopherbot
Copy link
Contributor

Backport issue(s) opened: #58419 (for 1.20).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://go.dev/wiki/MinorReleases.

@mknyszek mknyszek self-assigned this Feb 8, 2023
@danderson
Copy link
Contributor Author

Sorry, missed this today. I can cut another toolchain and app release tomorrow to test your change.

@rsc
Copy link
Contributor

rsc commented Feb 9, 2023

Thanks!

@dmitshur dmitshur added NeedsFix The path to resolution is known, but the work has not been done. and removed NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. labels Feb 9, 2023
@heschi
Copy link
Contributor

heschi commented Feb 9, 2023

This is coming right down to the wire for 1.20.1. Is there a strong reason it can't wait for .2, which will probably be released on March 7?

bradfitz pushed a commit to tailscale/go that referenced this issue Feb 9, 2023
Darwin needs the osinit_hack call to fix some bugs in the Apple libc
that surface when Go programs call exec. On iOS, the functions that
osinit_hack uses are not available, so signing fails. But on iOS exec
is also unavailable, so the hack is not needed. Disable it there,
which makes signing work again.

Fixes golang#58323.

Change-Id: I3f1472f852bb36c06854fe1f14aa27ad450c5945
(cherry picked from commit e95f1f1cedaa8284bb00dec23da292ef3821cd2e)
bradfitz pushed a commit to tailscale/go that referenced this issue Feb 9, 2023
Darwin needs the osinit_hack call to fix some bugs in the Apple libc
that surface when Go programs call exec. On iOS, the functions that
osinit_hack uses are not available, so signing fails. But on iOS exec
is also unavailable, so the hack is not needed. Disable it there,
which makes signing work again.

Fixes golang#58323.

Change-Id: I3f1472f852bb36c06854fe1f14aa27ad450c5945
(cherry picked from commit e95f1f1cedaa8284bb00dec23da292ef3821cd2e)
@danderson
Copy link
Contributor Author

No strong reason for us (Tailscale) since we are carrying the patch in our toolchain fork. If this doesn't ship in .1, Go development on iOS will be outright broken until .2 ships, since nobody will be able to ship any app that incorporates any Go code.

@danderson
Copy link
Contributor Author

We patched https://go.dev/cl/466516 into our toolchain and built an unstable iOS app release. It worked fine, Apple was happy with the resulting binaries. LGTM!

@rsc
Copy link
Contributor

rsc commented Feb 10, 2023

Thank you!

@gopherbot
Copy link
Contributor

Change https://go.dev/cl/467316 mentions this issue: [release-branch.go1.20] runtime: skip darwin osinit_hack on ios

gopherbot pushed a commit that referenced this issue Feb 10, 2023
Darwin needs the osinit_hack call to fix some bugs in the Apple libc
that surface when Go programs call exec. On iOS, the functions that
osinit_hack uses are not available, so signing fails. But on iOS exec
is also unavailable, so the hack is not needed. Disable it there,
which makes signing work again.

Fixes #58323.
Fixes #58419.

Change-Id: I3f1472f852bb36c06854fe1f14aa27ad450c5945
Reviewed-on: https://go-review.googlesource.com/c/go/+/466516
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Dave Anderson <danderson@tailscale.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/467316
johanbrandhorst pushed a commit to Pryz/go that referenced this issue Feb 12, 2023
Darwin needs the osinit_hack call to fix some bugs in the Apple libc
that surface when Go programs call exec. On iOS, the functions that
osinit_hack uses are not available, so signing fails. But on iOS exec
is also unavailable, so the hack is not needed. Disable it there,
which makes signing work again.

Fixes golang#58323.

Change-Id: I3f1472f852bb36c06854fe1f14aa27ad450c5945
Reviewed-on: https://go-review.googlesource.com/c/go/+/466516
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Dave Anderson <danderson@tailscale.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
romaindoumenc pushed a commit to TroutSoftware/go that referenced this issue Feb 14, 2023
Darwin needs the osinit_hack call to fix some bugs in the Apple libc
that surface when Go programs call exec. On iOS, the functions that
osinit_hack uses are not available, so signing fails. But on iOS exec
is also unavailable, so the hack is not needed. Disable it there,
which makes signing work again.

Fixes golang#58323.
Fixes golang#58419.

Change-Id: I3f1472f852bb36c06854fe1f14aa27ad450c5945
Reviewed-on: https://go-review.googlesource.com/c/go/+/466516
Run-TryBot: Russ Cox <rsc@golang.org>
Reviewed-by: Dave Anderson <danderson@tailscale.com>
Reviewed-by: Michael Knyszek <mknyszek@google.com>
TryBot-Result: Gopher Robot <gobot@golang.org>
Auto-Submit: Russ Cox <rsc@golang.org>
Reviewed-by: Bryan Mills <bcmills@google.com>
Reviewed-by: Than McIntosh <thanm@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/467316
jyyi1 added a commit to Jigsaw-Code/outline-apps that referenced this issue Feb 27, 2023
…1584)

In this PR, I fixed the Android release script, and synced `BundleVersion` from Apple's App Store back to this repository. In addition, due to [a bug in golang's compiler for iOS](golang/go#58323), I rebuilt the `outline-go-tun2socks` binary and updated the checksum here.
@golang golang locked and limited conversation to collaborators Feb 10, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
compiler/runtime Issues related to the Go compiler and/or runtime. FrozenDueToAge mobile Android, iOS, and x/mobile NeedsFix The path to resolution is known, but the work has not been done. OS-Darwin
Projects
None yet
Development

No branches or pull requests

8 participants