Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/mobile: gomobile build fails for iOS targets #32963

Open
dlo opened this issue Jul 6, 2019 · 34 comments

Comments

@dlo
Copy link

@dlo dlo commented Jul 6, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12 darwin/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

macOS Mojave 10.14.5 (18F132)

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/dan/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/dan/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/pv/rmyyshm95r78n31p86nnbznw0000gn/T/go-build908091193=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

$ go get golang.org/x/mobile/cmd/gomobile
$ go get -d golang.org/x/mobile/example/basic
$ gomobile build -bundleid basic.app -target=ios golang.org/x/mobile/example/basic

Note: the wiki instructions leave out the -bundleid parameter even though it appears to be required.

Output
gomobile: go build -tags ios -ldflags=-w -o=/var/folders/pv/rmyyshm95r78n31p86nnbznw0000gn/T/gomobile-work-749904680/arm golang.org/x/mobile/example/basic failed: exit status 2
# golang.org/x/mobile/app
darwin_ios.m:139:77: warning: null passed to a callee that requires a non-null argument [-Wnonnull]
# golang.org/x/mobile/example/basic
/usr/local/Cellar/go/1.12/libexec/pkg/tool/darwin_amd64/link: running /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang failed: exit status 1
ld: -headerpad and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together
clang: error: linker command failed with exit code 1 (use -v to see invocation

What did you expect to see?

See https://github.com/golang/go/wiki/Mobile#building-and-deploying-to-ios.

The build command will build an application bundle, named basic.app.

What did you see instead?

gomobile: go build -tags ios -ldflags=-w -o=/var/folders/pv/rmyyshm95r78n31p86nnbznw0000gn/T/gomobile-work-749904680/arm golang.org/x/mobile/example/basic failed: exit status 2
# golang.org/x/mobile/app
darwin_ios.m:139:77: warning: null passed to a callee that requires a non-null argument [-Wnonnull]
# golang.org/x/mobile/example/basic
/usr/local/Cellar/go/1.12/libexec/pkg/tool/darwin_amd64/link: running /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang failed: exit status 1
ld: -headerpad and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together
clang: error: linker command failed with exit code 1 (use -v to see invocation
@bcmills bcmills changed the title gomobile build fails for iOS targets x/mobile: gomobile build fails for iOS targets Jul 8, 2019
@gopherbot gopherbot added this to the Unreleased milestone Jul 8, 2019
@gopherbot gopherbot added the mobile label Jul 8, 2019
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jul 8, 2019

@dlo

This comment has been minimized.

Copy link
Author

@dlo dlo commented Jul 23, 2019

@bcmills Should I go ahead and see if I can figure out what's going on here? Happy to chip in.

@andydotxyz

This comment has been minimized.

Copy link

@andydotxyz andydotxyz commented Jul 28, 2019

I have the same problem on Mojave with XCode 10.2 and 10.3

@andydotxyz

This comment has been minimized.

Copy link

@andydotxyz andydotxyz commented Jul 28, 2019

The warning can be fixed with this patch, but it does not fix the build error...

index 08bb72f..e17b150 100644
--- a/vendor/golang.org/x/mobile/app/darwin_ios.m
+++ b/vendor/golang.org/x/mobile/app/darwin_ios.m
@@ -135,8 +135,9 @@ - (void)touchesCanceled:(NSSet*)touches withEvent:(UIEvent*)event {
 @end
 
 void runApp(void) {
+       char * argv[] = {};
        @autoreleasepool {
-               UIApplicationMain(0, nil, nil, NSStringFromClass([GoAppAppDelegate class]));
+               UIApplicationMain(0, argv, nil, NSStringFromClass([GoAppAppDelegate class]));
        }
 }
@andydotxyz

This comment has been minimized.

Copy link

@andydotxyz andydotxyz commented Jul 28, 2019

Aha, the problem seems to be that the go tools we are using force -headerpad and that bitcode is now enabled by default and these conflict. The following patch to gomobile fixed it for me 😎.

diff --git a/cmd/gomobile/build_iosapp.go b/cmd/gomobile/build_iosapp.go
index 9cc22ab..79b2c3f 100644
--- a/cmd/gomobile/build_iosapp.go
+++ b/cmd/gomobile/build_iosapp.go
@@ -426,6 +426,7 @@ const projPbxproj = `// !$*UTF8*$!
         SDKROOT = iphoneos;
         TARGETED_DEVICE_FAMILY = "1,2";
         VALIDATE_PRODUCT = YES;
+        ENABLE_BITCODE = NO;
       };
       name = Release;
     };
diff --git a/cmd/gomobile/env.go b/cmd/gomobile/env.go
index cb42d1c..a707965 100644
--- a/cmd/gomobile/env.go
+++ b/cmd/gomobile/env.go
@@ -140,7 +140,6 @@ func envInit() (err error) {
                default:
                        panic(fmt.Errorf("unknown GOARCH: %q", arch))
                }
-               cflags += " -fembed-bitcode"
                if err != nil {
                        return err
                }
@andydotxyz

This comment has been minimized.

Copy link

@andydotxyz andydotxyz commented Jul 28, 2019

I don't know how to create a PR for gomobile, so here is a consolidated patch file instead...

gomobile.diff.txt

@rob-deutsch

This comment has been minimized.

Copy link

@rob-deutsch rob-deutsch commented Sep 4, 2019

I just ran into the exact same problem. @andydotxyz solution fixed it!

@andydotxyz why did you decide to turn off bitcode instead of turning off headerpad?

I ask because I understand that Apple seems to prefer that bitcode is turned on.

@andydotxyz

This comment has been minimized.

Copy link

@andydotxyz andydotxyz commented Sep 4, 2019

That is a great question @rob-deutsch - it was not my preferred approach either. Unfortunately headerpad is coded into the Go tools which gomobile delegates to (as far as I can tell) - this means that the patch would have to go upstream to the main project which seemed much harder!

@rob-deutsch

This comment has been minimized.

Copy link

@rob-deutsch rob-deutsch commented Sep 5, 2019

Ha, that's a great reason! Thanks for the info @andydotxyz .

I know its unlikely, but would you happen to remember where exactly headerpad is hardcoded? I'm happy to take this upstream.

@hyangah

This comment has been minimized.

Copy link
Contributor

@hyangah hyangah commented Sep 19, 2019

@rob-deutsch I see in ld and some hints in macho_combine_dwarf.go

@andydotxyz

This comment has been minimized.

Copy link

@andydotxyz andydotxyz commented Sep 30, 2019

Sorry I don't recall @rob-deutsch - I just used grep for "headerpad" I think...

@hyangah

This comment has been minimized.

Copy link
Contributor

@hyangah hyangah commented Oct 2, 2019

For people who didn't follow the cl -
@eliasnaur commented in the cl.

This CL effectively reverts https://golang.org/cl/168062, potentially re-opening https://golang.org/issue/22395. What changed?

Does anyone have an answer?

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Oct 2, 2019

@hyangah, which CL? (I don't see one linked here — it's possible that GopherBot missed a cross-reference.)

@eliasnaur

This comment has been minimized.

Copy link
Contributor

@eliasnaur eliasnaur commented Oct 7, 2019

@eliasnaur

This comment has been minimized.

Copy link
Contributor

@eliasnaur eliasnaur commented Oct 7, 2019

I did some digging and found that the answer to my own question is: nothing changed, but gomobile build mode broke with my bitcode-enabling CL168062. Presumably I tested gomobile bind but failed to test gomobile build when preparing that CL.

gomobile build builds an iOS executable, while gomobile bind builds a C archive (using buildmode=c-archive). Building an iOS executable with -fembed-bitcode is not compatible with the flags cmd/link pass to the host linker; I've found -headerpad, -pagezero_size and -fno_pie to be incompatible.

It seems to me the fix is to figure out why cmd/link needs to pass the incompatible flags, and if it doesn't, remove or replace them with bitcode compatible flags. Removing the incomptable flags succeeded in producing a binary but I haven't run it nor tried to upload it to the App Store for verification.

The next best fix is to work around the issue and switch gomobile build to use the buildmode=c-archive like gomobile bind. As a side effect, gomobile build will be much more similar to gomobile bind, decreasing the likelyhood of future breaks.

The easiest workaround is for gomobile build to omit -fembed-bitcode. The downside is that executables built this way will not have bitcode enabled. Since bitcode is not (yet) a requirement for iOS, I don't think missing bitcode matters.

CL 189857 implements the third option, except that it also omits the flag for gomobile bind which is wrong.

@eliasnaur

This comment has been minimized.

Copy link
Contributor

@eliasnaur eliasnaur commented Oct 8, 2019

I asked on golang-dev for the purpose of the incompatible flags:

https://groups.google.com/forum/#!topic/golang-dev/U1jK3xmmGAk

It seems possible that my first suggestion could work: remove -headerpad, -fno_pie, -pagezero_size and see whether gomobile build programs still work on iOS, and whether the App Store will accept them.

I won't have much time before GoLab Florence, and we're approaching the freeze. If someone else could do the experiment, I'd be very grateful.

@eliasnaur

This comment has been minimized.

Copy link
Contributor

@eliasnaur eliasnaur commented Oct 16, 2019

With #34501 (comment) we can now target iOS builders with the TRY= directive.

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Oct 16, 2019

Change https://golang.org/cl/201358 mentions this issue: cmd/link/internal/ld: remove flags incompatible with -fembed-bitcode

@eliasnaur

This comment has been minimized.

Copy link
Contributor

@eliasnaur eliasnaur commented Oct 16, 2019

https://golang.org/cl/201358 seems to complete all.bash (on at least darwin/arm64). Can someone using gomobile build check whether App Store still accepts binaries with bitcode and CL 201358 applied? Thanks.
@hajimehoshi @andydotxyz @rob-deutsch @dlo

@andydotxyz

This comment has been minimized.

Copy link

@andydotxyz andydotxyz commented Oct 26, 2019

Testing on the latest macOS with gomobile build I get a new error:

ld: -no_pie and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together
clang: error: linker command failed with exit code 1 (use -v to see invocation)
@eliasnaur

This comment has been minimized.

Copy link
Contributor

@eliasnaur eliasnaur commented Oct 26, 2019

@andydotxyz

This comment has been minimized.

Copy link

@andydotxyz andydotxyz commented Oct 26, 2019

Ah yes, apologies.
It compiles and runs on device - I will attempt an upload to app store shortly.
First I need to figure how to manage different certification profiles using gomobile ;)

@andydotxyz

This comment has been minimized.

Copy link

@andydotxyz andydotxyz commented Oct 28, 2019

OK, I have attempted the upload and it seems to have no bitcode related errors :)

@eliasnaur

This comment has been minimized.

Copy link
Contributor

@eliasnaur eliasnaur commented Oct 28, 2019

gopherbot pushed a commit that referenced this issue Oct 30, 2019
The flags -headerpad, -Wl,-no_pie and -pagezero_size are incompatible with
the -fembed-bitcode flag used by `gomobile build`. Than McIntosh
suggested we might not need the offending flags; this change removes
the flags on darwin/arm64 and -headerpad, -pagezero_size on darwin/arm.

The -Wl,-no_pie flag is left for darwin/arm because linking fails
without it:

ld: warning: PIE disabled. Absolute addressing (perhaps -mdynamic-no-pic) not allowed in code signed PIE, but used in _runtime.rodata from /var/folders/qq/qxn86k813bn9fjxydm095rxw0000gp/T/workdir-host-darwin-amd64-zenly-ios/tmp/go-link-225285265/go.o. To fix this warning, don't compile with -mdynamic-no-pic or link with -Wl,-no_pie

Discussion: https://groups.google.com/d/msg/golang-dev/U1jK3xmmGAk/j0_ty46EDAAJ

I've verified the CL on the builders, built the "flappy" example from
gomobile with `gomobile build`, and verified that flappy runs on an
iPhone 5S.

Updates #32963

Change-Id: I783abc93ccf3c1d2b7ca00144b7164ba223d3529
Reviewed-on: https://go-review.googlesource.com/c/go/+/201358
Reviewed-by: Cherry Zhang <cherryyz@google.com>
@eliasnaur

This comment has been minimized.

Copy link
Contributor

@eliasnaur eliasnaur commented Nov 4, 2019

@andydotxyz could you be persuaded to check out Cherry's https://golang.org/cl/205060? It should fix the darwin/arm build as well.

@andydotxyz

This comment has been minimized.

Copy link

@andydotxyz andydotxyz commented Nov 4, 2019

I would be happy to but am only at a Mac on Wednesday this week - I’ll do my best to get it checked then :)

gopherbot pushed a commit that referenced this issue Nov 4, 2019
We used to pass -no_pie to external linker on darwin/arm, which
is incompatible with -fembed-bitcode. CL 201358 attempted to
remove the -no_pie flag, but it resulted the darwin linker to
complain about absolute addressing in TEXT segment.

On darwin/arm, we already get away from absolute addressing in
the TEXT section. The complained absolute addressing is in
RODATA, which was embedded in the TEXT segment. This CL moves
RODATA to the DATA segment, like what we already did on ARM64
and on AMD64 in c-archive/c-shared buildmodes for the same reason.
So there is no absolute addressing in the TEXT segment, which
allows us to remove -no_pie flag.

Fixes #35252.
Updates #32963.

Change-Id: Id6e3a594cb066d257d4f58fadb4a3ee4672529f7
Reviewed-on: https://go-review.googlesource.com/c/go/+/205060
Reviewed-by: Elias Naur <mail@eliasnaur.com>
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Nov 4, 2019

Change https://golang.org/cl/205060 mentions this issue: cmd/link: enable PIE on darwin/arm

@andydotxyz

This comment has been minimized.

Copy link

@andydotxyz andydotxyz commented Nov 6, 2019

With both patches applied I get:

/Users/andy/Code/Go/goroot/pkg/tool/darwin_amd64/link: running /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin/clang failed: exit status 1
ld: warning: OS version (7.0.0) too small, changing to 8.0.0
ld: -headerpad and -bitcode_bundle (Xcode setting ENABLE_BITCODE=YES) cannot be used together
clang: error: linker command failed with exit code 1 (use -v to see invocation)
@eliasnaur

This comment has been minimized.

Copy link
Contributor

@eliasnaur eliasnaur commented Nov 6, 2019

Both patches are in go tip, where headerpad is guarded by

      if !ctxt.Arch.InFamily(sys.ARM, sys.ARM64) {
          // -headerpad is incompatible with -fembed-bitcode.
          argv = append(argv, "-Wl,-headerpad,1144")
      }

so -headerpad shouldn't be emitted on darwin/arm nor darwin/arm64. Can you try again with Go tip?

@andydotxyz

This comment has been minimized.

Copy link

@andydotxyz andydotxyz commented Nov 6, 2019

arm and arm64 are both working, however the "ios" target is not - so it's still not building for simulator.

works:

  • gomobile build -target ios/arm
  • gomobile build -target ios/arm64

does not:

  • gomobile build -target ios
@eliasnaur

This comment has been minimized.

Copy link
Contributor

@eliasnaur eliasnaur commented Nov 6, 2019

Thank you, I had forgotten about the simulators. Let me know whether golang.org/cl/205340 works, which fixes the problem for:

$ gomobile build -target ios -bundleid <...> golang.org/x/mobile/example/flappy
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Nov 6, 2019

Change https://golang.org/cl/205340 mentions this issue: cmd/link/internal/ld: omit bitcode-incompatible flags on iOS simulator

@andydotxyz

This comment has been minimized.

Copy link

@andydotxyz andydotxyz commented Nov 6, 2019

Excellent, thanks - that fixes "-target ios"

$ file myapp.app/main
Mach-O universal binary with 4 architectures: [i386:Mach-O executable i386] [x86_64] [arm_v7] [arm64]
(for architecture i386):	Mach-O executable i386
(for architecture x86_64):	Mach-O 64-bit executable x86_64
(for architecture armv7):	Mach-O executable arm_v7
(for architecture arm64):	Mach-O 64-bit executable arm64
gopherbot pushed a commit that referenced this issue Nov 6, 2019
The -Wl,-headerpad, -Wl,-no_pie, -Wl,-pagezero_size flags are
incompatible with the bitcode-related flags used for iOS.

We already omitted the flags on darwin/arm and darwin/arm64; this change
omits the flags on all platforms != macOS so that building for the iOS
simulator works.

Updates #32963

Change-Id: Ic9af0daf01608f5ae0f70858e3045e399de7e95b
Reviewed-on: https://go-review.googlesource.com/c/go/+/205340
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Nov 10, 2019

Change https://golang.org/cl/206337 mentions this issue: cmd/link/internal/ld: set MachO platform to PLATFORM_IOS on iOS

gopherbot pushed a commit that referenced this issue Nov 15, 2019
CL 205340 changed the linker to skip certain flags when linking
for iOS. However, for host linking on iOS (such as on the Corellium
darwin/arm64 builders) the MachO platform defaults to PLATFORM_MACOS,
and the flags are not skipped.

Avoids warnings such as

    ld: warning: -no_pie ignored for arm64

Updates #32963

Change-Id: Ib6b4c2375fd14cf89410bf5ff1537b692b7a1c15
Reviewed-on: https://go-review.googlesource.com/c/go/+/206337
Run-TryBot: Elias Naur <mail@eliasnaur.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Ian Lance Taylor <iant@golang.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
7 participants
You can’t perform that action at this time.