Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/mobile: build failing when using go modules #27234

Closed
mirza-s opened this issue Aug 26, 2018 · 77 comments
Closed

x/mobile: build failing when using go modules #27234

mirza-s opened this issue Aug 26, 2018 · 77 comments

Comments

@mirza-s
Copy link

@mirza-s mirza-s commented Aug 26, 2018

What version of Go are you using (go version)?

go version go1.11 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

GOARCH="amd64"
GOBIN="/Users/mbp/go/bin"
GOCACHE="/Users/mbp/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/mbp/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/mbp/wrk/src/libfp/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/nt/0hzfppd92jvbk99fhvnhy8ph0000gn/T/go-build526348344=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

gomobile bind -target=android .

What did you expect to see?

Normal build as when building inside GOPATH

What did you see instead?

gomobile: go build -buildmode=c-shared -o=/var/folders/nt/0hzfppd92jvbk99fhvnhy8ph0000gn/T/gomobile-work-327189385/android/src/main/jniLibs/armeabi-v7a/libgojni.so gobind failed: exit status 1
go: finding github.com/dustin/go-humanize v0.0.0-20180713052910-9f541cc9db5d
go: finding golang.org/x/mobile v0.0.0-20180808221059-bceb7ef27cc6
can't load package: package gobind: unknown import path "gobind": cannot find module providing package gobind
@hajimehoshi
Copy link
Member

@hajimehoshi hajimehoshi commented Sep 10, 2018

Hi, what's going on this?

@wangyun
Copy link

@wangyun wangyun commented Sep 19, 2018

I have the same issue but with some different error message:

gomobile: /home/wangyun/dev/gopath/bin/gobind -lang=go,java -outdir=/tmp/gomobile-work-948078978 testaar failed: exit status 1
unable to import bind: cannot find package "golang.org/x/mobile/bind" in any of:
/home/wangyun/dev/software/go/src/golang.org/x/mobile/bind (from $GOROOT)
/home/wangyun/dev/gopath/src/golang.org/x/mobile/bind (from $GOPATH)
unable to import bind: cannot find package "golang.org/x/mobile/bind" in any of:
/home/wangyun/dev/software/go/src/golang.org/x/mobile/bind (from $GOROOT)
/home/wangyun/dev/gopath/src/golang.org/x/mobile/bind (from $GOPATH)
unable to import bind/java: cannot find package "golang.org/x/mobile/bind/java" in any of:
/home/wangyun/dev/software/go/src/golang.org/x/mobile/bind/java (from $GOROOT)
/home/wangyun/dev/gopath/src/golang.org/x/mobile/bind/java (from $GOPATH)

I'm using go 1.11 and updated gomobile, gobind to the latest version.

@evenardo
Copy link

@evenardo evenardo commented Oct 10, 2018

'''
env go111module=off gomobile bind
'''
it's will temporarily worked fine

@myitcv
Copy link
Member

@myitcv myitcv commented Nov 14, 2018

cc @eliasnaur - some time has elapsed since this issue and I'm not sufficiently up to speed to know whether mobile is now modules-aware.

@eliasnaur
Copy link
Contributor

@eliasnaur eliasnaur commented Nov 14, 2018

@glenfordwilliams
Copy link

@glenfordwilliams glenfordwilliams commented Nov 28, 2018

is there work currently being done on this front?

@eliasnaur
Copy link
Contributor

@eliasnaur eliasnaur commented Nov 28, 2018

Not yet, at least not from my side. Partly because of time available, partly because I prefer to wait for module support to mature (GO111MODULE always on, the go/packages package feature complete and so on), but the main stumbling block is that I can't see the gomobile bind mode work as usual in module mode, at least not in its current form.
The problem is vendoring: gomobile bind generates a main package that in turn imports the packages you want to export to Java or Objective C. That works well in GOPATH mode, but not in module mode. Suppose gomobile bind in module mode generates a module containing a main package and your module as a dependency. Then, because vendored packages are only respected for the main module, any vendored packages from your module will be ignored by the go tool.
Perhaps some form of #26640 will help gomobile. Perhaps gomobile bind should work in a different way. I haven't thought that much about it.

@eliasnaur
Copy link
Contributor

@eliasnaur eliasnaur commented Nov 28, 2018

I started a conversation about gomobile and modules here: https://groups.google.com/forum/#!topic/golang-dev/C0ip7pFC2Oc

One question I would like you to ponder on that thread is whether it would be acceptable to require gomobile users to first generate glue code with the gobind tool and then only let the gomobile tool do the final packaging.

@gopherbot
Copy link

@gopherbot gopherbot commented Mar 12, 2019

Change https://golang.org/cl/167180 mentions this issue: all: add go.mod

@gopherbot
Copy link

@gopherbot gopherbot commented Mar 12, 2019

Change https://golang.org/cl/167179 mentions this issue: all: set GO111MODULE=off explicitly in bind tests

gopherbot pushed a commit to golang/mobile that referenced this issue Mar 13, 2019
Updates golang/go#27234
Updates golang/go#30228

Change-Id: Ibe281480398ff9d8c27657f399159885a75f646e
Reviewed-on: https://go-review.googlesource.com/c/mobile/+/167179
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
gopherbot pushed a commit to golang/mobile that referenced this issue Mar 13, 2019
Commands run:
	go mod init
	go mod edit -go=1.11
	go mod tidy
	go list -m all
	go test ./...

Updates golang/go#30228
Updates golang/go#27234

Change-Id: Ibf3b104bfaa037e1a94a5327b5a56983d9744e73
Reviewed-on: https://go-review.googlesource.com/c/mobile/+/167180
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Run-TryBot: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
@gopherbot
Copy link

@gopherbot gopherbot commented Mar 16, 2019

Change https://golang.org/cl/167659 mentions this issue: cmd/gomobile: explicitly disable modules

gopherbot pushed a commit to golang/mobile that referenced this issue Mar 16, 2019
Gomobile doesn't support modules.

Updates golang/go#27234

Change-Id: I426db1600eafd4a9ddc7eb239f5f264756f1ba35
Reviewed-on: https://go-review.googlesource.com/c/mobile/+/167659
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
TryBot-Result: Gobot Gobot <gobot@golang.org>
@mehrvarz
Copy link

@mehrvarz mehrvarz commented Mar 17, 2019

Hi, I use modules heavily in my projects. Unfortunately, this makes using gomobile rather difficult. When do you expect gomobile to support modules? Any updates on this?

@hajimehoshi
Copy link
Member

@hajimehoshi hajimehoshi commented Mar 17, 2019

I was wondering whether packages that use major versions like /v2 are available with go mobile or not.

anacrolix added a commit to anacrolix/torrent that referenced this issue Mar 22, 2019
@chrisprobst
Copy link

@chrisprobst chrisprobst commented Apr 17, 2019

We also need this feature desperately!

@balupton
Copy link

@balupton balupton commented Apr 18, 2019

Are there any workarounds for the meantime?

@cryptix
Copy link

@cryptix cryptix commented Apr 18, 2019

@balupton / @chrisprobst: I use go mod vendor then switch GO111MODULE=off and build it as a regular vendored project.

@torbenschinke
Copy link
Contributor

@torbenschinke torbenschinke commented May 7, 2019

@balupton / @chrisprobst: We basically also use go mod vendor. However I've created a small tool (GoUp) which automates this process for us.

@hajimehoshi
Copy link
Member

@hajimehoshi hajimehoshi commented Jan 21, 2020

I think https://golang.org/cl/215421 will fix the issue. With this, I've confirmed replace directives worked.

gopherbot pushed a commit to golang/mobile that referenced this issue Jan 23, 2020
This CL gives CGO_ENABLED=1 explicitly when executing gobind since
Cgo is disabled by default when GOOS is given.

This CL also adds importing C to the tests to confirm that Cgo
works correctly.

This CL also updates go.mod since this change requries the change
in go/packages: golang.org/cl/214943

Updates golang/go#27234
Updates golang/go#36547

Change-Id: I66f9697f992f15b52fca7871e4e0ed64ca2b4965
Reviewed-on: https://go-review.googlesource.com/c/mobile/+/214498
Run-TryBot: Hajime Hoshi <hajimehoshi@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
gopherbot pushed a commit to golang/mobile that referenced this issue Jan 23, 2020
A replative path in go.mod works only in the directory of the
go.mod. When creating go.mod for gobind, copying relative paths
did not work since go.mod is in a different temporary directory.

This CL fixes the issue to use Dir, which is an absolute path to
the module, instead of Path.

Updates golang/go#27234

Change-Id: Ib009ec508aa3ce3d092af14f921a57192feaac61
Reviewed-on: https://go-review.googlesource.com/c/mobile/+/215421
Run-TryBot: Hajime Hoshi <hajimehoshi@gmail.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Hyang-Ah Hana Kim <hyangah@gmail.com>
@hajimehoshi
Copy link
Member

@hajimehoshi hajimehoshi commented Jan 23, 2020

Now all the problems I found have been fixed. Let's close this issue (I don't have the right to close this issue though), and file other issues when we find other problems.

@agnivade
Copy link
Contributor

@agnivade agnivade commented Jan 23, 2020

Great work @hajimehoshi !

@TTy32
Copy link

@TTy32 TTy32 commented Feb 17, 2020

Thanks a lot @hajimehoshi !

@themartorana
Copy link

@themartorana themartorana commented Feb 17, 2020

Forgive my lack of understanding - will this ship with a particular version of Go? I see it’s still part of an “Unreleased” milestone.

@mvdan
Copy link
Member

@mvdan mvdan commented Feb 17, 2020

@themartorana x/mobile is a repository and a module that you can use today. It's not shipped with Go releases, nor does it follow its release schedule. If a new version of gomobile is out, you can use it with Go 1.13.8, just like you can use it with 1.14rc1.

andydotxyz added a commit to fyne-io/fyne that referenced this issue Mar 5, 2020
This is a preparation to use Go modules at gomobile command.

Updates golang/go#27234
Cherry picked from github.com/golang/mobile
andydotxyz added a commit to fyne-io/fyne that referenced this issue Mar 5, 2020
This CL is a pure refactoring. This removes a global variable ctx,
which is a build.Default.

Before this change, ctx was used to keep build tags and its state
affected go command executions. As the variable is mutable, the
code was not readable.

This changes introduces another global variable buildTags instead,
but this is more consistent with other build flags, and this is
immutable.

Updates golang/go#27234

Conflict: cmd/gomobile/build.go
Cherry picked from github.com/golang/mobile
andydotxyz added a commit to fyne-io/fyne that referenced this issue Mar 5, 2020
Before this change, all the gomobile commands forced Go modules
to be off internally, regardless of the current Go modules state.

After this change, gomobile-init command follows the current Go
modules state. The other gomobile commands are not changed.

This is also a preparation to support Go modules in gomobile-bind
and gomobile-build.

Updates golang/go#27234
Cherry picked from github.com/golang/mobile
andydotxyz added a commit to fyne-io/fyne that referenced this issue Mar 5, 2020
This CL enables Go modules at gomobile-build command. This CL is
a counterpart for the change gomobile-bind: https://golang.org/cl/210380

Updates golang/go#27234
Cherry picked from github.com/golang/mobile
@karalabe
Copy link
Contributor

@karalabe karalabe commented Mar 16, 2020

@hajimehoshi Oh wow, so happy to see this closed. Buuut :D, I think I found a tiny case that could use some love. gomobile doesn't seem to cache binaries across builds. I have a dependency that takes 5-7 minutes to build for a single platform (https://github.com/ipsn/go-libtor), so for all 4, that's a whopping 30. That's acceptable if it gets reused for subsequent builds, but unfortunately currently it does not. Would it be possible to keep the build cache around?

@hajimehoshi
Copy link
Member

@hajimehoshi hajimehoshi commented Mar 16, 2020

Thanks.

gomobile doesn't seem to cache binaries across builds.

That's odd since gomobile just invokes go commands as usual and then module cache and build cache should be used.

I was wondering if my fix was the culprit of the issue you reported. Hasn't there been this issue before my fix?

@karalabe
Copy link
Contributor

@karalabe karalabe commented Mar 16, 2020

That's odd since gomobile just invokes go commands as usual and then module cache and build cache should be used.

Isn't gomobile using custom build and cache dirs? The packages are definitely downloaded into the tmp folder under gomobile-work-xxxxxxxx, new for every build.

Hasn't there been this issue before my fix?

I can't say, it's been a while since I picked up gomobile. Started doing it today. If you're looking for a painful repro, here's the code I just pushed https://github.com/coronanet/go-coronanet . Though it would possibly make sense to use something smaller to test than this monstrosity.

@hajimehoshi
Copy link
Member

@hajimehoshi hajimehoshi commented Mar 17, 2020

Isn't gomobile using custom build and cache dirs? The packages are definitely downloaded into the tmp folder under gomobile-work-xxxxxxxx, new for every build.

Aha, gobind, which is invoked by gomobile, generates a source code from Go packages for binding, and they are not cached indeed. Did you mean 'cache' of them?

@hajimehoshi
Copy link
Member

@hajimehoshi hajimehoshi commented Mar 17, 2020

I think the cache issue is not related to this issue #27234, so I'd be happy if you could file a new issue.

@karalabe
Copy link
Contributor

@karalabe karalabe commented Mar 17, 2020

I think the cache issue is not related to this issue #27234, so I'd be happy if you could file a new issue.

@hajimehoshi Done #37902

andydotxyz added a commit to fyne-io/fyne that referenced this issue Apr 5, 2020
This is a preparation to use Go modules at gomobile command.

Updates golang/go#27234
Cherry picked from github.com/golang/mobile
andydotxyz added a commit to fyne-io/fyne that referenced this issue Apr 5, 2020
This CL is a pure refactoring. This removes a global variable ctx,
which is a build.Default.

Before this change, ctx was used to keep build tags and its state
affected go command executions. As the variable is mutable, the
code was not readable.

This changes introduces another global variable buildTags instead,
but this is more consistent with other build flags, and this is
immutable.

Updates golang/go#27234

Conflict: cmd/gomobile/build.go
Cherry picked from github.com/golang/mobile
andydotxyz added a commit to fyne-io/fyne that referenced this issue Apr 5, 2020
Before this change, all the gomobile commands forced Go modules
to be off internally, regardless of the current Go modules state.

After this change, gomobile-init command follows the current Go
modules state. The other gomobile commands are not changed.

This is also a preparation to support Go modules in gomobile-bind
and gomobile-build.

Updates golang/go#27234
Cherry picked from github.com/golang/mobile
andydotxyz added a commit to fyne-io/fyne that referenced this issue Apr 5, 2020
This CL enables Go modules at gomobile-build command. This CL is
a counterpart for the change gomobile-bind: https://golang.org/cl/210380

Updates golang/go#27234
Cherry picked from github.com/golang/mobile
@marceloneil marceloneil mentioned this issue May 6, 2020
36 of 40 tasks complete
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
You can’t perform that action at this time.