Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

x/mobile: gobind type-checking fails with v2+ modules #33272

Open
triztian opened this issue Jul 25, 2019 · 7 comments
Open

x/mobile: gobind type-checking fails with v2+ modules #33272

triztian opened this issue Jul 25, 2019 · 7 comments

Comments

@triztian
Copy link

@triztian triztian commented Jul 25, 2019

What version of Go are you using (go version)?

go version go1.12.7 darwin/amd64

Does this issue reproduce with the latest release?

Yes, it also happens on 1.12.6

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN="/Users/Tristian/go/bin"
GOCACHE="/Users/Tristian/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/Tristian/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12.7/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12.7/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/7s/cnfg6ksj44gb3br_lc4v3gn40000gp/T/go-build089542378=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

The setup consists of 2 packages a gomobile-v2target (mobile) which imports a package that is
v2 suffixed, and a gomobile-v2mod (v2mod) package that has a sub package that is imported in the root package and that is declared as v2.

The project that produces the framework can be located here:

The project that is v2 and is imported by the previous one can be found here:

Both projects exist under the $GOPATH and I've verified that they build successfully without using gomobile and with GO111MODULE=off:

Framework producer Build Output
 λ gomobile-v2target ~> go build -a -v ./...
errors
math/bits
runtime/internal/sys
runtime/internal/atomic
internal/cpu
unicode/utf8
runtime/internal/math
internal/race
sync/atomic
unicode
internal/bytealg
math
internal/testlog
github.com/triztian/gomobile-v2mod/subpkg
github.com/triztian/gomobile-v2mod
runtime
strconv
sync
io
reflect
syscall
internal/syscall/unix
time
internal/poll
os
sort
internal/fmtsort
fmt
github.com/triztian/gomobile-v2target

v2 module Build Output
 λ gomobile-v2mod ~> go build -v -a ./...
github.com/triztian/gomobile-v2mod/subpkg
github.com/triztian/gomobile-v2mod

When trying to generate the framework with the following command:

$ gomobile bind -x -a -v -target ios .

We get the following output:

GOMOBILE=/Users/Tristian/go/pkg/gomobile
WORK=/var/folders/7s/cnfg6ksj44gb3br_lc4v3gn40000gp/T/gomobile-work-104674381
GOOS=darwin CGO_ENABLED=1 $GOPATH/bin/gobind -lang=go,objc -outdir=$WORK -tags=ios github.com/triztian/gomobile-v2target
type-checking package "github.com/triztian/gomobile-v2target" failed (/Users/Tristian/go/src/github.com/triztian/gomobile-v2target/export.go:6:8: could not import github.com/triztian/gomobile-v2mod (type-checking package "github.com/triztian/gomobile-v2mod" failed (/Users/Tristian/go/src/github.com/triztian/gomobile-v2mod/math.go:3:8: could not import github.com/triztian/gomobile-v2mod/v2/subpkg (cannot find package "github.com/triztian/gomobile-v2mod/v2/subpkg" in any of:
	/usr/local/Cellar/go/1.12.7/libexec/src/github.com/triztian/gomobile-v2mod/v2/subpkg (from $GOROOT)
	/Users/Tristian/go/src/github.com/triztian/gomobile-v2mod/v2/subpkg (from $GOPATH)))))

rm -r -f "$WORK"
gomobile: /Users/Tristian/go/bin/gobind -lang=go,objc -outdir=/var/folders/7s/cnfg6ksj44gb3br_lc4v3gn40000gp/T/gomobile-work-104674381 -tags=ios github.com/triztian/gomobile-v2target failed: exit status 1

if it helps, this is the structure of my GOPATH:

λ triztian ~> pwd && tree gomobile-v2*
/Users/Tristian/go/src/github.com/triztian
gomobile-v2mod
├── LICENSE
├── README.md
├── go.mod
├── math.go
└── subpkg
    └── sub.go
gomobile-v2target
├── README.md
├── export.go
└── go.mod

1 directory, 8 files

What did you expect to see?

A Mobile.framework bundle.

What did you see instead?

The following error message:

λ gomobile-v2target ~> gomobile bind -x -a -v -target ios .
GOMOBILE=/Users/Tristian/go/pkg/gomobile
WORK=/var/folders/7s/cnfg6ksj44gb3br_lc4v3gn40000gp/T/gomobile-work-431765436
GOOS=darwin CGO_ENABLED=1 $GOPATH/bin/gobind -lang=go,objc -outdir=$WORK -tags=ios github.com/triztian/gomobile-v2target
type-checking package "github.com/triztian/gomobile-v2target" failed (/Users/Tristian/go/src/github.com/triztian/gomobile-v2target/export.go:6:8: could not import github.com/triztian/gomobile-v2mod (type-checking package "github.com/triztian/gomobile-v2mod" failed (/Users/Tristian/go/src/github.com/triztian/gomobile-v2mod/math.go:3:8: could not import github.com/triztian/gomobile-v2mod/v2/subpkg (cannot find package "github.com/triztian/gomobile-v2mod/v2/subpkg" in any of:
	/usr/local/Cellar/go/1.12.7/libexec/src/github.com/triztian/gomobile-v2mod/v2/subpkg (from $GOROOT)
	/Users/Tristian/go/src/github.com/triztian/gomobile-v2mod/v2/subpkg (from $GOPATH)))))

rm -r -f "$WORK"
@gopherbot gopherbot added this to the Unreleased milestone Jul 25, 2019
@gopherbot gopherbot added the mobile label Jul 25, 2019
@AlexRouSg

This comment has been minimized.

Copy link
Contributor

@AlexRouSg AlexRouSg commented Jul 25, 2019

gomobile does not work in module mode. See #27234

@triztian

This comment has been minimized.

Copy link
Author

@triztian triztian commented Jul 25, 2019

@AlexRouSg Thank you for the reply yes and thank you for the link!, I was aware of this limitation, however this issue is slightly different, in the issue that you mention the build is done with GO111MODULE=on (and seemingly outside a GOPATH) as it can be seen from the command output:

gomobile: go build -buildmode=c-shared -o=/var/folders/nt/0hzfppd92jvbk99fhvnhy8ph0000gn/T/gomobile-work-327189385/android/src/main/jniLibs/armeabi-v7a/libgojni.so gobind failed: exit status 1
go: finding github.com/dustin/go-humanize v0.0.0-20180713052910-9f541cc9db5d
go: finding golang.org/x/mobile v0.0.0-20180808221059-bceb7ef27cc6
can't load package: package gobind: unknown import path "gobind": cannot find module providing package gobind

(notice the go: finding ... statements, those indicate that the module system is being used)

In this issue the the build is done inside a GOPATH and with GO111MODULE=off, I was under the impression that the rules defined here would take effect:

The type-checking package "github.com/triztian/gomobile-v2target" ... error message is generated from srcimporter.go however the minimal compatibility notes mentions:

The entirety of the logic is – when operating in GOPATH mode, an unresolvable import statement containing a /vN will be tried again after removing the /vN if the import statement is inside code that has opted in to modules (that is, import statements in .go files within a tree with a valid go.mod file).

And the type-check error points towards the module not being tried again (for whatever reason), in this situation the imported package gomobile-v2mod has opted into modules and is inside the GOPATH so I'd expect for the retry to happen but removing the /v2 suffix.

After all of that I'm not denying that the root issue is the one you linked, but it could be that it's a different one.

@AlexRouSg

This comment has been minimized.

Copy link
Contributor

@AlexRouSg AlexRouSg commented Jul 25, 2019

The issue is github.com/triztian/gomobile-v2target imports github.com/triztian/gomobile-v2mod, but github.com/triztian/gomobile-v2mod then imports github.com/triztian/gomobile-v2mod/v2/subpkg

This will not work since there is no such package.

@AlexRouSg

This comment has been minimized.

Copy link
Contributor

@AlexRouSg AlexRouSg commented Jul 25, 2019

cc @eliasnaur @hyangah for comments about retrying without /v2

@triztian

This comment has been minimized.

Copy link
Author

@triztian triztian commented Jul 25, 2019

I have a bit more info, after playing a bit with the source, I narrowed the error to this line:

Basically a syntax-check is performed before any build is attempted, the syntax check happens without the minimal version compat that the modloads and several other modyyy packages provide when the go build command is ran on source files.

I think it might be possible to provide an Importer (interface) that is able to handle the module that opted in packages through an in-memory rewrite of the source for the package that has the exports
for the framework:

Example Excerpt: https://golang.org/pkg/go/types/#example_Info

fset := token.NewFileSet()
f, err := parser.ParseFile(fset, "fib.go", input, 0)
if err != nil {
	log.Fatal(err)
}

// Type-check the package.
// We create an empty map for each kind of input
// we're interested in, and Check populates them.
info := types.Info{
	Types: make(map[ast.Expr]types.TypeAndValue),
	Defs:  make(map[*ast.Ident]types.Object),
	Uses:  make(map[*ast.Ident]types.Object),
}

var conf types.Config
pkg, err := conf.Check("fib", fset, []*ast.File{f}, &info)
if err != nil {
	log.Fatal(err)
}
@triztian

This comment has been minimized.

Copy link
Author

@triztian triztian commented Jul 27, 2019

I've provided a working patch here, well at least for the 2 sample projects:

Basically it provides a custom importer that performs an in-memory rewrite of import paths of packages that have major versions in their paths i.e: "example.org/lib/v6" into "example.org/lib"

This does not add support for modules but rather it allows you to import packages that have opted into go modules and that have a vN path component without having to edit their sources.

To test the patch, just clone the branch from here https://github.com/triztian/mobile/tree/issue/33272 and reinstall the gobind command:

go install ./cmd/gobind

You may also need to re-init gomobile but I'm not entirely sure if thats necessary:

gomobile init

Setup

Clone the sample projects into your gopath (they've been updated):

Then try and build the gomobile-v2target package:

The relevant proofs are that v2target imports v2mod and it has a v2 part in it's import:

// github.com/triztian/gomobile-v2target/export.go:8
v2mod "github.com/triztian/gomobile-v2mod/v2"

gomobile-v2target also imports one of it's subpackage with a v2 path component:

// github.com/triztian/gomobile-v2mod/math.go:4
import "github.com/triztian/gomobile-v2mod/v2/subpkg"

To build gomobile-v2target execute the following:

make clean run

You will need a swift compiler installed:

λ gomobile-v2target ~> swift --version
Apple Swift version 5.0.1 (swiftlang-1001.0.82.4 clang-1001.0.46.5)
Target: x86_64-apple-darwin18.6.0

Successful Output:

gomobile bind -target ios -work -a -v -tags "" .
WORK=/var/folders/7s/cnfg6ksj44gb3br_lc4v3gn40000gp/T/gomobile-work-132492778/go-build699643396
runtime/cgo
golang.org/x/mobile/internal/mobileinit
gobind
WORK=/var/folders/7s/cnfg6ksj44gb3br_lc4v3gn40000gp/T/gomobile-work-132492778/go-build206905632
runtime/cgo
golang.org/x/mobile/internal/mobileinit
gobind
WORK=/var/folders/7s/cnfg6ksj44gb3br_lc4v3gn40000gp/T/gomobile-work-132492778/go-build468413038
runtime/cgo
golang.org/x/mobile/internal/mobileinit
gobind
WORK=/var/folders/7s/cnfg6ksj44gb3br_lc4v3gn40000gp/T/gomobile-work-132492778/go-build230851473
runtime/cgo
golang.org/x/mobile/internal/mobileinit
gobind
write Mobile.framework/Versions/A/Headers/Mobile.objc.h
write Mobile.framework/Versions/A/Headers/Universe.objc.h
write Mobile.framework/Versions/A/Headers/ref.h
write Mobile.framework/Versions/A/Headers/Mobile.h
write Mobile.framework/Resources/Info.plist
write Mobile.framework/Versions/A/Modules/module.modulemap
WORK=/var/folders/7s/cnfg6ksj44gb3br_lc4v3gn40000gp/T/gomobile-work-132492778
swiftc -F. -framework Mobile -framework Foundation ./main.swift
-------------
./main
Testing
Result 21 + 21 =  42
Sum call: 18

There is a minimal swift file to test the framework and functionality:

import Mobile
import Foundation

print("Testing")
MobileMain()
print("Sum call: \(MobileSum(10, 8))")

I've not tested how it behaves when a vendor directory exists in the gomobile-v2target, happy to receive feedback and reports of issues, the test packages are quite simple so testing with more real-world complex packages would be beneficial.

@hajimehoshi

This comment has been minimized.

Copy link
Contributor

@hajimehoshi hajimehoshi commented Oct 25, 2019

We are discussing to support Go module at #27234

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.