Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: can't load package when GOPATH is under GOROOT #32621

Open
Helflym opened this issue Jun 14, 2019 · 17 comments

Comments

@Helflym
Copy link
Contributor

@Helflym Helflym commented Jun 14, 2019

The go command seems to have trouble parsing the package name when it's under $GOROOT/src.
The error is obvious with gccgo (so also with version 1.12). However, I'm not sure if the error from TIP is related or not.

$ go env
GOPATH="/opt/freeware/src/packages/BUILD/go-path"
GOROOT="/opt/freeware"
$ cd $GOPATH/src/golang.org/x/sys/unix/

WITH GCCGO
$ go.gcc version  
go version go1.12.2 gccgo (GCC) 9.1.0 aix/ppc64
$ go.gcc build 
can't load package: package packages/BUILD/go-path/src/golang.org/x/sys/unix: cannot find package "packages/BUILD/go-path/src/golang.org/x/sys/unix" in any of:
        /opt/freeware/src/packages/BUILD/go-path/src/golang.org/x/sys/unix (from $GOROOT)
        /opt/freeware/src/packages/BUILD/go-path/src/packages/BUILD/go-path/src/golang.org/x/sys/unix (from $GOPATH)

WITH GO 1.12 
$ go version
go version go1.12.5 aix/ppc64
$ go.golang build
can't load package: package packages/BUILD/go-path/src/golang.org/x/sys/unix: code in directory /opt/freeware/src/packages/BUILD/go-path/src/golang.org/x/sys/unix expects import "golang.org/x/sys/unix"

WITH TIP
$ go version
go version devel +d2f0628430 Thu Jun 13 09:51:55 2019 -0500 aix/ppc64
$ go build
go: inconsistent vendoring in /opt/freeware/src/packages/BUILD/go-path/src/golang.org/x/sys:
        go.mod requires golang.org/x/sys  but vendor/modules.txt does not include it.
        run 'go mod tidy; go mod vendor' to sync

This alos happen on Fedora 30 (at least with gccgo).
I do agree that having GOPATH under GOROOT isn't the best idea ever. But this is sometime the case on AIX (and maybe Fedora), because default GOROOT is "/opt/freeware" and the RPM are built under "/opt/freeware/src/packages/BUILD".

Therefore, is it a true bug in go command or a problem related to how Go RPMs are delivered ?

@FiloSottile

This comment has been minimized.

Copy link
Member

@FiloSottile FiloSottile commented Jun 14, 2019

Setting GOPATH within GOROOT is definitely not a supported configuration, but maybe we should generate a better error for it.

/cc @jayconrod

@FiloSottile FiloSottile added this to the Unplanned milestone Jun 14, 2019
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jun 17, 2019

A GOROOT of /opt/freeware seems like a problem in general. GOROOT has a lot of really generic directories such as src, test, and misc that are likely to conflict with other programs.

In particular, if we were to add a packages package to the standard library, then a GOROOT of /opt/freeware would place that package's source at /opt/freeware/src/packages.

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Jun 17, 2019

At any rate, we do have a similar existing diagnostic, if someone wants to expand it using search.InDir:

go/src/cmd/go/main.go

Lines 106 to 111 in 82cf8bc

// Diagnose common mistake: GOPATH==GOROOT.
// This setting is equivalent to not setting GOPATH at all,
// which is not what most people want when they do it.
if gopath := cfg.BuildContext.GOPATH; filepath.Clean(gopath) == filepath.Clean(runtime.GOROOT()) {
fmt.Fprintf(os.Stderr, "warning: GOPATH set to GOROOT (%s) has no effect\n", gopath)
} else {

@bcmills bcmills added the help wanted label Jun 17, 2019
@Helflym

This comment has been minimized.

Copy link
Contributor Author

@Helflym Helflym commented Jun 17, 2019

GOROOT is set to /opt/freeware only with gccgo. And I think this is because we are using --prefix="/opt/freeware" when building gcc in general.
Maybe @ianlancetaylor, you're aware of such problems in a different OS ?
Edit: I do confirm with this is linked with `--prefix`` and comes from https://github.com/golang/gofrontend/blob/master/libgo/Makefile.am#L556. Maybe it can be adapted for AIX.

@ianlancetaylor

This comment has been minimized.

Copy link
Contributor

@ianlancetaylor ianlancetaylor commented Jun 19, 2019

The gccgo default for GOROOT is definitely problematic. See https://gcc.gnu.org/PR89171. I've been intending to take a look at it before the next GCC release.

@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Jul 10, 2019

Change https://golang.org/cl/185540 mentions this issue: cmd/go: Warn when GOPATH is a subdir of GOROOT

@qbradq

This comment has been minimized.

Copy link
Contributor

@qbradq qbradq commented Jul 10, 2019

I am unable to reproduce the issue. The only way I can convince go to not find something under GOPATH is to reference it incorrectly. Even when GOPATH is a sub-directory of GOROOT. Can you post the code or at least the import block @Helflym ?

@Helflym

This comment has been minimized.

Copy link
Contributor Author

@Helflym Helflym commented Jul 11, 2019

You're right, my error occurs only because GOPATH is a sub-directory of $GOROOT/src (and not simply $GOROOT). It's very unlikely that it'll ever occur with Golang as GOROOT as a true meaning. However, with the current gccgo, GOROOT is more a placeholder AFAIK.

$ go env 
GOROOT="/tmp/go-root"
GOPATH="/tmp/go-root/src/go-path"
$ cd /tmp/go-root/src/go-path/src/golang.org/x/sys/unix 
$ GO111MODULE=off go build
can't load package: package go-path/src/golang.org/x/sys/unix: code in directory /tmp/go-root/src/go-path/src/golang.org/x/sys/unix expects import "golang.org/x/sys/unix"

GO111MODULE=off is just to make a more explicit error.

@qbradq

This comment has been minimized.

Copy link
Contributor

@qbradq qbradq commented Jul 11, 2019

@Helflym I still have not be able to reproduce this. Can you at least post the import block of your main go file? I have tried all the combinations I can think of as far as file and import structure to cause this, but I get no errors.

qbradq@qbradq-AB350-Gaming:~/golang/src/go-path/src/golang.org/x/sys/unix$ set | grep GO
GOPATH=/home/qbradq/golang/src/go-path
GOROOT=/home/qbradq/golang
qbradq@qbradq-AB350-Gaming:~/golang/src/go-path/src/golang.org/x/sys/unix$ cat main.go
package main

import "golang.org/x/sys/unix/sub"

func main() {
	sub.SayHello()
}
qbradq@qbradq-AB350-Gaming:~/golang/src/go-path/src/golang.org/x/sys/unix$ cat sub/lib.go
package sub

import "fmt"

// SayHello says hello
func SayHello() {
	fmt.Println("Hello, #32621!")
}
qbradq@qbradq-AB350-Gaming:~/golang/src/go-path/src/golang.org/x/sys/unix$ GO111MODULE=off $GOROOT/bin/go build
qbradq@qbradq-AB350-Gaming:~/golang/src/go-path/src/golang.org/x/sys/unix$ ./unix
Hello, #32621!

@Helflym

This comment has been minimized.

Copy link
Contributor Author

@Helflym Helflym commented Jul 12, 2019

I was using the whole x/sys/unix package (https://github.com/golang/sys) not a simple main.go and its sub package.
But indeed, it does work with your simple example. It could be interesting to know why it does with this simple package but it doesn't with a complex package like x/sys/unix.
Anyway, here is the imports of x/sys/unix,

$ go list -f '{{ .Imports }}'
[bytes encoding/binary net runtime sort strings sync syscall time unsafe]
@qbradq

This comment has been minimized.

Copy link
Contributor

@qbradq qbradq commented Jul 12, 2019

@Helflym Thank you the patience :) I understand now that you are trying to build the golang.org/x/sys/unix package. I was able to reproduce the issue when building this package. I am continuing analysis now.

To reproduce:

$ mkdir /tmp/issue-32621
$ cd /tmp/issue-32621
$ export GOROOT=$(pwd)
$ export GOPATH=$GOROOT/src/gopath
$ mkdir -p $GOPATH
$ go get golang.org/x/sys/unix
package bytes: unrecognized import path "bytes" (import path does not begin with hostname)
package encoding/binary: unrecognized import path "encoding/binary" (import path does not begin with hostname)
package runtime: unrecognized import path "runtime" (import path does not begin with hostname)
package sort: unrecognized import path "sort" (import path does not begin with hostname)
package strings: unrecognized import path "strings" (import path does not begin with hostname)
package sync: unrecognized import path "sync" (import path does not begin with hostname)
package syscall: unrecognized import path "syscall" (import path does not begin with hostname)
package time: unrecognized import path "time" (import path does not begin with hostname)
package unsafe: unrecognized import path "unsafe" (import path does not begin with hostname)
$ cd src/gopath/src/golang.org/x/sys/unix/
$ go build
can't load package: package gopath/src/golang.org/x/sys/unix: code in directory /tmp/issue-32621/src/gopath/src/golang.org/x/sys/unix expects import "golang.org/x/sys/unix"

@FiloSottile Should we just add a warning about this unsupported configuration (and fix the GCC default prefix) or try to find the root cause?

@FiloSottile

This comment has been minimized.

Copy link
Member

@FiloSottile FiloSottile commented Jul 15, 2019

Let's just detect it early, print a helpful error, and exit.

@qbradq

This comment has been minimized.

Copy link
Contributor

@qbradq qbradq commented Jul 16, 2019

@FiloSottile We can't exit. The go build process and as well as some internal test cases change GOPATH to be a subdirectory of GOROOT and expects it to work. So should it just print a helpful warning?

@FiloSottile

This comment has been minimized.

Copy link
Member

@FiloSottile FiloSottile commented Jul 16, 2019

Which part of the build process, and what tests? It seems weird to rely on an unsupported and apparently broken configuration.

@qbradq

This comment has been minimized.

Copy link
Contributor

@qbradq qbradq commented Jul 16, 2019

qbradq@qbradq-AB350-Gaming:~/godev/goroot/src$ ./all.bash
Building Go cmd/dist using /usr/lib/go-1.10.
Building Go toolchain1 using /usr/lib/go-1.10.
Building Go bootstrap cmd/go (go_bootstrap) using Go toolchain1.
Building Go toolchain2 using go_bootstrap and Go toolchain1.
go: GOPATH (/home/qbradq/godev/goroot/pkg/obj/gopath) under GOROOT (/home/qbradq/godev/goroot) is not supported
go tool dist: FAILED: /home/qbradq/godev/goroot/pkg/tool/linux_amd64/go_bootstrap install -gcflags=all= -ldflags=all= -i cmd/asm cmd/cgo cmd/compile cmd/link: exit status 2

Building toolchain2 apparently uses this configuration.

For an example of a test that uses this, see

func TestShadowingLogic(t *testing.T) {

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Aug 1, 2019

@qbradq, can you get things to work by narrowing the check to GOROOT/src rather than all of GOROOT?

@qbradq

This comment has been minimized.

Copy link
Contributor

@qbradq qbradq commented Aug 8, 2019

This was going to be my next try :) Unfortunately I broke my hand. That slowed me down. The Go build and test cases are all going to be under $GOROOT/src though, so it doesn't help.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
6 participants
You can’t perform that action at this time.