Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: module loader fails to resolve imports within symlinked source file #28107

Closed
zx2c4 opened this issue Oct 9, 2018 · 10 comments

Comments

Projects
None yet
5 participants
@zx2c4
Copy link
Contributor

commented Oct 9, 2018

This succeeds:

mkdir source-files && cd source-files
cp /path/to/somewhere/file1.go file1.go
cp /path/to/somewhere/file2.go file2.go
cp /path/to/somewhere/file3.go file3.go
cp /path/to/somewhere/go.mod go.mod
cp /path/to/somewhere/go.sum go.sum
go build

However, this fails:

mkdir source-files && cd source-files
ln -s /path/to/somewhere/file1.go file1.go
ln -s /path/to/somewhere/file2.go file2.go
ln -s /path/to/somewhere/file3.go file3.go
ln -s /path/to/somewhere/go.mod go.mod
ln -s /path/to/somewhere/go.sum go.sum
go build
[...FAILURES...]

It seems like go's module system might be trying to resolve things relative to the symlink target and not the actual symlink, which makes all sorts of nice overlay arrangements impossible unless you either hardlink (ew) or copy (yuck).

Here's a full sequence to reproduce

zx2c4@thinkpad ~ $ cd /tmp
zx2c4@thinkpad /tmp $ mkdir blah
zx2c4@thinkpad /tmp $ cd blah/
zx2c4@thinkpad /tmp/blah $ export GOPATH=$PWD/gopath
zx2c4@thinkpad /tmp/blah $ printf 'package main\nimport "golang.org/x/sys/unix"\nfunc main() { unix.Exit(0) }\n' > blah.go
zx2c4@thinkpad /tmp/blah $ go mod init blah
go: creating new go.mod: module blah
zx2c4@thinkpad /tmp/blah $ go build
go: finding golang.org/x/sys/unix latest
go: finding golang.org/x/sys latest
go: downloading golang.org/x/sys v0.0.0-20181005133103-4497e2df6f9e
zx2c4@thinkpad /tmp/blah $ ./blah
zx2c4@thinkpad /tmp/blah $ chmod -R +w gopath && rm -rf gopath
zx2c4@thinkpad /tmp/blah $ mkdir symlink
zx2c4@thinkpad /tmp/blah $ cd symlink
zx2c4@thinkpad /tmp/blah/symlink $ export GOPATH=$PWD/gopath
zx2c4@thinkpad /tmp/blah/symlink $ ln -s ../go.mod ../go.sum ../blah.go .
zx2c4@thinkpad /tmp/blah/symlink $ ls -l *
lrwxrwxrwx 1 zx2c4 zx2c4 10 Oct 10 06:27 blah.go -> ../blah.go
lrwxrwxrwx 1 zx2c4 zx2c4  9 Oct 10 06:27 go.mod -> ../go.mod
lrwxrwxrwx 1 zx2c4 zx2c4  9 Oct 10 06:27 go.sum -> ../go.sum
zx2c4@thinkpad /tmp/blah/symlink $ go build
go: finding golang.org/x/sys v0.0.0-20181005133103-4497e2df6f9e
blah.go:2:8: unknown import path "golang.org/x/sys/unix": cannot find package
@agnivade

This comment has been minimized.

Copy link
Member

commented Oct 10, 2018

@bcmills

This comment has been minimized.

Copy link
Member

commented Oct 10, 2018

Thanks for the straightforward reproducer. The problem seems to be the .go source file.


~/go/src/cmd/go$ pushd $(mktemp -d)
/tmp/tmp.0t9zEAmI9v ~/go/src/cmd/go

$ export GOPATH=$PWD/gopath

$ printf 'package main\nimport "golang.org/x/sys/unix"\nfunc main() { unix.Exit(0) }\n' > blah.go

$ go mod init blah
go: creating new go.mod: module blah

$ go build
go: finding golang.org/x/sys/unix latest
go: finding golang.org/x/sys latest
go: downloading golang.org/x/sys v0.0.0-20181005133103-4497e2df6f9e

$ ./blah

$ mkdir symlink

$ cd symlink

symlink$ export GOPATH=$PWD/gopath

symlink$ ln -s ../go.mod ../go.sum ../blah.go .

symlink$ go build
go: finding golang.org/x/sys v0.0.0-20181005133103-4497e2df6f9e
blah.go:2:8: unknown import path "golang.org/x/sys/unix": cannot find package

symlink$ rm blah.go

symlink$ cp ../blah.go ./

symlink$ go build
go: downloading golang.org/x/sys v0.0.0-20181005133103-4497e2df6f9e

symlink$

@bcmills bcmills added this to the Go1.12 milestone Oct 10, 2018

@agnivade agnivade changed the title modules fail with symlinked files cmd/go: modules fail with symlinked files Oct 10, 2018

@gopherbot

This comment has been minimized.

Copy link

commented Oct 10, 2018

Change https://golang.org/cl/141097 mentions this issue: cmd/go/internal/imports: resolve symlinks in ScanDir

@bcmills bcmills added NeedsFix and removed NeedsInvestigation labels Oct 10, 2018

@bcmills bcmills changed the title cmd/go: modules fail with symlinked files cmd/go: module loader fails to resolve imports within symlinked source file Oct 10, 2018

@zx2c4

This comment has been minimized.

Copy link
Contributor Author

commented Oct 10, 2018

@bcmills I'm not too certain on the cause of this, but broadly speaking, shouldn't the fix involve resolving fewer symlinks, not more?

@bcmills

This comment has been minimized.

Copy link
Member

commented Oct 11, 2018

Nope. The bug turned out to be in how we detect whether an entry within the package directory is a file or a directory: in order to do that for a symlink, we have to first resolve it.

@zx2c4

This comment has been minimized.

Copy link
Contributor Author

commented Oct 11, 2018

👍

@gopherbot gopherbot closed this in 440368d Nov 27, 2018

@gopherbot

This comment has been minimized.

Copy link

commented Feb 22, 2019

Change https://golang.org/cl/163517 mentions this issue: cmd/go/internal/imports: use the full path to resolve symlinks

gopherbot pushed a commit that referenced this issue Feb 23, 2019

cmd/go/internal/imports: use the full path to resolve symlinks
info.Name returns a name relative to the directory, so we need to
prefix that directory in the Stat call.

(This was missed in CL 141097 due to the fact that the test only
happened to check symlinks in the current directory.)

This allows the misc/ tests to work in module mode on platforms that
support symlinks.

Updates #30228
Updates #28107

Change-Id: Ie31836382df0cbd7d203b7a8b637c4743d68b6f3
Reviewed-on: https://go-review.googlesource.com/c/163517
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
@katiehockman

This comment has been minimized.

Copy link
Contributor

commented Apr 30, 2019

@gopherbot please open a backport to 1.12. This is a low risk patch that has enough benefits to warrant a cherrypick.

@gopherbot

This comment has been minimized.

Copy link

commented Apr 30, 2019

Backport issue(s) opened: #31763 (for 1.12).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

@gopherbot

This comment has been minimized.

Copy link

commented May 6, 2019

Change https://golang.org/cl/175441 mentions this issue: [release-branch.go1.12] cmd/go/internal/imports: use the full path to resolve symlinks

gopherbot pushed a commit that referenced this issue May 6, 2019

[release-branch.go1.12] cmd/go/internal/imports: use the full path to…
… resolve symlinks

info.Name returns a name relative to the directory, so we need to
prefix that directory in the Stat call.

(This was missed in CL 141097 due to the fact that the test only
happened to check symlinks in the current directory.)

This allows the misc/ tests to work in module mode on platforms that
support symlinks.

Updates #30228
Updates #28107
Fixes #31763

Change-Id: Ie31836382df0cbd7d203b7a8b637c4743d68b6f3
Reviewed-on: https://go-review.googlesource.com/c/163517
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Reviewed-on: https://go-review.googlesource.com/c/go/+/175441
Reviewed-by: Andrew Bonventre <andybons@golang.org>
Reviewed-by: Brad Fitzpatrick <bradfitz@golang.org>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.