Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: `go get` fails when repository ends with `.go` #32483

Closed
wallyqs opened this issue Jun 7, 2019 · 30 comments
Closed

cmd/go: `go get` fails when repository ends with `.go` #32483

wallyqs opened this issue Jun 7, 2019 · 30 comments

Comments

@wallyqs
Copy link

@wallyqs wallyqs commented Jun 7, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/himitsu/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/himitsu/repos/lkjh-dev"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.12/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.12/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/xl/vzfvbyrs29s3t2mmh15ygmv00000gn/T/go-build506667951=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

When trying to fetch the latest version or a tag of the NATS Go client as follows, it fails with a no such file or directory error.

GO111MODULE=on go get github.com/nats-io/nats.go@latest
stat github.com/nats-io/nats.go: no such file or directory

GO111MODULE=on go get github.com/nats-io/nats.go@v1.8.1
stat github.com/nats-io/nats.go: no such file or directory

One workaround for this when using Go 1.12 is to append an extra slash at the end of the repository name and then add the explicit version as follows:

GO111MODULE=on go get github.com/nats-io/nats.go/@latest
...

# Explicit tag version
GO111MODULE=on go get github.com/nats-io/nats.go/@v1.8.0
go: finding github.com/nats-io/nats.go v1.8.0
go: downloading github.com/nats-io/nats.go v1.8.0
go: extracting github.com/nats-io/nats.go v1.8.0

What did you expect to see?

I expected go get to download the library as follows:

GO111MODULE=on go get github.com/nats-io/nats.go@v1.8.0
go: finding github.com/nats-io/nats.go v1.8.0
go: downloading github.com/nats-io/nats.go v1.8.0
go: extracting github.com/nats-io/nats.go v1.8.0

Or when not specifying a version either, expected for it to download the latest one:

GO111MODULE=on go get github.com/nats-io/nats.go
go: finding github.com/nats-io/nats.go v1.8.1
go: downloading github.com/nats-io/nats.go v1.8.1
go: extracting github.com/nats-io/nats.go v1.8.1

What did you see instead?

GO111MODULE=on go get github.com/nats-io/nats.go@v1.8.0
stat github.com/nats-io/nats.go: no such file or directory

Also when not specifying any version always fail, regardless of trailing slash or not:

GO111MODULE=on go get github.com/nats-io/nats.go/
stat github.com/nats-io/nats.go: no such file or directory

GO111MODULE=on go get github.com/nats-io/nats.go
stat github.com/nats-io/nats.go: no such file or directory
@bcmills bcmills changed the title cmd/go/: `go get` fails when repository ends with `.go` unless it has a trailing slash cmd/go: `go get` fails when repository ends with `.go` unless it has a trailing slash Jun 7, 2019
@bcmills bcmills added this to the Go1.14 milestone Jun 7, 2019
@bcmills
Copy link
Member

@bcmills bcmills commented Jun 7, 2019

Paths ending in .go usually refer to source files rather than package or module paths; presumably that is the problem here.

CC @jayconrod

@gopherbot
Copy link

@gopherbot gopherbot commented Jun 10, 2019

Change https://golang.org/cl/180923 mentions this issue: cmd/goget: support package ended with '.go'

@letientai299
Copy link
Contributor

@letientai299 letientai299 commented Jun 10, 2019

@dmitshur, I'm still quite new to golang project workflow, so, thanks for the reminder, and please guide me through this second CL.

My fix in above CL is basically trying load patterns ended with .go as go files, and if it fails, fall back to loading the patterns as packages.

@dmitshur
Copy link
Member

@dmitshur dmitshur commented Jun 10, 2019

Thanks @letientai299. You should discuss the approach you've considered with @bcmills and @jayconrod here, since they're owners of cmd/go behavior, and see what they think.

trying load patterns ended with .go as go files, and if it fails, fall back to loading the patterns as packages.

I have a potential concern about it, in that it introduces unpredictable behavior. The same command invocation may act differently depending on whether or not a .go file exists and can be accessed without errors. I'm not sure of a better solution, I just wanted to point out the potential concern.

@bcmills bcmills removed the modules label Jun 10, 2019
@bcmills
Copy link
Member

@bcmills bcmills commented Jun 10, 2019

Note that the same command also fails in GOPATH mode, at least as far back as 1.11.10:

$ GO111MODULE=off go1.12.5 get github.com/nats-io/nats.go
stat github.com/nats-io/nats.go: no such file or directory

$ GO111MODULE=off go1.11.10 get github.com/nats-io/nats.go
stat github.com/nats-io/nats.go: no such file or directory

Given the lack of other reports of this issue, perhaps nats-go should just pick a less ambiguous name for their module‽

(Or are there substantially more examples of this in the wild?)

@wallyqs
Copy link
Author

@wallyqs wallyqs commented Jun 10, 2019

@bcmills yes you also have to add trailing slash if not using Go modules. From a similar past issue that is open it seems that this may have been introduced around Go 1.11, still ok in Go 1.10: #27096

@bcmills
Copy link
Member

@bcmills bcmills commented Jun 10, 2019

@wallyqs, note that the concrete example of such a package in #27096 is also nats-io/nats.go.

@wallyqs
Copy link
Author

@wallyqs wallyqs commented Jun 10, 2019

@bcmills I mean that the issue mentions others running into a similar issue, though we also commented on that issue. Btw thanks @letientai299 for looking at the possible fix!

@letientai299
Copy link
Contributor

@letientai299 letientai299 commented Jul 3, 2019

@dmitshur

I have a potential concern about it, in that it introduces unpredictable behavior. The same command invocation may act differently depending on whether or not a .go file exists and can be accessed without errors. I'm not sure of a better solution, I just wanted to point out the potential concern.

Previous implementation also have that issue. The code check if there's any pattern ends with .go and treat all of them as go files.

@wallyqs
Copy link
Author

@wallyqs wallyqs commented Sep 9, 2019

It looks like this behavior has worsened in Go 1.13, now it is not possible at all to go get the nats module repository:

# --- Go 1.13
GO111MODULE=on go get github.com/nats-io/nats.go/@v1.8.0
go: finding github.com v1.8.0
go: finding github.com/nats-io v1.8.0
go get github.com/nats-io/nats.go/@v1.8.0: malformed module path "github.com/nats-io/nats.go/": trailing slash

GO111MODULE=on go get github.com/nats-io/nats.go/@latest
go get github.com/nats-io/nats.go/@latest: malformed module path "github.com/nats-io/nats.go/": trailing slash

GO111MODULE=on /usr/local/go/bin/go get github.com/nats-io/nats.go
stat github.com/nats-io/nats.go: no such file or directory

GO111MODULE=on /usr/local/go/bin/go get github.com/nats-io/nats.go/
stat github.com/nats-io/nats.go: no such file or directory

GO111MODULE=on go get github.com/nats-io/nats.go@latest
stat github.com/nats-io/nats.go: no such file or directory

GO111MODULE=on go get github.com/nats-io/nats.go@v1.8.0
go: finding github.com v1.8.0
go: finding github.com/nats-io v1.8.0
stat github.com/nats-io/nats.go: no such file or directory
@wallyqs wallyqs changed the title cmd/go: `go get` fails when repository ends with `.go` unless it has a trailing slash cmd/go: `go get` fails when repository ends with `.go` Sep 9, 2019
@derekcollison
Copy link

@derekcollison derekcollison commented Sep 9, 2019

@bcmills Please keep us posted on this. Seems pretty bad behavior that has totally broke the NATS.io ecosystem. NATS has been with Go since 0.52 IIRC.

/cc @bradfitz @Sajmani

@wallyqs
Copy link
Author

@wallyqs wallyqs commented Sep 9, 2019

Only variation that seems to work is when disabling go modules and adding a trailing slash to go get:

GO111MODULE=off go get github.com/nats-io/nats.go
stat github.com/nats-io/nats.go: no such file or directory

GO111MODULE=off go get github.com/nats-io/nats.go/
# No `trailing slash` error
@gopherbot
Copy link

@gopherbot gopherbot commented Sep 10, 2019

Change https://golang.org/cl/194600 mentions this issue: cmd/go: strip trailing slash from versioned arguments

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Sep 10, 2019

CL 194600 should fix the regression in 1.13. I'd like to backport that for 1.13.1 if possible.

go get operates in two phases: it updates go.mod, then runs go build on the arguments (kind of). Both phases are having problems.

When we do the work needed to update go.mod, we weren't stripping trailing slashes from arguments with versions. So go get -d github.com/nats-io/nats.go/@latest would fail when we went to validate the proxy base URL, since trailing slashes aren't allowed in a module path. It works without the trailing slash though (go get -d github.com/nats-io/nats.go@latest succeeds).

If we do strip the trailing slashes, then the second go build phase fails because it treats arguments ending with .go as files, unlike the rest of go get. So CL 194600 will strip trailing slashes but preserve them for this phase.

@bcmills
Copy link
Member

@bcmills bcmills commented Sep 10, 2019

Note that the nats.go maintainers in particular knew that this repo name was a problem literally four days after renaming the repository and didn't roll back at that point.
(Compare the dates on #27096 (comment) and nats-io/nats.go@4c154eb.)

I think I'm ok with a minimal change to restore compatibility with commands that happened to work in 1.12, but I can't justify sinking a lot of time into ongoing support for these names that never really worked to begin with.

@derekcollison
Copy link

@derekcollison derekcollison commented Sep 10, 2019

@bcmills For ecosystems like NATS that have many client implementations it has become pretty common to name the client repo .. This is not just a NATS thing.

This is a silly problem that the Go team should resolve.

@derekcollison
Copy link

@derekcollison derekcollison commented Sep 10, 2019

gopherbot pushed a commit that referenced this issue Sep 11, 2019
'go get' accepts arguments of the form path@version, and it passes
them through search.CleanPatterns before querying proxies. With this
change, CleanPatterns preserves text after '@' and will strip trailing
slashes from the patn.

Previously, we did not strip trailing slashes when a version was
present, which caused proxy base URL validation to fail. Module paths
that end with ".go" (for example, github.com/nats-io/nats.go) use
trailing slashes to prevent 'go build' and other commands from
interpreting packages as source file names, so this caused unnecessary
problems for them.

Updates #32483

Change-Id: Id3730c52089e52f1cac446617c20132a3021a808
Reviewed-on: https://go-review.googlesource.com/c/go/+/194600
Run-TryBot: Jay Conrod <jayconrod@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Bryan C. Mills <bcmills@google.com>
@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Sep 11, 2019

@gopherbot please backport to 1.13, this was a regression

@gopherbot
Copy link

@gopherbot gopherbot commented Sep 11, 2019

Backport issue(s) opened: #34243 (for 1.13).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

@derekcollison
Copy link

@derekcollison derekcollison commented Sep 11, 2019

Thanks, much appreciated.

@wallyqs
Copy link
Author

@wallyqs wallyqs commented Sep 11, 2019

Thanks @jayconrod @bcmills just tried it building Go from master and it works great 🙌
Looking forward to the 1.13.1 release!

@halega
Copy link

@halega halega commented Sep 26, 2019

1.13.1 is out, but the problem still exists:

$ go version
go version go1.13.1 windows/amd64

$ go get github.com/nats-io/nats.go/@latest
go get github.com/nats-io/nats.go/@latest: malformed module path "github.com/nats-io/nats.go/": trailing slash

$ go get github.com/nats-io/nats.go@latest
CreateFile github.com/nats-io/nats.go: The system cannot find the path specified.

How to add NATS client to a project?

@dmitshur
Copy link
Member

@dmitshur dmitshur commented Sep 26, 2019

@halega, Go 1.13.1 was a security release. Such releases contain only the relevant security fix and no other changes, in order to make them safer to update to. All other changes will be a part of the next minor release, Go 1.13.2, which will happen soon (beginning of October).

@gopherbot
Copy link

@gopherbot gopherbot commented Oct 4, 2019

Change https://golang.org/cl/198459 mentions this issue: cmd/go: fix listing of ambiguous paths

@gopherbot
Copy link

@gopherbot gopherbot commented Oct 4, 2019

Change https://golang.org/cl/198957 mentions this issue: cmd/go: fix listing of ambiguous paths

@derekcollison
Copy link

@derekcollison derekcollison commented Oct 17, 2019

Was expecting this to be in 1.13.2 but it was not. Any estimations or timeframe to have this released?

@bcmills
Copy link
Member

@bcmills bcmills commented Oct 17, 2019

1.13.2 ended up being a security release, so everything not security-related got bumped to 1.13.3.

@derekcollison
Copy link

@derekcollison derekcollison commented Oct 17, 2019

We just figured out that the fix was not in there. Do you folks have an eta?

@agnivade
Copy link
Contributor

@agnivade agnivade commented Oct 18, 2019

Should be out soon enough. Feel free to subscribe to #34970.

@derekcollison
Copy link

@derekcollison derekcollison commented Oct 18, 2019

Thanks!

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

9 participants
You can’t perform that action at this time.