Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go/internal/get: VCS path regexp omits characters accepted by CheckImportPath #31376

Open
stub42 opened this issue Apr 10, 2019 · 7 comments
Labels
Milestone

Comments

@stub42
Copy link

@stub42 stub42 commented Apr 10, 2019

What version of Go are you using (go version)?

$ go version
go version go1.12.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/stub/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/stub/go"
GOPROXY=""
GORACE=""
GOROOT="/snap/go/3540"
GOTMPDIR=""
GOTOOLDIR="/snap/go/3540/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build922868437=/tmp/go-build -gno-record-gcc-switches"

What did you do?

Attempted to fetch the git repo containing a go package, publicly available at:

Per go help importpath, the following should attempt a git download using https://, falling back to git+ssh://

go get -v git.launchpad.net/~stub/+git/go-eggs.git

Attempting to force ssh per git help environment does not help

GIT_ALLOW_PROTOCOL=ssh go get -v git.launchpad.net/~stub/+git/go-eggs.git

What did you expect to see?

go get attempt to clone the git repo at https://git.launchpad.net/~stub/+git/go-eggs, and if that failed, fall back to cloning the git repo at git+ssh://git.launchpad.net/~stub/+git/go-eggs.

Per go help importpath, because the import path has a version control qualifier (.git) then no attempt should be made to attempt to get the import over https: to look for a tag.

What did you see instead?

Go get attempts to parse meta tags from https://, which fails. No attempt is made to clone the git repo.

$ go get -v git.launchpad.net/~stub/+git/go-eggs.git
Fetching https://git.launchpad.net/~stub/+git/go-eggs.git?go-get=1
Parsing meta tags from https://git.launchpad.net/~stub/+git/go-eggs.git?go-get=1 (status code 200)
package git.launchpad.net/~stub/+git/go-eggs.git: unrecognized import path "git.launchpad.net/~stub/+git/go-eggs.git" (parse https://git.launchpad.net/~stub/+git/go-eggs.git?go-get=1: no go-import meta tags ())
@stub42 stub42 changed the title Unable to 'go get' git repo Unable to 'go get' git repo, VCS qualifier ignored Apr 10, 2019
@bcmills bcmills changed the title Unable to 'go get' git repo, VCS qualifier ignored cmd/go: Unable to 'go get' git repo, VCS qualifier ignored Apr 10, 2019
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Apr 10, 2019

Duplicate of #26134

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Apr 11, 2019

The regular expression that looks for the .git suffix is here:

re: `^(?P<root>(?P<repo>([a-z0-9.\-]+\.)+[a-z0-9.\-]+(:[0-9]+)?(/~?[A-Za-z0-9_.\-]+)+?)\.(?P<vcs>bzr|fossil|git|hg|svn))(/~?[A-Za-z0-9_.\-]+)*$`,

Testing against that (playground) reveals that the problem here is that the + character fails to match the regular expression, which is more restrictive about + and ~ than our validation for import paths in general:

// pathOK reports whether r can appear in an import path element.
//
// NOTE: This function DIVERGES from module mode pathOK by accepting Unicode letters.
func pathOK(r rune) bool {
if r < utf8.RuneSelf {
return r == '+' || r == '-' || r == '.' || r == '_' || r == '~' ||
'0' <= r && r <= '9' ||
'A' <= r && r <= 'Z' ||
'a' <= r && r <= 'z'
}
return unicode.IsLetter(r)
}

@bcmills bcmills changed the title cmd/go: Unable to 'go get' git repo, VCS qualifier ignored cmd/go/internel/get: VCS path regexp omits characters accepted by CheckImportPAth Apr 11, 2019
@bcmills bcmills changed the title cmd/go/internel/get: VCS path regexp omits characters accepted by CheckImportPAth cmd/go/internel/get: VCS path regexp omits characters accepted by CheckImportPath Apr 11, 2019
@bcmills bcmills added this to the Go1.13 milestone Apr 11, 2019
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Apr 11, 2019

This needs a fix, but it's not obvious to me whether the VCS path should be made more permissive, or CheckImportPath should be made more restrictive. I'll follow up with @rsc and @jayconrod.

@rsc

This comment has been minimized.

Copy link
Contributor

@rsc rsc commented Apr 30, 2019

I'm confused by whether the +git is essential here. The URL already starts and ends with git. Must it also have git in the middle? Somehow we have gotten this far without allowing + in go get paths, and I want to understand why.

@stub42

This comment has been minimized.

Copy link
Author

@stub42 stub42 commented May 1, 2019

When Launchpad code hosting added git support it was necessary to add +git as a path component to avoid namespace clashes with Bazaar branches. Launchpad code hosting is pretty much stuck with it, but yes, + will be very uncommon, unlike other special characters such as ~. The trailing .git is optional, and only in my example because I wanted to skip the https: VCS detection (probably necessary for private repositories, when I get that far).

@rsc rsc changed the title cmd/go/internel/get: VCS path regexp omits characters accepted by CheckImportPath cmd/go/internal/get: VCS path regexp omits characters accepted by CheckImportPath May 1, 2019
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented May 8, 2019

I checked the new module index for other existing packages that use the + character, but haven't been able to locate any so far.

(That's not to say that they don't exist, but if they do they're rare.)

Perhaps we should disallow + from import paths entirely?

@rsc

This comment has been minimized.

Copy link
Contributor

@rsc rsc commented May 9, 2019

If this is not a breakage introduced after Go 1.12, I'm inclined to leave deciding what to do until after Go 1.13 is out.

@bcmills bcmills modified the milestones: Go1.13, Go1.14 May 9, 2019
@rsc rsc modified the milestones: Go1.14, Backlog Oct 9, 2019
@bcmills bcmills modified the milestones: Backlog, Go1.14 Oct 10, 2019
@ianlancetaylor ianlancetaylor modified the milestones: Go1.14, Go1.15 Dec 5, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
4 participants
You can’t perform that action at this time.