Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: document lookup process for qualified module paths #41267

Open
dwschulze opened this issue Sep 7, 2020 · 6 comments
Open

cmd/go: document lookup process for qualified module paths #41267

dwschulze opened this issue Sep 7, 2020 · 6 comments

Comments

@dwschulze
Copy link

@dwschulze dwschulze commented Sep 7, 2020

What version of Go are you using (go version)?

go version go1.15.1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/dean/.cache/go-build"
GOENV="/home/dean/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/home/dean/src/golang/go3p/pkg/mod"
GONOPROXY="ubuntu-18-extssd"
GONOSUMDB="ubuntu-18-extssd"
GOOS="linux"
GOPATH="/home/dean/src/golang/go3p:/home/dean/src/golang/examples/go-module-package-test/package/package-driver"
GOPRIVATE="ubuntu-18-extssd"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/home/dean/bin/go1.15.1.linux-amd64/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/home/dean/bin/go1.15.1.linux-amd64/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build013931004=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I have a git repo on a computer on my network (hostname ubuntu-18-extssd, IP 192.168.0.12):

$ git remote --v
origin	dean@ubuntu-18-extssd:gitrepo/go-package-test-stringutil (fetch)

Inside of the repo is a package stringpackage.

The go get command fails:

$ go get -v ubuntu-18-extssd/gitrepo/go-package-test-stringutil/stringpackage
package ubuntu-18-extssd/gitrepo/go-package-test-stringutil/stringpackage: unrecognized import path "ubuntu-18-extssd/gitrepo/go-package-test-stringutil/stringpackage": import path does not begin with hostname

So I try with the IP address:

$ go get -v 192.168.0.12/gitrepo/go-package-test-stringutil/stringpackage
package 192.168.0.12/gitrepo/go-package-test-stringutil/stringpackage: unrecognized import path "192.168.0.12/gitrepo/go-package-test-stringutil/stringpackage": https fetch: Get "https://192.168.0.12/gitrepo/go-package-test-stringutil/stringpackage?go-get=1": dial tcp 192.168.0.12:443: connect: connection refused

I add the following to ~/.gitconfig:

[url "ssh://dean@192.168.0.12:"]
	insteadOf = https://192.168.0.12/

I get the same error.

I've tried multiple ways of setting GOPRIVATE

export GOPRIVATE=192.168.0.12/gitrepo/*
export GOPRIVATE=ubuntu-18-extssd

but get the same error.

You need to actually try this on a local computer with a non-routeable IP address. I've tried the same thing on github.com and it works. The go get command only wants to work on public, routeable IP addresses.

Many companies keep their source code internally on hosts with non-routable IP addresses, and this prevents sharing code and makes using modules impossible.

@ALTree ALTree changed the title go get can't work with a non-routeable IP address cmd/go: get can't work with a non-routeable IP address Sep 8, 2020
@seankhliao
Copy link
Contributor

@seankhliao seankhliao commented Sep 8, 2020

This looks like 2 separate errors

import path does not begin with hostname

go currently does not consider names without dots as hosts, as of #37641 these are considered reserved names for the stdlib

https fetch: Get "https://192.168.0.12/gitrepo/go-package-test-stringutil/stringpackage?go-get=1": dial tcp 192.168.0.12:443: connect: connection refused

remote imports need to follow https://golang.org/cmd/go/#hdr-Remote_import_paths
specifically either end with .git for a git repo or answer to a http/https request with the meta tag

@dwschulze
Copy link
Author

@dwschulze dwschulze commented Sep 8, 2020

I added .git to the url like this

go get 192.168.0.12/gitrepo/go-package-test-stringutil.git/stringpackage

and it gives a different message:

package 192.168.0.12/gitrepo/go-package-test-stringutil.git/stringpackage: cannot download, git://192.168.0.12/gitrepo/go-package-test-stringutil uses insecure protocol

My ~/.gitconfig has this

[url "ssh://dean@192.168.0.12:"] insteadOf = https://192.168.0.12/

When go get calls git it seems to be picking up what I have in ~/.gitconfig but converts the ssh: prefix to git:. I believe that is the same thing, though. I have no idea why go get considers that an insecure protocol.

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Sep 8, 2020

@dwschulze Could you run with -x and post the log? That should show the actual commands run.

The git protocol is not secure. The Protocols has more information. git+ssh may be what you want instead. Or if you trust your local network, set GOINSECURE=192.168.0.12.

@seankhliao
Copy link
Contributor

@seankhliao seankhliao commented Sep 8, 2020

your git config also needs to rewrite the git protocol to ssh

@dwschulze
Copy link
Author

@dwschulze dwschulze commented Sep 8, 2020

I removed the ssh:// prefix from my ~/.gitconfig and now it is working:

[url "dean@192.168.0.12:"] insteadOf = https://192.168.0.12/

The docs really need to cover this and explain why having the ssh:// prefix gives an insecure protocol message. That is very confusing.

@jayconrod
Copy link
Contributor

@jayconrod jayconrod commented Sep 21, 2020

I'd like to guide this issue to some resolution. I don't think we really know what went wrong. Having the -x log would help.

This doesn't seem related to whether the host contains a non-routable address or not. We don't allow module paths that don't have a dot in the first path element (see Module paths and versions, but local names and IP addresses are fine.

Finding a repository for a module path describes the process. That section doesn't cover qualified module paths, so it should definitely be updated.

When the module path ends with a VCS qualifier like .git, the go command will derive a repository URL directly from that path instead of sending a ?go-get=1 request. It doesn't know the scheme, so it tries git, https, http, git+ssh, and ssh (git and http are only allowed if GOINSECURE matches).

@jayconrod jayconrod changed the title cmd/go: get can't work with a non-routeable IP address cmd/go: document lookup process for qualified module paths Sep 21, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
5 participants
You can’t perform that action at this time.