Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: cannot go get package from private Gitlab repo that is in a subgroup #42632

Open
leighhopcroft opened this issue Nov 16, 2020 · 5 comments
Open

Comments

@leighhopcroft
Copy link

@leighhopcroft leighhopcroft commented Nov 16, 2020

What version of Go are you using (go version)?

$ go version
go version go1.15.5 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE="auto"
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/vscode/.cache/go-build"
GOENV="/home/vscode/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOINSECURE=""
GOMODCACHE="/go/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="linux"
GOPATH="/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/workspaces/iam-vpn/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build576004399=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I tried to go get a package from a private Gitlab repo that is in a subgroup.

What did you expect to see?

I expected the go get command to download the package dependency without an error i.e.

$ GOPRIVATE=myprivategitlabinstance.com go get -v myprivategitlabinstance.com/mygroup/mysubgroup/myrepo
get "myprivategitlabinstance.com/mygroup/mysubgroup/myrepo": found meta tag get.metaImport{Prefix:"myprivategitlabinstance.com/mygroup/mysubgroup/myrepo", VCS:"git", RepoRoot:"https://myprivategitlabinstance.com/mygroup/mysubgroup/myrepo.git"} at //myprivategitlabinstance.com/mygroup/mysubgroup/myrepo?go-get=1
go: downloading myprivategitlabinstance.com/mygroup/mysubgroup/myrepo $VERSION
...

What did you see instead?

An error when running go get. Parsing of the repo root is incorrect, I think due to go get interpereting the import as mysubgroup/myrepo i.e. myrepo is a subpackage of the mysubgroup pacakge. go get therefore tries to pull from "https://myprivategitlabinstance.com/mygroup/mysubgroup.git" which is not a git repository but a Gitlab subgroup which contains git repositories.

$ GOPRIVATE=myprivategitlabinstance.com go get -v myprivategitlabinstance.com/mygroup/mysubgroup/myrepo
get "myprivategitlabinstance.com/mygroup/mysubgroup": found meta tag get.metaImport{Prefix:"myprivategitlabinstance.com/mygroup/mysubgroup", VCS:"git", RepoRoot:"https://myprivategitlabinstance.com/mygroup/mysubgroup.git"} at //myprivategitlabinstance.com/mygroup/mysubgroup?go-get=1
get "myprivategitlabinstance.com/mygroup/mysubgroup/myrepo": found meta tag get.metaImport{Prefix:"myprivategitlabinstance.com/mygroup/mysubgroup", VCS:"git", RepoRoot:"https://myprivategitlabinstance.com/mygroup/mysubgroup.git"} at //myprivategitlabinstance.com/mygroup/mysubgroup/myrepo?go-get=1
get "myprivategitlabinstance.com/mygroup/mysubgroup/myrepo": verifying non-authoritative meta tag
go get myprivategitlabinstance.com/mygroup/mysubgroup/myrepo: module myprivategitlabinstance.com/mygroup/mysubgroup/myrepo: git ls-remote -q origin in /go/pkg/mod/cache/vcs/$HASH: exit status 128:
        The project you were looking for could not be found.
        fatal: Could not read from remote repository.

        Please make sure you have the correct access rights
        and the repository exists.

@leighhopcroft
Copy link
Author

@leighhopcroft leighhopcroft commented Nov 16, 2020

I see #34094 which looks like the same problem - this is closed and resolved, but it still seems to be an issue for me.

Edit: It seems actually that #34094 is related, but not the same issue as the one I'm experiencing. That issue relates to getting gitlab.com/group/project/submodule whereas mine is effectively gitlab.com/group/subgroup/project

@leighhopcroft
Copy link
Author

@leighhopcroft leighhopcroft commented Nov 16, 2020

FYI I'm using SSH to access the private repo, using the git insteadOf config:

$ git config --global git@myprivategitlabinstance.com:.insteadOf https://myprivategitlabinstance.com/
$ cat ~/.gitconfig
[url "git@myprivategitlabinstance.com:"]
    insteadOf = https://myprivategitlabinstance.com/

I see some mention of a .netrc file working in #34094 but I'm not sure how I would get that working with SSH on Windows (the above go env was run in a linux vs-code devcontainer that shares SSH credentials through the SSH agent from Windows).

@leighhopcroft
Copy link
Author

@leighhopcroft leighhopcroft commented Nov 16, 2020

Also, this set up works with a different repo on the same private gitlab instance, that is not in a subgroup i.e. has the format myprivategitlabinstance.com/mygroup/my-other-repo

@seankhliao
Copy link
Contributor

@seankhliao seankhliao commented Nov 16, 2020

netrc is independent of ssh, the http calls happen before git is invoked to determine the repo root. Gitlab has decided to only respond with the meta tags on authenticated calls to prevent leaking data. netrc needs to be set for wherever your go toolchain is running

@bcmills
Copy link
Member

@bcmills bcmills commented Nov 16, 2020

Or, if you really want to bypass the HTTPS resolution entirely, you need an explicit .git suffix at the repo root in the import path (and therefore also in the module path declared in the go.mod file: module myprivategitlabinstance.com/mygroup/mysubgroup/myrepo.git and go get myprivategitlabinstance.com/mygroup/mysubgroup/myrepo.git.

Otherwise, as @seankhliao notes, you'll need a .netrc file so that the Go command can authenticate to the GitLab server in order to identify the repo root for the requested path.

@bcmills bcmills changed the title Cannot go get package from private Gitlab repo that is in a subgroup cmd/go: cannot go get package from private Gitlab repo that is in a subgroup Nov 16, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants
You can’t perform that action at this time.