Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: modules do not work with Gitlab.com/Company/Team/Repo Path format #29094

Closed
Marvalero opened this Issue Dec 4, 2018 · 10 comments

Comments

Projects
None yet
5 participants
@Marvalero
Copy link

Marvalero commented Dec 4, 2018

What version of Go are you using (go version)?

root@670ea5e97a56:/app# go version
go version go1.11.2 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
root@670ea5e97a56:/app# go version
go version go1.11.2 linux/amd64
root@670ea5e97a56:/app# go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/app/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build057206232=/tmp/go-build -gno-record-gcc-switches"

What did you do?

I was testing if go modules made sense for my company. We want to use it to create private libraries. So I wrote a simple test to check I can import submodules:

// File: main.go 
package main

import (
	"gitlab.com/Beamery/DevOps/gotalker"
	"gitlab.com/Beamery/gozzlightyear"
	"gitlab.com/maria.campana/sounds"
)

func main() {
	sounds.Run() // This just calls fmt.Println("Muuu")
	gozzlightyear.Run() // This just calls fmt.Println("to infinity and beyond")
	gotalker.Run()  // This just calls fmt.Println("there is a snake in my boot")
}
// File: go.mod 
module gitlab.com/Beamery/DevOps/maria-modules-test

Then I try to build the code. I am actually running my code on a container. This is the setup I do:

$ docker run -it --entrypoint bash -v /tmp/maria-modules-test:/app -v /home/maria/.ssh/:/root/.ssh/ golang:1.11.2-stretch
root@670ea5e97a56# cd /app
root@670ea5e97a56# git config --global url."ssh://git@gitlab.com/".insteadOf "https://gitlab.com/"
root@670ea5e97a56:/app# go build
build gitlab.com/Beamery/DevOps/maria-modules-test: cannot find module for path gitlab.com/Beamery/DevOps/gotalker

I think the problem is that the repo name is server.com/Company/Team/Repo and go expects to see server.com/Company/Repo. I tried removing the path "gitlab.com/Beamery/DevOps/gotalker" the app gets built correctly

What did you expect to see?

I expect Go Modules to support longer paths like server.com/Company/Team/Repo

What did you see instead?

I see build gitlab.com/Beamery/DevOps/maria-modules-test: cannot find module for path gitlab.com/Beamery/DevOps/gotalker

And If I manually add a require for it on go.mod

module gitlab.com/Beamery/DevOps/maria-modules-test

require (
	gitlab.com/Beamery/gozzlightyear v1.0.0
	gitlab.com/maria.campana/sounds v1.0.0
	gitlab.com/Beamery/DevOps/gotalker v1.0.0
)

I see I try to import server.com/Company/Team/Repo and it will try to clone server.com/Company/Team and look for the tag version Repo/v1.0.0

root@670ea5e97a56:/app# go build
go: finding gitlab.com/Beamery/DevOps/gotalker v1.0.0
go: gitlab.com/Beamery/DevOps/gotalker@v1.0.0: unknown revision gotalker/v1.0.0
go: error loading module requirements

@Marvalero Marvalero changed the title Go modules do not work with Gitlab Path format Go modules do not work with Gitlab.com/Company/Team/Repo Path format Dec 4, 2018

@agnivade

This comment has been minimized.

Copy link
Member

agnivade commented Dec 5, 2018

I think you have to add a go.mod to the gotalker repo. Without that, the module path does not match.

Roping in @myitcv for a deeper look.

@agnivade agnivade changed the title Go modules do not work with Gitlab.com/Company/Team/Repo Path format cmd/go: modules do not work with Gitlab.com/Company/Team/Repo Path format Dec 5, 2018

@Marvalero

This comment has been minimized.

Copy link
Author

Marvalero commented Dec 5, 2018

I tried adding a go.mod to my "gitlab.com/Beamery/DevOps/gotalker" library but I cannot fetch it from my other repo. Thanks for looking into it tho 😄

@bcmills bcmills added this to the Go1.12 milestone Dec 5, 2018

@JAicewizard

This comment has been minimized.

Copy link

JAicewizard commented Dec 6, 2018

https://gitlab.com/antipy/antibuild/api works. maybe see if there is any diferences (or check if you can import that?)

@myitcv

This comment has been minimized.

Copy link
Member

myitcv commented Dec 6, 2018

@Marvalero - please include the output from go build -x -v. Although from first glance it appears to suggest that gitlab.com/Beamery/DevOps/gotalker has not been tagged with v1.0.0.

@Marvalero

This comment has been minimized.

Copy link
Author

Marvalero commented Dec 10, 2018

I created gotalker, gozzlightyear and sounds and they all have a file with a Run() function and tags. If there was something missing in gotalker, it would be missing in the other 2 repos.

This is the output of go build -x -v (Also: thanks! nice command)

root@fd4caec26a46:/app# go build -x -v 
WORK=/tmp/go-build291933567
Fetching https://gitlab.com/Beamery/DevOps/gotalker?go-get=1
Parsing meta tags from https://gitlab.com/Beamery/DevOps/gotalker?go-get=1 (status code 200)
get "gitlab.com/Beamery/DevOps/gotalker": found meta tag get.metaImport{Prefix:"gitlab.com/Beamery/DevOps", VCS:"git", RepoRoot:"https://gitlab.com/Beamery/DevOps.git"} at https://gitlab.com/Beamery/DevOps/gotalker?go-get=1
get "gitlab.com/Beamery/DevOps/gotalker": verifying non-authoritative meta tag
Fetching https://gitlab.com/Beamery/DevOps?go-get=1
Parsing meta tags from https://gitlab.com/Beamery/DevOps?go-get=1 (status code 200)
# /go/pkg/mod/cache/vcs/0001a1afefa10f4157c4164eafd56dab52e8d34658da16e33602533f22c91849 for git2 https://gitlab.com/Beamery/DevOps.git
cd /go/pkg/mod/cache/vcs/0001a1afefa10f4157c4164eafd56dab52e8d34658da16e33602533f22c91849; git ls-remote -q https://gitlab.com/Beamery/DevOps.git
2.255s # cd /go/pkg/mod/cache/vcs/0001a1afefa10f4157c4164eafd56dab52e8d34658da16e33602533f22c91849; git ls-remote -q https://gitlab.com/Beamery/DevOps.git
Fetching https://gitlab.com/Beamery/DevOps?go-get=1
Parsing meta tags from https://gitlab.com/Beamery/DevOps?go-get=1 (status code 200)
get "gitlab.com/Beamery/DevOps": found meta tag get.metaImport{Prefix:"gitlab.com/Beamery/DevOps", VCS:"git", RepoRoot:"https://gitlab.com/Beamery/DevOps.git"} at https://gitlab.com/Beamery/DevOps?go-get=1
Fetching https://gitlab.com/Beamery?go-get=1
Parsing meta tags from https://gitlab.com/Beamery?go-get=1 (status code 200)
Fetching https://gitlab.com?go-get=1
Parsing meta tags from https://gitlab.com?go-get=1 (status code 200)
build gitlab.com/Beamery/DevOps/maria-modules-test: cannot find module for path gitlab.com/Beamery/DevOps/gotalker
@Marvalero

This comment has been minimized.

Copy link
Author

Marvalero commented Dec 10, 2018

Also, just in case you had doubts:

  1. Beamery/Devops repo does not exists
root@fd4caec26a46:/app# git clone https://gitlab.com/Beamery/DevOps.git
Cloning into 'DevOps'...
GitLab: The project you were looking for could not be found.
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.
  1. Gotalker exists (but it is private):
root@fd4caec26a46:/# git clone https://gitlab.com/Beamery/DevOps/gotalker.git
Cloning into 'gotalker'...
remote: Enumerating objects: 11, done.
remote: Counting objects: 100% (11/11), done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 11 (delta 1), reused 0 (delta 0)
Receiving objects: 100% (11/11), done.
Resolving deltas: 100% (1/1), done.

And it has many tags (I created them to see if I could fix this import problem):

root@fd4caec26a46:/gotalker# git tag --list
gotalker/v1.0.0
gotalker/v1.0.1
v0.4.0
v1.0.0
v1.0.1

@Marvalero Marvalero closed this Dec 12, 2018

@Marvalero Marvalero reopened this Dec 12, 2018

@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Dec 12, 2018

In general it is up to the server to tell the go tool where to find a given repository. We have a few hosting services hard-coded, but gitlab.com is not one of them.

The protocol for other services uses an HTTPS request to resolve the repo root, and is documented here. So in this case the go command will issue a request for https://gitlab.com/Beamery/DevOps/gotalker?go-get=1.

When I do that (using curl https://gitlab.com/Beamery/DevOps/gotalker?go-get=1), I get the following response:

<html><head><meta name="go-import" content="gitlab.com/Beamery/DevOps git https://gitlab.com/Beamery/DevOps.git" /></head></html>

If I understand your description correctly, the actual repo location should instead be https://gitlab.com/Beamery/DevOps/gotalker.git.

So it seems that the issue here is in @gitlabhq's server, and you'll need to escalate to them.

@JAicewizard

This comment has been minimized.

Copy link

JAicewizard commented Jan 29, 2019

gitlab is not willing to propperly support the protocol for repo privacy reasons, would it be an option to add a hardcoded solution for gitlab with a special indicator indicating the repo(and the rest the path from the root) or something like that?
the fact that gitlab doesnt work propperly with go is frustating a lot of people and pretty much marked as no-fix on gitlabs end(privacy reasons)

@bcmills

This comment has been minimized.

Copy link
Member

bcmills commented Jan 29, 2019

Sorry, but “privacy reasons” — without any sort of detail as to what those reasons are — is not a sufficient justification for hard-coding provider details into the go command.

We do need some sort of general authentication support for HTTPS fetches, but that's already tracked in #26232. Is there reason to believe that that won't work for GitLab?

@JAicewizard

This comment has been minimized.

Copy link

JAicewizard commented Feb 1, 2019

the issue with gitlab is documented here(https://gitlab.com/gitlab-org/gitlab-ce/issues/1337)
this is indeed discussed in #26232 and if that would come to live it could difinitly fix the issue.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.
You signed in with another tab or window. Reload to refresh your session. You signed out in another tab or window. Reload to refresh your session.