Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: go get fails with go modules and with private repositories without a go.mod in Gitea #34075

Closed
rutgerbrf opened this issue Sep 4, 2019 · 8 comments

Comments

@rutgerbrf
Copy link

@rutgerbrf rutgerbrf commented Sep 4, 2019

What version of Go are you using (go version)?

$ go version
go version go1.13 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/rutgerb/.cache/go-build"
GOENV="/home/rutgerb/.config/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GONOPROXY="git.profects.com"
GONOSUMDB="git.profects.com"
GOOS="linux"
GOPATH="/home/rutgerb/go"
GOPRIVATE="git.profects.com"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/lib/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/go/pkg/tool/linux_amd64"
GCCGO="gccgo"
AR="ar"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD="/home/rutgerb/go/src/git.profects.com/profects/integration-foo-srv/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build707620397=/tmp/go-build -gno-record-gcc-switches"

What did you do?

go get

What did you expect to see?

A successful retrieval of a dependency.

What did you see instead?

build git.profects.com/profects/integration-foo-srv: cannot load git.profects.com/profects/bar-srv/sdk: git ls-remote -q https://git.profects.com/profects/.git in /home/rutgerb/go/pkg/mod/cache/vcs/004fd41ef60cf213426a7c48239770da27fb967a10e0561646dba467c0fe6104: exit status 128:
        fatal: repository 'https://git.profects.com/profects/.git/' not found
@rutgerbrf rutgerbrf changed the title Error running go get with private repositories without a go.mod go get fails with go modules and with private repositories without a go.mod Sep 4, 2019
@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Sep 4, 2019

This repository appears to require authentication. Did you place appropriate credentials in your .netrc file or make them available via a credential store in your global .gitconfig?

$ curl https://git.profects.com/profects/?go-get=1
<!doctype html>
<html>
        <head>
                <meta name="go-import" content="git.profects.com/profects git https://git.profects.com/profects/.git">
                <meta name="go-source" content="git.profects.com/profects _ https://git.profects.com/profects/src/branch/master{/dir} https://git.profects.com/profects/src/branch/master{/dir}/{file}#L{line}">
        </head>
        <body>
                go get git.profects.com/profects
        </body>
</html>

$ git clone https://git.profects.com/profects/.git
Cloning into 'profects'...
Username for 'https://git.profects.com': ^C
@bcmills bcmills changed the title go get fails with go modules and with private repositories without a go.mod cmd/go: go get fails with go modules and with private repositories without a go.mod Sep 4, 2019
@rutgerbrf

This comment has been minimized.

Copy link
Author

@rutgerbrf rutgerbrf commented Sep 4, 2019

We run Gitea on a dedicated machine.
I do have my credentials configured in my .netrc, and cloning with HTTPS has always worked fine.

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Sep 4, 2019

There isn't much we can do without steps to reproduce the problem.

I suggest using the -x build flag (to print intermediate commands) and then trying those commands in a clean temp directory with GIT_TERMINAL_PROMPT=0 in your environment.

The commands will probably be something like:

git init --bare
git remote add origin -- https://git.profects.com/profects/.git
git ls-remote -q origin

Please try that and report what you find.

@rutgerbrf

This comment has been minimized.

Copy link
Author

@rutgerbrf rutgerbrf commented Sep 4, 2019

I see that go get attempts to run git ls-remote -q https://git.profects.com/profects/.git, but this is not a repository that exists. Under our domain we have organizations (profects is one of them), so trying to clone the .git directory in the organization won't work.

In the error

build git.profects.com/profects/integration-foo-srv: cannot load git.profects.com/profects/bar-srv/sdk: git ls-remote -q https://git.profects.com/profects/.git in /home/rutgerb/go/pkg/mod/cache/vcs/004fd41ef60cf213426a7c48239770da27fb967a10e0561646dba467c0fe6104: exit status 128:
        fatal: repository 'https://git.profects.com/profects/.git/' not found

an attempt is done to clone into this directory, but the actual remote would be https://git.profects.com/profects/bar-srv/sdk/.git, which is what I get reported when running curl:

$ curl https://git.profects.com/profects/bar-srv\?go-get\=1  
<!doctype html>
<html>
        <head>
                <meta name="go-import" content="git.profects.com/profects/bar-srv git https://git.profects.com/profects/bar-srv.git">
                <meta name="go-source" content="git.profects.com/profects/bar-srv _ https://git.profects.com/profects/bar-srv/src/branch/master{/dir} https://git.profects.com/profects/bar-srv/src/branch/master{/dir}/{file}#L{line}">
        </head>
        <body>
                go get git.profects.com/profects/bar-srv
        </body>
</html>
@rutgerbrf

This comment has been minimized.

Copy link
Author

@rutgerbrf rutgerbrf commented Sep 4, 2019

There are other packages which we import from our private server which are resolved and retrieved correctly.

@bcmills

This comment has been minimized.

Copy link
Member

@bcmills bcmills commented Sep 4, 2019

If git.profects.com/profects/.git is not a repository that exists, then you should update the server at https://git.profects.com/profects so that it does not return a go-import <meta> tag referring to such a repository.

(In module mode in Go 1.13, the go command checks all possible module paths in parallel for the requested package. Your server is telling the go command that a module exists at https://git.profects.com/profects/.git, and the go command is erroring out because it can't fetch that module to determine whether it contains the requested package. The solution is to not tell the go command to look there in the first place.)

@rutgerbrf

This comment has been minimized.

Copy link
Author

@rutgerbrf rutgerbrf commented Sep 4, 2019

I indeed see this happening with go get -x:

go get -x
(...)
# get //git.profects.com/profects?go-get=1: 200 OK (0.312s)
# get //git.profects.com/?go-get=1: 200 OK (0.315s)
mkdir -p /home/rutgerb/go/pkg/mod/cache/vcs # git3 https://git.profects.com//.git
# lock /home/rutgerb/go/pkg/mod/cache/vcs/f2b15ea9a55d983be763a4c15193e816589f01cc189c95ab8a0b8a67a28faa0e.lockmkdir -p /home/rutgerb/go/pkg/mod/cache/vcs # git3 https://git.profects.com/profects/.git
# /home/rutgerb/go/pkg/mod/cache/vcs/f2b15ea9a55d983be763a4c15193e816589f01cc189c95ab8a0b8a67a28faa0e for git3 https://git.profects.com//.git
cd /home/rutgerb/go/pkg/mod/cache/vcs/f2b15ea9a55d983be763a4c15193e816589f01cc189c95ab8a0b8a67a28faa0e; git ls-remote -q https://git.profects.com//.git
# lock /home/rutgerb/go/pkg/mod/cache/vcs/004fd41ef60cf213426a7c48239770da27fb967a10e0561646dba467c0fe6104.lock# /home/rutgerb/go/pkg/mod/cache/vcs/004fd41ef60cf213426a7c48239770da27fb967a10e0561646dba467c0fe6104 for git3 https://git.profects.com/profects/.git
cd /home/rutgerb/go/pkg/mod/cache/vcs/004fd41ef60cf213426a7c48239770da27fb967a10e0561646dba467c0fe6104; git ls-remote -q https://git.profects.com/profects/.git

So I suppose it is time to open an issue in Gitea.

@rutgerbrf rutgerbrf closed this Sep 4, 2019
@thepudds thepudds changed the title cmd/go: go get fails with go modules and with private repositories without a go.mod cmd/go: go get fails with go modules and with private repositories without a go.mod in Gitea Sep 4, 2019
@gopherbot

This comment has been minimized.

Copy link

@gopherbot gopherbot commented Sep 4, 2019

Change https://golang.org/cl/193262 mentions this issue: cmd/go/internal/web: log complete URLs

@rutgerbrf rutgerbrf mentioned this issue Sep 5, 2019
2 of 7 tasks complete
gopherbot pushed a commit that referenced this issue Sep 11, 2019
Incoming URLs may omit the scheme to indicate “either HTTP or HTTPS”.
For such URLs, log the scheme actually used instead of leaving it out.

(This issue was noticed while triaging #34075.)

Updates #34075

Change-Id: I39e5ca83543dd780258d41d5c2c4ba907cd20e5c
Reviewed-on: https://go-review.googlesource.com/c/go/+/193262
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
3 participants
You can’t perform that action at this time.