Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: mod download swallows BitBucket git errors when VCS lookup fails #47311

Open
tomkcook opened this issue Jul 21, 2021 · 2 comments
Open

cmd/go: mod download swallows BitBucket git errors when VCS lookup fails #47311

tomkcook opened this issue Jul 21, 2021 · 2 comments

Comments

@tomkcook
Copy link

@tomkcook tomkcook commented Jul 21, 2021

What version of Go are you using (go version)?

$ go version
go version go1.16.5 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOOS="linux"
GOPRIVATE=bitbucket.org

The OS is Ubuntu 18.04.

What did you do?

go mod download -x

What did you expect to see?

I expected this to download the modules specified in go.mod.

What did you see instead?

cd .
git ls-remote https://bitbucket.org/veea/golog
go: missing Mercurial command. See https://golang.org/s/gogetcmd
go: bitbucket.org/veea/golog@v2.0.0+incompatible: reading https://api.bitbucket.org/2.0/repositories/veea/golog?fields=scm: 403 Forbidden
	server response: Access denied. You must have write or admin access.

More configuration info

I've also set git config --global url."git@bitbucket.org:".insteadOf "https://bitbucket.org"

Cause

The underlying problem here is a git problem, not a go problem and I will report it on git, but the way go handles the git failure is very unhelpful.

The git failure happens because a git submodule is mapped using an overlay mount so that .git specifies a gitdir which doesn't exist and this submodule directory is the cwd. Under these conditions, any git operation fails, even if it is one that doesn't require you to be in a git repository at the time. For instance, the failure of git ls-remote mentioned in the output above is actually:

$ git ls-remote https://bitbucket.org/veea/golog
fatal: not a git repository: /home/tkcook/git/demo/VHP-3375-veeadb-on-vhe09/build/iesv10/programs/veeadb/../../../../../../.git/worktrees/VHP-3375-veeadb-on-vhe09/modules/node/platform/ies-os/programs/veeadb

Note that the same command succeeds if the current working directory is $HOME:

$ cd ~
$ git ls-remote https://bitbucket.org/veea/golog
18587b1fa99c724154ca312c6a3ca001bfb63f1e	HEAD

But, as noted above, this failure is reported by go mod download as:

go: bitbucket.org/veea/golog@v2.0.0+incompatible: reading https://api.bitbucket.org/2.0/repositories/veea/golog?fields=scm: 403 Forbidden
	server response: Access denied. You must have write or admin access.

This makes it appear as though the GOPRIVATE setting is being ignored.

@seankhliao seankhliao changed the title go mod download swallows git errors and mis-reports them cmd/go: mod download swallows git errors and mis-reports them Jul 21, 2021
@seankhliao
Copy link
Contributor

@seankhliao seankhliao commented Jul 21, 2021

@bcmills
Copy link
Member

@bcmills bcmills commented Jul 21, 2021

The failing URL comes from here:

url := &urlpkg.URL{
Scheme: "https",
Host: "api.bitbucket.org",
Path: expand(match, "/2.0/repositories/{bitname}"),
RawQuery: "fields=scm",
}

Fetching that URL does indeed fail with the error reported by the go command:

~/tmp$ curl -i https://api.bitbucket.org/2.0/repositories/veea/golog?fields=scm
HTTP/2 403
server: nginx
vary: Authorization, Origin
cache-control: max-age=900
content-type: text/plain
x-b3-traceid: f8d0d5e9089234da
x-usage-output-ops: 0
x-dc-location: Micros
strict-transport-security: max-age=31536000; includeSubDomains; preload
date: Wed, 21 Jul 2021 18:20:22 GMT
x-usage-user-time: 0.023356
x-usage-system-time: 0.000000
x-served-by: 7e942f486f54
x-view-name: bitbucket.apps.repo2.api.v20.repo.RepositoryHandler
etag: "b81e7e7393783ab8624c6b051083dce8"
x-static-version: 05d7ee4287f2
x-render-time: 0.0276589393616
x-accepted-oauth-scopes: repository
x-usage-input-ops: 0
x-version: 05d7ee4287f2
x-request-count: 1163
x-frame-options: SAMEORIGIN
x-cache-info: caching
content-length: 51

Access denied. You must have write or admin access.

If it fails with a 403 (which is the case here), the go command falls back to implicitly trying git and hg individually:

// this may be a private repository. If so, attempt to determine which
// VCS it uses. See issue 5375.
root := match["root"]
for _, vcs := range []string{"git", "hg"} {
if vcsByCmd(vcs).Ping("https", root) == nil {
resp.SCM = vcs
break
}
}

Here the git attempt also fails (because of the aforementioned Git bug), and the hg attempt fails because BitBucket doesn't do Mecurial any more.

So the go command has three possible errors to report, and it (arbitrarily) chooses the first one. It's not obvious to me that either of the others would help, but given that BitBucket doesn't even serve Mercurial repos any more perhaps we should tear out all of this source-control probing and just assume git, and report only the git error if it fails.

@bcmills bcmills changed the title cmd/go: mod download swallows git errors and mis-reports them cmd/go: mod download swallows BitBucket git errors when VCS lookup fails Jul 22, 2021
@bcmills bcmills added this to the Backlog milestone Aug 10, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
3 participants