Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

proxy.golang.org: '400 bad request' for pseudo-versions that refer to commits that also have canonical version tags #32879

Open
it-philpennock opened this issue Jul 1, 2019 · 13 comments

Comments

Projects
None yet
5 participants
@it-philpennock
Copy link

commented Jul 1, 2019

What version of Go are you using (go version)?

$ go version
go version go1.13beta1 linux/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOHOSTARCH="amd64"
GOHOSTOS="linux"

Ubuntu Disco Dingo, go1.13beta1 downloaded from the Golang site.

What did you do?

go get -d github.com/future-architect/vuls
cd ~/go/src/github.com/future-architect/vuls
  # see that it has a `go.mod` already, so ..
go install

What did you expect to see?

Either a successful build or some kind of build failure

What did you see instead?

A bad request from proxy.golang.org:

go: github.com/tomoyamachi/reg@v0.16.2-0.20190418055600-c6010b917a55: reading https://proxy.golang.org/github.com/tomoyamachi/reg/@v/v0.16.2-0.20190418055600-c6010b917a55.mod: 400 Bad Request

I repeated the command, to rule out transient glitch. I then ran GOPROXY=direct go install which continued past that point to get to a different inscrutable error but one which is more likely tied to the project itself. Limiting this issue to the observed proxy error.

@bcmills

This comment has been minimized.

Copy link
Member

commented Jul 1, 2019

to get to a different inscrutable error but one which is more likely tied to the project itself

That suggests that the proxy probably should have instead returned a 404 or 410 to prompt the go command to fall back to direct.

CC @hyangah @katiehockman @heschik

@bcmills bcmills added this to the Go1.13 milestone Jul 1, 2019

@heschik

This comment has been minimized.

Copy link
Contributor

commented Jul 1, 2019

If you visit the URL directly you'll see that it says "invalid version". That could be improved, but I don't know how much we should invest in it until #30748 is fixed; odds are nobody will see it anyway.

@heschik

This comment has been minimized.

Copy link
Contributor

commented Jul 1, 2019

@bcmills: https://proxy.golang.org/github.com/tomoyamachi/reg/@v/v0.16.2-0.20190418055600-c6010b917a55.info is (for better or worse) a pointer to v0.16.1. Do you know why the go command is still reading v0.16.2-0.20190418055600-c6010b917a55.mod? This seems like a variation of the wrong .zip bug that was fixed in 1.13?

@bcmills

This comment has been minimized.

Copy link
Member

commented Jul 1, 2019

@heschik, the validation rule in 1.13beta1 is that the base tag for the pseudo version must be on the commit itself or one of its ancestors, so v0.16.2-0.20190418055600-c6010b917a55.info is an allowed name for that commit.

I suppose we could tighten up that validation so that the base tag must be on some actual ancestor rather than the commit itself. I don't have a solid intuition one way or the other on that (cc @jayconrod).

@bcmills bcmills referenced this issue Jul 1, 2019

Merged

add Library Scan (with image scan) #829

6 of 9 tasks complete
@bcmills

This comment has been minimized.

Copy link
Member

commented Jul 1, 2019

Hrm, that's odd. Using go1.13beta1, I get a different result from what @it-philpennock observed:

$ GO111MODULE=on go1.13beta1 get -d github.com/tomoyamachi/reg@v0.16.2-0.20190418055600-c6010b917a55
go: finding github.com/tomoyamachi/reg v0.16.2-0.20190418055600-c6010b917a55
go: finding github.com v0.16.2-0.20190418055600-c6010b917a55
go: finding github.com/tomoyamachi v0.16.2-0.20190418055600-c6010b917a55
go get github.com/tomoyamachi/reg@v0.16.2-0.20190418055600-c6010b917a55: github.com/tomoyamachi/reg@v0.16.2-0.20190418055600-c6010b917a55: proxy returned info for version v0.16.1 instead of requested version
@it-philpennock

This comment has been minimized.

Copy link
Author

commented Jul 1, 2019

nb: I was installing github.com/future-architect/vuls and tomoyamachi/reg is a dependency of that.

@bcmills

This comment has been minimized.

Copy link
Member

commented Jul 1, 2019

Ah, here we go. I guess we try to download the .mod without bothering with the .info if it's already in the go.mod and go.sum files.

For completeness:

$ GO111MODULE=on go1.13beta1 get -d github.com/future-architect/vuls
go: finding github.com/future-architect/vuls v0.8.2
verifying github.com/genuinetools/reg@v0.16.1/go.mod: github.com/genuinetools/reg@v0.16.1/go.mod: reading https://sum.golang.org/lookup/github.com/genuinetools/reg@v0.16.1: 410 Gone

$ git clone https://github.com/future-architect/vuls
Cloning into 'vuls'...
remote: Enumerating objects: 24, done.
remote: Counting objects: 100% (24/24), done.
remote: Compressing objects: 100% (16/16), done.
remote: Total 5310 (delta 10), reused 16 (delta 8), pack-reused 5286
Receiving objects: 100% (5310/5310), 5.33 MiB | 27.58 MiB/s, done.
Resolving deltas: 100% (3679/3679), done.

$ cd vuls

vuls$ go1.13beta1 list all
go: github.com/tomoyamachi/reg@v0.16.2-0.20190418055600-c6010b917a55: reading https://proxy.golang.org/github.com/tomoyamachi/reg/@v/v0.16.2-0.20190418055600-c6010b917a55.mod: 400 Bad Request
@bcmills

This comment has been minimized.

Copy link
Member

commented Jul 1, 2019

FWIW, there are other bogus versions mentioned in that same go.mod file:

vuls$ GOPROXY=direct go1.13beta1 list all
go: github.com/kotakanbe/goval-dictionary@v0.1.3-0.20190613053258-078b163b76ec: invalid pseudo-version: does not match version-control timestamp (2019-06-13T06:46:14Z)
@gopherbot

This comment has been minimized.

Copy link

commented Jul 2, 2019

Change https://golang.org/cl/184720 mentions this issue: cmd/go: tighten the check for pseudo-version base tags

@bcmills

This comment has been minimized.

Copy link
Member

commented Jul 2, 2019

CL 184720 will tighten up the validation so that this pseudo-version is treated as actually-invalid.

However, there remains a bug in the proxy for valid pseudo-versions naming commits that have subsequently had canonical tags applied:

example.com$ go mod init example.com
go: creating new go.mod: module example.com

example.com$ go version
go version devel +fbde753a58 Tue Jul 2 13:52:21 2019 +0000 linux/amd64

example.com$ GOPROXY='' go get -d golang.org/x/text@v0.1.1-0.20171213102548-c4d099d611ac
go: finding golang.org/x/text v0.1.1-0.20171213102548-c4d099d611ac
go: finding golang.org v0.1.1-0.20171213102548-c4d099d611ac
go: finding golang.org/x v0.1.1-0.20171213102548-c4d099d611ac
go get golang.org/x/text@v0.1.1-0.20171213102548-c4d099d611ac: golang.org/x/text@v0.1.1-0.20171213102548-c4d099d611ac: proxy returned info for version v0.2.0 instead of requested version

example.com$ GOPROXY=direct go get -d golang.org/x/text@v0.1.1-0.20171213102548-c4d099d611ac
go: finding golang.org/x/text v0.1.1-0.20171213102548-c4d099d611ac
verifying golang.org/x/text@v0.1.1-0.20171213102548-c4d099d611ac/go.mod: golang.org/x/text@v0.1.1-0.20171213102548-c4d099d611ac/go.mod: reading https://sum.golang.org/lookup/golang.org/x/text@v0.1.1-0.20171213102548-c4d099d611ac: 410 Gone

example.com$ GOPROXY=direct GOSUMDB=off go get -d golang.org/x/text@v0.1.1-0.20171213102548-c4d099d61
1ac
go: downloading golang.org/x/text v0.1.1-0.20171213102548-c4d099d611ac
go: extracting golang.org/x/text v0.1.1-0.20171213102548-c4d099d611ac

@bcmills bcmills changed the title proxy.golang.org: 400 bad request github.com/tomoyamachi/reg proxy.golang.org: 400 bad request for pseudo-versions that refer to commits that also have canonical version tags Jul 2, 2019

@bcmills bcmills added the NeedsFix label Jul 2, 2019

@bcmills bcmills changed the title proxy.golang.org: 400 bad request for pseudo-versions that refer to commits that also have canonical version tags proxy.golang.org: '400 bad request' for pseudo-versions that refer to commits that also have canonical version tags Jul 2, 2019

@tomoyamachi tomoyamachi referenced this issue Jul 7, 2019

Merged

fix go module problems & update trivy version #864

3 of 3 tasks complete

@andybons andybons modified the milestones: Go1.13, Go1.14 Jul 8, 2019

@it-philpennock

This comment has been minimized.

Copy link
Author

commented Jul 8, 2019

Another instance of this error message spotted in the wild, fingers crossed on my part that you've already fixed it with the above?

In github.com/minio/mc a go build hits:

go: github.com/minio/minio@v0.0.0-20190626173654-be72609d1f8f requires
	github.com/minio/mc@v0.0.0-20190529152718-f4bb0b8850cb requires
	github.com/minio/minio@v0.0.0-20190510004154-ac3b59645e92 requires
	github.com/minio/lsync@v0.0.0-20190207022115-a4e43e3d0887: reading https://proxy.golang.org/github.com/minio/lsync/@v/v0.0.0-20190207022115-a4e43e3d0887.mod: 400 Bad Request

@bcmills bcmills modified the milestones: Go1.14, Go1.13 Jul 8, 2019

@heschik

This comment has been minimized.

Copy link
Contributor

commented Jul 8, 2019

This is the case Bryan mentioned in #32879 (comment). We don't know what we're going to do about it yet, but it's on our radar.

gopherbot pushed a commit that referenced this issue Jul 16, 2019

cmd/go: tighten the check for pseudo-version base tags
Do not allow a pseudo-version derived from a canonical tag to refer to
the same revision as the tag itself. It's unnecessary (because
canonical tags already have a total ordering) and confusing (the
pseudo-version appears to come after the tag, but actually refers to
the exact same revision).

Updates #32879
Updates #27173

Change-Id: I02befedbe89c8819bdd93e470783ce63fc813193
Reviewed-on: https://go-review.googlesource.com/c/go/+/184720
Run-TryBot: Bryan C. Mills <bcmills@google.com>
TryBot-Result: Gobot Gobot <gobot@golang.org>
Reviewed-by: Jay Conrod <jayconrod@google.com>
@bcmills

This comment has been minimized.

Copy link
Member

commented Jul 16, 2019

@it-philpennock, I just checked go get -d github.com/minio/minio@v0.0.0-20190626173654-be72609d1f8f and go build ./... within a clone of the same, and they both seem to be working now. (There have been some changes in the proxy configuration in the time since your last comment.)

Are you still observing this issue? If not, I'll close it out.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.