Latest git does not follow redirects by default, breaking gopkg.in #50

Closed
mikedanese opened this Issue Jan 9, 2017 · 21 comments

Comments

Projects
None yet
@mikedanese

See kubernetes/kubernetes#39583 for an example of this failure

@mikedanese mikedanese referenced this issue in kubernetes/kubernetes Jan 9, 2017

Closed

hack/verify-bazel.sh failing with HTTP error 301 #39583

@niemeyer

This comment has been minimized.

Show comment
Hide comment
@niemeyer

niemeyer Jan 10, 2017

Owner

Thanks for filing it. We'll need to change the gopkg.in implementation a bit to adapt to this.

Owner

niemeyer commented Jan 10, 2017

Thanks for filing it. We'll need to change the gopkg.in implementation a bit to adapt to this.

@ixdy

This comment has been minimized.

Show comment
Hide comment
@ixdy

ixdy Jan 17, 2017

FYI, my workaround for the time being was to run

$ git config --global http.https://gopkg.in.followRedirects true

ixdy commented Jan 17, 2017

FYI, my workaround for the time being was to run

$ git config --global http.https://gopkg.in.followRedirects true
@alecthomas

This comment has been minimized.

Show comment
Hide comment
@alecthomas

alecthomas Jan 28, 2017

Any way to fix this on the gopkg.in server?

Any way to fix this on the gopkg.in server?

@broady

This comment has been minimized.

Show comment
Hide comment
@broady

broady Feb 4, 2017

@niemeyer are you willing to proxy the contents of git-upload-pack?

broady commented Feb 4, 2017

@niemeyer are you willing to proxy the contents of git-upload-pack?

@kyoh86

This comment has been minimized.

Show comment
Hide comment
@kyoh86

kyoh86 Feb 6, 2017

@niemeyer are you willing to fix it???

kyoh86 commented Feb 6, 2017

@niemeyer are you willing to fix it???

@ajlanghorn

This comment has been minimized.

Show comment
Hide comment
@ajlanghorn

ajlanghorn Feb 9, 2017

Thanks for this; this helped me fix a few problems I was having installed dependencies. Now to find the fix to the rest. Would be awesome if Git would just follow 301s by default, though.

Thanks for this; this helped me fix a few problems I was having installed dependencies. Now to find the fix to the rest. Would be awesome if Git would just follow 301s by default, though.

@niemeyer

This comment has been minimized.

Show comment
Hide comment
@niemeyer

niemeyer Feb 9, 2017

Owner

Yeah, I'll fix it. We'll need to proxy the remaining of the request through the gopkg.in server as well so git doesn't complain about it. Unfortunate, as it will increase the bandwidth requirements, but can't see another workaround given the latest git changes.

Owner

niemeyer commented Feb 9, 2017

Yeah, I'll fix it. We'll need to proxy the remaining of the request through the gopkg.in server as well so git doesn't complain about it. Unfortunate, as it will increase the bandwidth requirements, but can't see another workaround given the latest git changes.

@niemeyer

This comment has been minimized.

Show comment
Hide comment
@niemeyer

niemeyer Feb 9, 2017

Owner

@ajlanghorn It did for several years. The very latest git decided to break that.

Owner

niemeyer commented Feb 9, 2017

@ajlanghorn It did for several years. The very latest git decided to break that.

@netroby

This comment has been minimized.

Show comment
Hide comment
@netroby

netroby Feb 10, 2017

please fix it . all packages require gopkg.in broken

netroby commented Feb 10, 2017

please fix it . all packages require gopkg.in broken

@ixdy

This comment has been minimized.

Show comment
Hide comment
@ixdy

ixdy Feb 10, 2017

git/git@50d3413 contains an explanation of why this change was implemented in git.

ixdy commented Feb 10, 2017

git/git@50d3413 contains an explanation of why this change was implemented in git.

@caiofilipini caiofilipini referenced this issue in fatih/vim-go Feb 10, 2017

Closed

GoMetaLinter Won't Install #698

@alecthomas alecthomas referenced this issue in alecthomas/gometalinter Feb 12, 2017

Closed

Cannot install, remote hung up unexpectedly #235

alecthomas added a commit to alecthomas/gometalinter that referenced this issue Feb 12, 2017

@niemeyer

This comment has been minimized.

Show comment
Hide comment
@niemeyer

niemeyer Feb 12, 2017

Owner

I have deployed a modified server version at https://p3.gopkg.in.. it would be very helpful if people here affected by this issue could give it a try and let me know whether it works properly for you and your project, and what's the timing impact compared to https://gopkg.in.

Thanks for your help.

Owner

niemeyer commented Feb 12, 2017

I have deployed a modified server version at https://p3.gopkg.in.. it would be very helpful if people here affected by this issue could give it a try and let me know whether it works properly for you and your project, and what's the timing impact compared to https://gopkg.in.

Thanks for your help.

@niemeyer

This comment has been minimized.

Show comment
Hide comment
@niemeyer

niemeyer Feb 12, 2017

Owner

Note you can test this with git directly, without involving the go tool:

$ git clone https://p3.gopkg.in/<your project>
Owner

niemeyer commented Feb 12, 2017

Note you can test this with git directly, without involving the go tool:

$ git clone https://p3.gopkg.in/<your project>
@alecthomas

This comment has been minimized.

Show comment
Hide comment
@alecthomas

alecthomas Feb 12, 2017

Thanks @niemeyer, seems to work fine for me. Oddly, this seems faster than the original: roughly 2.5 seconds vs. 4.5 seconds. Could be load related on the server I guess? The repo's seem to be basically identical.

[aat@cavern:~/Projects/foo]time git clone https://gopkg.in/alecthomas/kingpin.v3-unstable old
Cloning into 'old'...
remote: Counting objects: 1353, done.
remote: Total 1353 (delta 0), reused 0 (delta 0), pack-reused 1353
Receiving objects: 100% (1353/1353), 403.51 KiB | 296.00 KiB/s, done.
Resolving deltas: 100% (902/902), done.
git clone https://gopkg.in/alecthomas/kingpin.v3-unstable old  0.08s user 0.07s system 3% cpu 4.721 total
[aat@cavern:~/Projects/foo]unset git
[aat@cavern:~/Projects/foo]time git clone https://p3.gopkg.in/alecthomas/kingpin.v3-unstable new
Cloning into 'new'...
remote: Counting objects: 1353, done.
remote: Total 1353 (delta 0), reused 0 (delta 0), pack-reused 1353
Receiving objects: 100% (1353/1353), 403.51 KiB | 164.00 KiB/s, done.
Resolving deltas: 100% (902/902), done.
git clone https://p3.gopkg.in/alecthomas/kingpin.v3-unstable new  0.08s user 0.07s system 5% cpu 2.589 total
[aat@cavern:~/Projects/foo]rm -rf *
[aat@cavern:~/Projects/foo]time git clone https://p3.gopkg.in/alecthomas/kingpin.v3-unstable new
Cloning into 'new'...
remote: Counting objects: 1353, done.
remote: Total 1353 (delta 0), reused 0 (delta 0), pack-reused 1353
Receiving objects: 100% (1353/1353), 403.51 KiB | 214.00 KiB/s, done.
Resolving deltas: 100% (902/902), done.
git clone https://p3.gopkg.in/alecthomas/kingpin.v3-unstable new  0.07s user 0.06s system 6% cpu 2.117 total
[aat@cavern:~/Projects/foo]time git clone https://gopkg.in/alecthomas/kingpin.v3-unstable old
Cloning into 'old'...
remote: Counting objects: 1353, done.
remote: Total 1353 (delta 0), reused 0 (delta 0), pack-reused 1353
Receiving objects: 100% (1353/1353), 403.51 KiB | 198.00 KiB/s, done.
Resolving deltas: 100% (902/902), done.
git clone https://gopkg.in/alecthomas/kingpin.v3-unstable old  0.08s user 0.07s system 3% cpu 4.512 total

Thanks @niemeyer, seems to work fine for me. Oddly, this seems faster than the original: roughly 2.5 seconds vs. 4.5 seconds. Could be load related on the server I guess? The repo's seem to be basically identical.

[aat@cavern:~/Projects/foo]time git clone https://gopkg.in/alecthomas/kingpin.v3-unstable old
Cloning into 'old'...
remote: Counting objects: 1353, done.
remote: Total 1353 (delta 0), reused 0 (delta 0), pack-reused 1353
Receiving objects: 100% (1353/1353), 403.51 KiB | 296.00 KiB/s, done.
Resolving deltas: 100% (902/902), done.
git clone https://gopkg.in/alecthomas/kingpin.v3-unstable old  0.08s user 0.07s system 3% cpu 4.721 total
[aat@cavern:~/Projects/foo]unset git
[aat@cavern:~/Projects/foo]time git clone https://p3.gopkg.in/alecthomas/kingpin.v3-unstable new
Cloning into 'new'...
remote: Counting objects: 1353, done.
remote: Total 1353 (delta 0), reused 0 (delta 0), pack-reused 1353
Receiving objects: 100% (1353/1353), 403.51 KiB | 164.00 KiB/s, done.
Resolving deltas: 100% (902/902), done.
git clone https://p3.gopkg.in/alecthomas/kingpin.v3-unstable new  0.08s user 0.07s system 5% cpu 2.589 total
[aat@cavern:~/Projects/foo]rm -rf *
[aat@cavern:~/Projects/foo]time git clone https://p3.gopkg.in/alecthomas/kingpin.v3-unstable new
Cloning into 'new'...
remote: Counting objects: 1353, done.
remote: Total 1353 (delta 0), reused 0 (delta 0), pack-reused 1353
Receiving objects: 100% (1353/1353), 403.51 KiB | 214.00 KiB/s, done.
Resolving deltas: 100% (902/902), done.
git clone https://p3.gopkg.in/alecthomas/kingpin.v3-unstable new  0.07s user 0.06s system 6% cpu 2.117 total
[aat@cavern:~/Projects/foo]time git clone https://gopkg.in/alecthomas/kingpin.v3-unstable old
Cloning into 'old'...
remote: Counting objects: 1353, done.
remote: Total 1353 (delta 0), reused 0 (delta 0), pack-reused 1353
Receiving objects: 100% (1353/1353), 403.51 KiB | 198.00 KiB/s, done.
Resolving deltas: 100% (902/902), done.
git clone https://gopkg.in/alecthomas/kingpin.v3-unstable old  0.08s user 0.07s system 3% cpu 4.512 total
@alecthomas

This comment has been minimized.

Show comment
Hide comment
@alecthomas

alecthomas Feb 12, 2017

Note that go get does not seem to work. I'm not sure if this is intentional at this stage.

[aat@cavern:~/Projects/foo]go get -u -v p3.gopkg.in/alecthomas/kingpin.v3-unstable
Fetching https://p3.gopkg.in/alecthomas/kingpin.v3-unstable?go-get=1
Parsing meta tags from https://p3.gopkg.in/alecthomas/kingpin.v3-unstable?go-get=1 (status code 200)
package p3.gopkg.in/alecthomas/kingpin.v3-unstable: unrecognized import path "p3.gopkg.in/alecthomas/kingpin.v3-unstable" (parse https://p3.gopkg.in/alecthomas/kingpin.v3-unstable?go-get=1: no go-import meta tags)

Note that go get does not seem to work. I'm not sure if this is intentional at this stage.

[aat@cavern:~/Projects/foo]go get -u -v p3.gopkg.in/alecthomas/kingpin.v3-unstable
Fetching https://p3.gopkg.in/alecthomas/kingpin.v3-unstable?go-get=1
Parsing meta tags from https://p3.gopkg.in/alecthomas/kingpin.v3-unstable?go-get=1 (status code 200)
package p3.gopkg.in/alecthomas/kingpin.v3-unstable: unrecognized import path "p3.gopkg.in/alecthomas/kingpin.v3-unstable" (parse https://p3.gopkg.in/alecthomas/kingpin.v3-unstable?go-get=1: no go-import meta tags)
@niemeyer

This comment has been minimized.

Show comment
Hide comment
@niemeyer

niemeyer Feb 12, 2017

Owner

@alecthomas Thanks for the test. The difference is most likely related to networking connectivity rather than load at this instance, although in the future once we convert gopkg.in itself to that new logic load will definitely be a concern.

For the latter issue, indeed go get won't work out of gopkg.in because the metadata still points at gopkg.in itself.

Owner

niemeyer commented Feb 12, 2017

@alecthomas Thanks for the test. The difference is most likely related to networking connectivity rather than load at this instance, although in the future once we convert gopkg.in itself to that new logic load will definitely be a concern.

For the latter issue, indeed go get won't work out of gopkg.in because the metadata still points at gopkg.in itself.

@johanbrandhorst

This comment has been minimized.

Show comment
Hide comment
@johanbrandhorst

johanbrandhorst Feb 15, 2017

What is the status of this? Is there a roadmap for further deployment?

What is the status of this? Is there a roadmap for further deployment?

@niemeyer

This comment has been minimized.

Show comment
Hide comment
@niemeyer

niemeyer Feb 15, 2017

Owner

Yes, this week a percentage of repositories will start using the new logic. We'll then gradually increase until everything is behind it.

Owner

niemeyer commented Feb 15, 2017

Yes, this week a percentage of repositories will start using the new logic. We'll then gradually increase until everything is behind it.

@eyalpost eyalpost referenced this issue in go-redis/redis Feb 16, 2017

Closed

Go get throws error #497

@wvierber wvierber referenced this issue in gin-gonic/gin Feb 16, 2017

Closed

Download and install error #804

@ilbambino ilbambino referenced this issue in honeycombio/logspout-honeycomb Feb 17, 2017

Closed

building the docker image doesn't work #6

@xiam xiam referenced this issue in upper/db Feb 18, 2017

Closed

Unable to install #347

@niemeyer

This comment has been minimized.

Show comment
Hide comment
@niemeyer

niemeyer Feb 21, 2017

Owner

HEADS UP: I've just switched the DNS to a server addressing the redirect issue in the latest git.

Please shout on problems!

Owner

niemeyer commented Feb 21, 2017

HEADS UP: I've just switched the DNS to a server addressing the redirect issue in the latest git.

Please shout on problems!

@niemeyer

This comment has been minimized.

Show comment
Hide comment
@niemeyer

niemeyer Feb 21, 2017

Owner

No reported issues and 10GB/h flowing through the system. I'm considering this issue sorted for the time being.

Owner

niemeyer commented Feb 21, 2017

No reported issues and 10GB/h flowing through the system. I'm considering this issue sorted for the time being.

@niemeyer niemeyer closed this Feb 21, 2017

@tmm1 tmm1 referenced this issue in gin-contrib/gzip Feb 23, 2017

Merged

Use gopkg.in #1

@mblair mblair referenced this issue in src-d/proteus Feb 27, 2017

Closed

master doesn't build #58

@vimalk78

This comment has been minimized.

Show comment
Hide comment
@vimalk78

vimalk78 Mar 20, 2017

not able to do
go get -v gopkg.in/kataras/iris.v6

added the gitconfig
git config --global http.https://gopkg.in.followRedirects true

getting package gopkg.in/kataras/iris.v6: unrecognized import path "gopkg.in/kataras/iris.v6"

is there some issue

vimalk78 commented Mar 20, 2017

not able to do
go get -v gopkg.in/kataras/iris.v6

added the gitconfig
git config --global http.https://gopkg.in.followRedirects true

getting package gopkg.in/kataras/iris.v6: unrecognized import path "gopkg.in/kataras/iris.v6"

is there some issue

@jpwilliams

This comment has been minimized.

Show comment
Hide comment
@jpwilliams

jpwilliams Mar 20, 2017

Working okay for me, @vimalk78.

Might well be an issue with your $GOROOT/$GOPATH, but either way I'm relatively confident it's not to do with this issue.

Working okay for me, @vimalk78.

Might well be an issue with your $GOROOT/$GOPATH, but either way I'm relatively confident it's not to do with this issue.

Konboi added a commit to Konboi/dotfiles that referenced this issue Mar 21, 2017

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment