Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

cmd/go: retry failed fetches #28194

Open
chai2010 opened this issue Oct 14, 2018 · 12 comments

Comments

@chai2010
Copy link
Contributor

commented Oct 14, 2018

$ go version
go version go1.11.1 darwin/amd64
package main

import (
	_ "github.com/google/go-cloud/wire/cmd/wire"
)

func main() {
	//
}
go build -o a.out
go: finding github.com/google/go-cloud/wire/cmd/wire latest
go: finding github.com/google/go-cloud/wire/cmd latest
go: finding github.com/google/go-cloud/wire latest
go: finding google.golang.org/api v0.0.0-20180606215403-8e9de5a6de6d
go: google.golang.org/api@v0.0.0-20180606215403-8e9de5a6de6d: git fetch -f https://code.googlesource.com/google-api-go-client refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /Users/chai/go/pkg/mod/cache/vcs/9e62a95b0409d58bc0130bae299bdffbc7b7e74f3abe1ecf897474cc474b8bc0: exit status 128:
        error: RPC failed; curl 18 transfer closed with outstanding read data remaining
        fatal: The remote end hung up unexpectedly
        fatal: early EOF
        fatal: index-pack failed
go: error loading module requirements
@mvdan

This comment has been minimized.

Copy link
Member

commented Oct 15, 2018

@chai2010 it looks like git had an issue fetching data from the remote repository. Have you tried running the same command again?

@bcmills perhaps we should try to capture network flakiness errors from HTTP/VCS and report a more user friendly error, such as "perhaps try again in a few minutes".

@mvdan mvdan changed the title module: build failed cmd/go: module build failed due to 'git fetch' error Oct 15, 2018

@chai2010

This comment has been minimized.

Copy link
Contributor Author

commented Oct 17, 2018

$ HTTPS_PROXY=socks5://127.0.0.1:2080 go build -o a.out
go: finding github.com/google/go-cloud/wire/cmd/wire latest
go: finding github.com/google/go-cloud/wire/cmd latest
go: finding github.com/google/go-cloud/wire latest
go: finding google.golang.org/api v0.0.0-20180606215403-8e9de5a6de6d
go: google.golang.org/api@v0.0.0-20180606215403-8e9de5a6de6d: git fetch -f https://code.googlesource.com/google-api-go-client refs/heads/*:refs/heads/* refs/tags/*:refs/tags/* in /Users/chai/go/pkg/mod/cache/vcs/9e62a95b0409d58bc0130bae299bdffbc7b7e74f3abe1ecf897474cc474b8bc0: exit status 128:
        error: RPC failed; curl 18 transfer closed with outstanding read data remaining
        fatal: The remote end hung up unexpectedly
        fatal: early EOF
        fatal: index-pack failed
go: error loading module requirements
@thepudds

This comment has been minimized.

Copy link

commented Oct 21, 2018

@gopherbot, please add label modules

@gopherbot gopherbot added the modules label Oct 21, 2018

@mrxliu

This comment has been minimized.

Copy link

commented Dec 19, 2018

感觉是并行下载外加网速不好导致的.

@mvdan

This comment has been minimized.

Copy link
Member

commented Dec 19, 2018

Please write in English.

@agnivade

This comment has been minimized.

Copy link
Member

commented Dec 19, 2018

In any case, this is indeed a network issue. I was able to reproduce this once out of 3 tries by just running the plain git command (git fetch -f https://code.googlesource.com/google-api-go-client ).

``` $git fetch -f https://code.googlesource.com/google-api-go-client remote: Sending approximately 87.36 MiB ... remote: Counting objects: 45, done remote: Finding sources: 100% (71244/71244) error: RPC failed; curl 56 GnuTLS recv error (-9): A TLS packet with unexpected length was received. fatal: The remote end hung up unexpectedly fatal: early EOF fatal: index-pack failed 14:13:31-agniva-~/play/gg$rm -rf .git/ 14:13:48-agniva-~/play/gg$git init Initialized empty Git repository in /home/agniva/play/gg/.git/ 14:13:53-agniva-~/play/gg$git fetch -f https://code.googlesource.com/google-api-go-client remote: Sending approximately 87.36 MiB ... remote: Counting objects: 45, done remote: Finding sources: 100% (71244/71244) remote: Total 71244 (delta 44173), reused 71141 (delta 44173) Receiving objects: 100% (71244/71244), 86.49 MiB | 999.00 KiB/s, done. Resolving deltas: 100% (44173/44173), done. From https://code.googlesource.com/google-api-go-client * branch HEAD -> FETCH_HEAD 14:15:46-agniva-~/play/gg$rm -rf .git/ 14:15:51-agniva-~/play/gg$git init Initialized empty Git repository in /home/agniva/play/gg/.git/ 14:15:52-agniva-~/play/gg$git fetch -f https://code.googlesource.com/google-api-go-client remote: Sending approximately 87.36 MiB ... remote: Counting objects: 45, done remote: Finding sources: 100% (71244/71244) remote: Total 71244 (delta 44173), reused 71141 (delta 44173) Receiving objects: 100% (71244/71244), 86.49 MiB | 1.10 MiB/s, done. Resolving deltas: 100% (44173/44173), done. From https://code.googlesource.com/google-api-go-client * branch HEAD -> FETCH_HEAD ```

@chai2010 - Could you run the git command manually as shown above and see if you get any error ? If yes, then it's a network issue. If not, then try to run go build once more and see if you get the same error.

@smallnest

This comment has been minimized.

Copy link

commented Jan 17, 2019

I have the same issue:

workspace git clone https://code.googlesource.com/google-api-go-client
Cloning into 'google-api-go-client'...
remote: Sending approximately 139.63 MiB ...
error: RPC failed; curl 18 transfer closed with outstanding read data remaining
fatal: The remote end hung up unexpectedly
fatal: early EOF
fatal: index-pack failed

I have modified the git buffer:

git config --global http.postBuffer 524288000

@bcmills bcmills changed the title cmd/go: module build failed due to 'git fetch' error cmd/go: retry failed fetches Jan 17, 2019

@bcmills bcmills added this to the Go1.13 milestone Jan 17, 2019

@bcmills bcmills self-assigned this Mar 4, 2019

@northfun

This comment has been minimized.

Copy link

commented Mar 29, 2019

@mvdan Is there any solutions, such as copy the git repository directly to the mod directory?
I always get this problem,under $GOPATH I can download the zip file from git then do some other work to solve it, but with go mod I have no idea.
Thanks~

@bcmills

This comment has been minimized.

Copy link
Member

commented Mar 29, 2019

@northfun, this issue is about retrying past temporary failures, not permanent ones.

If you are unable to download from some origin in general, you may need to set up a proxy for that module. (See https://tip.golang.org/cmd/go/#hdr-Module_proxy_protocol.)

@northfun

This comment has been minimized.

Copy link

commented Apr 1, 2019

@bcmills ,thanks for your advice~
I solved it using GOPROXY in local file system finally, it cost me some time (- -|||)

@horacimacias

This comment has been minimized.

Copy link

commented May 22, 2019

@northfun would you mind sharing a bit more information on what you did? I'm also hitting this same issue

@rsc

This comment has been minimized.

Copy link
Contributor

commented Jun 6, 2019

It's unclear how to decide what is worth retrying and what is not, especially as it interacts with GOPROXY list fallback as well as executing VCS commands (the example above is git itself failing). Let's at least move this off to Go 1.14. It may be that there's nothing to do anyway, but certainly not for Go 1.13.

@rsc rsc modified the milestones: Go1.13, Go1.14 Jun 6, 2019

@rsc rsc added the early-in-cycle label Jun 6, 2019

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.