Jenkins: Error cloning remote repo 'origin' (HTTP code 408)

[jtattermusch]

Fairly often, Jenkins fails to clone the git repo.
This results in the entire build failing, before even trying start a matrix build.
Corresponding http code seems to be 408 Request timeout

https://grpc-testing.appspot.com/job/gRPC_master/7973/console

ERROR: Error cloning remote repo 'origin'
hudson.plugins.git.GitException: Command "git -c core.askpass=true fetch --tags --progress http://github.com/grpc/grpc.git +refs/heads/*:refs/remotes/origin/*" returned status code 128:
stdout: 
stderr: error: RPC failed; result=22, HTTP code = 408
fatal: The remote end hung up unexpectedly

    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandIn(CliGitAPIImpl.java:1640)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.launchCommandWithCredentials(CliGitAPIImpl.java:1388)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl.access$300(CliGitAPIImpl.java:62)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$1.execute(CliGitAPIImpl.java:313)
    at org.jenkinsci.plugins.gitclient.CliGitAPIImpl$2.execute(CliGitAPIImpl.java:505)
    at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:152)
    at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler$1.call(RemoteGitImpl.java:145)
    at hudson.remoting.UserRequest.perform(UserRequest.java:121)
    at hudson.remoting.UserRequest.perform(UserRequest.java:49)
    at hudson.remoting.Request$2.run(Request.java:326)
    at hudson.remoting.InterceptingExecutorService$1.call(InterceptingExecutorService.java:68)
    at java.util.concurrent.FutureTask.run(FutureTask.java:262)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
    at java.lang.Thread.run(Thread.java:745)
    at ......remote call to grpc-jenkins-worker3(Native Method)
    at hudson.remoting.Channel.attachCallSiteStackTrace(Channel.java:1413)
    at hudson.remoting.UserResponse.retrieve(UserRequest.java:221)
    at hudson.remoting.Channel.call(Channel.java:778)
    at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.execute(RemoteGitImpl.java:145)
    at sun.reflect.GeneratedMethodAccessor120.invoke(Unknown Source)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.jenkinsci.plugins.gitclient.RemoteGitImpl$CommandInvocationHandler.invoke(RemoteGitImpl.java:131)
    at com.sun.proxy.$Proxy48.execute(Unknown Source)
    at hudson.plugins.git.GitSCM.retrieveChanges(GitSCM.java:1003)
    at hudson.plugins.git.GitSCM.checkout(GitSCM.java:1043)
    at hudson.scm.SCM.checkout(SCM.java:485)
    at hudson.model.AbstractProject.checkout(AbstractProject.java:1277)
    at hudson.model.AbstractBuild$AbstractBuildExecution.defaultCheckout(AbstractBuild.java:610)
    at jenkins.scm.SCMCheckoutStrategy.checkout(SCMCheckoutStrategy.java:86)
    at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:532)
    at hudson.model.Run.execute(Run.java:1741)
    at hudson.matrix.MatrixBuild.run(MatrixBuild.java:301)
    at hudson.model.ResourceController.execute(ResourceController.java:98)
    at hudson.model.Executor.run(Executor.java:408)
ERROR: null
Finished: FAILURE

[jtattermusch]

CC @ctiller @nicolasnoble we discussed that today

[jtattermusch]

https://grpc-testing.appspot.com/job/gRPC_master/7972/console

https://grpc-testing.appspot.com/job/gRPC_master/7971/console

[jtattermusch]

http://stackoverflow.com/questions/22369200/git-pull-push-error-rpc-failed-result-22-http-code-408

[jtattermusch]

This also happens for cells in our test matrix, not just for the "master" job.

https://grpc-testing.appspot.com/job/gRPC_master/config=dbg,language=csharp,platform=linux/7964/console

[jtattermusch]

Seen many times today, e.g. here:
https://grpc-testing.appspot.com/job/gRPC_master/12733/config=dbg,language=sanity,platform=linux/console

https://grpc-testing.appspot.com/job/gRPC_master/12733/config=asan,language=node,platform=linux/console

https://grpc-testing.appspot.com/job/gRPC_master/12733/config=opt,language=php,platform=linux/console

Escalating this to P0.

[ctiller]

I won't get a chance to work on this: assigning to @adelez to watch and see if it needs work.

[adelez]

I checked recent master builds and didn't see this error anymore. Ok to close?

[soltanmm]

I might just be remembering this wrong, but, wasn't this just the issue that one had to hit the 'rebuild' button post-failure to get amended commits to build properly? Some sort of upstream Jenkins bug?

[adelez]

Closing since it hasn't occurred for a while.