Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Push fails running in parallel #328

Closed
maffelbaffel opened this issue Jan 10, 2017 · 3 comments
Closed

Push fails running in parallel #328

maffelbaffel opened this issue Jan 10, 2017 · 3 comments
Assignees
@orzeh
Labels
help wanted

Comments

@maffelbaffel
Copy link

maffelbaffel commented Jan 10, 2017 •
edited

Running multiple DockerPushImage tasks in parallel fails for me.

My tasks look like this (important parts):

    def fullImageName = "$regUrl/$dockerImageName"

    task buildDockerImage(type: DockerBuildImage, group: 'docker') {
        inputDir = project.projectDir

        tag = fullImageName
    }
    task tagTagImage(type: DockerTagImage, group: 'docker') {
        dependsOn buildDockerImage
        targetImageId { buildDockerImage.imageId }
        repository = fullImageName
        tag = gitTagName
    }
    task pushTag(type: DockerPushImage, group: 'docker') {
        dependsOn tagTagImage
        imageName = fullImageName
        tag = gitTagName
    }
    task pushLatest(type: DockerPushImage, group: 'docker') {
        dependsOn buildDockerImage
        imageName = fullImageName
        tag = 'latest'
    }

Notice how pushTag and pushLatest both only depend on buildDockerImage which means after my image is built both pushTag and pushLatest run in parallel.

I am calling them with gradle pushTag pushLatest.

org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':pushLatest'.
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:84)
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:55)
	at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:62)
	at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:58)
	at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:88)
	at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:46)
	at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:51)
	at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)
	at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
	at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34)
	at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker$1.execute(DefaultTaskGraphExecuter.java:236)
	at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker$1.execute(DefaultTaskGraphExecuter.java:228)
	at org.gradle.internal.Transformers$4.transform(Transformers.java:169)
	at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:106)
	at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:61)
	at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:228)
	at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter$EventFiringTaskWorker.execute(DefaultTaskGraphExecuter.java:215)
	at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.processTask(AbstractTaskPlanExecutor.java:77)
	at org.gradle.execution.taskgraph.AbstractTaskPlanExecutor$TaskExecutorWorker.run(AbstractTaskPlanExecutor.java:58)
	at org.gradle.execution.taskgraph.DefaultTaskPlanExecutor.process(DefaultTaskPlanExecutor.java:32)
	at org.gradle.execution.taskgraph.DefaultTaskGraphExecuter.execute(DefaultTaskGraphExecuter.java:113)
	at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:37)
	at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:37)
	at org.gradle.execution.DefaultBuildExecuter.access$000(DefaultBuildExecuter.java:23)
	at org.gradle.execution.DefaultBuildExecuter$1.proceed(DefaultBuildExecuter.java:43)
	at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:32)
	at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:37)
	at org.gradle.execution.DefaultBuildExecuter.execute(DefaultBuildExecuter.java:30)
	at org.gradle.initialization.DefaultGradleLauncher$3.execute(DefaultGradleLauncher.java:196)
	at org.gradle.initialization.DefaultGradleLauncher$3.execute(DefaultGradleLauncher.java:193)
	at org.gradle.internal.Transformers$4.transform(Transformers.java:169)
	at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:106)
	at org.gradle.internal.progress.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:56)
	at org.gradle.initialization.DefaultGradleLauncher.doBuildStages(DefaultGradleLauncher.java:193)
	at org.gradle.initialization.DefaultGradleLauncher.doBuild(DefaultGradleLauncher.java:119)
	at org.gradle.initialization.DefaultGradleLauncher.run(DefaultGradleLauncher.java:102)
	at org.gradle.launcher.exec.GradleBuildController.run(GradleBuildController.java:71)
	at org.gradle.tooling.internal.provider.ExecuteBuildActionRunner.run(ExecuteBuildActionRunner.java:28)
	at org.gradle.launcher.exec.ChainingBuildActionRunner.run(ChainingBuildActionRunner.java:35)
	at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:41)
	at org.gradle.launcher.exec.InProcessBuildActionExecuter.execute(InProcessBuildActionExecuter.java:26)
	at org.gradle.tooling.internal.provider.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:75)
	at org.gradle.tooling.internal.provider.ContinuousBuildActionExecuter.execute(ContinuousBuildActionExecuter.java:49)
	at org.gradle.tooling.internal.provider.ServicesSetupBuildActionExecuter.execute(ServicesSetupBuildActionExecuter.java:44)
	at org.gradle.tooling.internal.provider.ServicesSetupBuildActionExecuter.execute(ServicesSetupBuildActionExecuter.java:29)
	at org.gradle.launcher.daemon.server.exec.ExecuteBuild.doBuild(ExecuteBuild.java:67)
	at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.WatchForDisconnection.execute(WatchForDisconnection.java:47)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.ResetDeprecationLogger.execute(ResetDeprecationLogger.java:26)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.RequestStopIfSingleUsedDaemon.execute(RequestStopIfSingleUsedDaemon.java:34)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:74)
	at org.gradle.launcher.daemon.server.exec.ForwardClientInput$2.call(ForwardClientInput.java:72)
	at org.gradle.util.Swapper.swap(Swapper.java:38)
	at org.gradle.launcher.daemon.server.exec.ForwardClientInput.execute(ForwardClientInput.java:72)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.LogAndCheckHealth.execute(LogAndCheckHealth.java:55)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.LogToClient.doBuild(LogToClient.java:60)
	at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.EstablishBuildEnvironment.doBuild(EstablishBuildEnvironment.java:72)
	at org.gradle.launcher.daemon.server.exec.BuildCommandOnly.execute(BuildCommandOnly.java:36)
	at org.gradle.launcher.daemon.server.api.DaemonCommandExecution.proceed(DaemonCommandExecution.java:120)
	at org.gradle.launcher.daemon.server.exec.StartBuildOrRespondWithBusy$1.run(StartBuildOrRespondWithBusy.java:50)
	at org.gradle.launcher.daemon.server.DaemonStateCoordinator$1.run(DaemonStateCoordinator.java:297)
	at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:54)
	at org.gradle.internal.concurrent.StoppableExecutorImpl$1.run(StoppableExecutorImpl.java:40)
Caused by: com.github.dockerjava.api.exception.DockerClientException: Could not push image: Repository does not exist: gitlab-bot
	at com.github.dockerjava.core.command.PushImageResultCallback.awaitSuccess(PushImageResultCallback.java:49)
	at com.github.dockerjava.core.command.PushImageResultCallback$awaitSuccess.call(Unknown Source)
	at com.github.dockerjava.core.command.PushImageResultCallback$awaitSuccess.call(Unknown Source)
	at com.bmuschko.gradle.docker.tasks.image.DockerPushImage.runRemoteCommand(DockerPushImage.groovy:61)
	at com.bmuschko.gradle.docker.tasks.AbstractDockerRemoteApiTask$_runReactiveStream_closure1.doCall(AbstractDockerRemoteApiTask.groovy:57)
	at com.bmuschko.gradle.docker.utils.DockerThreadContextClassLoader.withClasspath(DockerThreadContextClassLoader.groovy:54)
	at com.bmuschko.gradle.docker.utils.ThreadContextClassLoader$withClasspath.call(Unknown Source)
	at com.bmuschko.gradle.docker.tasks.AbstractDockerRemoteApiTask.runInDockerClassPath(AbstractDockerRemoteApiTask.groovy:62)
	at com.bmuschko.gradle.docker.tasks.AbstractDockerRemoteApiTask$runInDockerClassPath.callCurrent(Unknown Source)
	at com.bmuschko.gradle.docker.tasks.AbstractDockerRemoteApiTask.runReactiveStream(AbstractDockerRemoteApiTask.groovy:56)
	at com.bmuschko.gradle.docker.tasks.AbstractReactiveStreamsTask.start(AbstractReactiveStreamsTask.groovy:50)
	at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:73)
	at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.doExecute(DefaultTaskClassInfoStore.java:141)
	at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.execute(DefaultTaskClassInfoStore.java:134)
	at org.gradle.api.internal.project.taskfactory.DefaultTaskClassInfoStore$StandardTaskAction.execute(DefaultTaskClassInfoStore.java:123)
	at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:632)
	at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:615)
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:95)
	at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:76)
	... 70 more

Whats weird about that error is that docker wants to push to the repository gitlab-bot which is actually my docker login user. Dunno why he picks that.
Caused by: com.github.dockerjava.api.exception.DockerClientException: Could not push image: Repository does not exist: gitlab-bot

If i let pushLatest dependOn pushTag (not running in parallel then) my build works fine and both images, the tagged one and latest push successfully.
@orzeh orzeh self-assigned this Jan 10, 2017
@orzeh
Copy link
Collaborator

orzeh commented Jan 10, 2017

@maffelbaffel I don't think you run pushTag and pushLatest in parallel - we do not support this yet (see this thread for details).

Notice how pushTag and pushLatest both only depend on buildDockerImage

I see that pushTag directly depends on tagTagImage.

According to the error message there is probably an issue in your build script when you construct tag, but the snippet provides not enough information to diagnose it. How does regUrl is constructed?

@cdancy
Copy link
Collaborator

cdancy commented Jan 10, 2017

@maffelbaffel just to add to what @orzeh noted: if you're having issues pushing I would first start by getting 1 of these tasks working correctly, from beginning to end, before trying multiples of them.

@maffelbaffel
Copy link
Author

Ok somehow i cannot reproduce this myself :) I'm not exactly sure why it works now, because i did not change much.
Will close this issue then.
Thanks for the quick responds thought :)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@orzeh orzeh

Labels
help wanted
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@cdancy @orzeh @maffelbaffel