Support tag task name for 'tags' #205

devkanro · 2018-10-17T12:04:43Z

Description

This PR make 'tags' more powerful, you can use new 'tag' DSL to give the tag a task name.
There are some samples:

docker {
    name 'fake-service-name'
    tag 'withTaskName', 'fake-service-name:2.0'
    tag 'newImageName', 'new-image:latest'
}

It will generate tag tasks with name dockerTagWithTaskName and dockerTagNewImageName, and push tasks with name dockerPushWithTaskName and dockerPushNewImageName.

With task name, we can build some diff name image of one image, push to diff repo host.

Note

'tags' DSL can resolve the tag name auto, but 'tag' can not. It means you must specify the image name for 'tag' DSL.

There are some sample:
Tag with just new version:
name: old-name:test
tags dsl: tags 'latest'
tag dsl: tag 'latest', 'docker-name:latest'
image result name: docker-name:latest

Tag with new name:
name: old-name:test
tags dsl: tags 'newName@docker-name:latest'
tag dsl: tag 'newName', 'docker-name:latest'
image result name: docker-name:latest

Tag with new repo host:
name: old-name:test
tag: newName@host:port/repo/docker-name:latest
tags: newName', 'host:port/repo/docker-name:latest'
image result name: host:port/repo/docker-name:latest

Note:
Notice that tag must have a task name if you want to replace the repo or image name for 'tags' DSL

New tasks

dockerTagsPush
Pushes all tagged Docker images to configured Docker Hub.

dockerTagsPush= dockerPush<Tag1>+ dockerPush<Tag2> + ... + dockerPush<Tagn>

dockerAllPush
Pushes all tagged Docker images and named Docker image to configured Docker Hub.

dockerAllPush = dockerTagsPush + dockerPush

Solved issues

#195 Allow dockerPush to target multiple repositories
Use dockerPush<Tag> to diff repositories

docker {
    name 'repository1.foo.com/bar:lastest'
    tag 'repository2', 'repository2.foo.com/bar:lastest'
}

Run this command:

./gradle dockerPushRepository2

#181 Pushing just the defined tags
Use dockerTagsPush task can solve this issue

docker {
    name 'repository1.foo.com/bar:lastest'
    tag 'tag1', 'bar:v1'
    tag 'tag2', 'bar:v2'
}

Run this command:

./gradle dockerTagsPush

#95 Make push task depend on push tasks for each tag if tags are present
Use dockerTagsPush and dockerPush can solve this issue/PR without change behavior of dockerPush

docker {
    name 'repository1.foo.com/bar:lastest'
    tag 'tag1', 'bar:v1'
    tag 'tag2', 'bar:v2'
}

Run this command:

./gradle dockerPush
./gradle dockerTagsPush

#149 dockerPush doesn't generate (and thus push) all Tags
#94 Ability to easily push all tags for an image
#209 Docker tag should support full retagging

Test sample

There are some sample which used in test:

name	tag	result
v1	latest	v1:latest
v1:1	latest	v1:latest
host/v1	latest	host/v1:latest
host/v1:1	latest	host/v1:latest
host:port/v1	latest	host:port/v1:latest
host:port/v1:1	latest	host:port/v1:latest
v1	name@latest	v1:latest
v1:1	name@latest	v1:latest
host/v1	name@latest	host/v1:latest
host/v1:1	name@latest	host/v1:latest
host:port/v1	name@latest	host:port/v1:latest
host:port/v1:1	name@latest	host:port/v1:latest
v1	name@v2:latest	v2:latest
v1:1	name@v2:latest	v2:latest
host/v1	name@v2:latest	v2:latest
host/v1:1	name@v2:latest	v2:latest
host:port/v1	name@v2:latest	v2:latest
host:port/v1:1	name@v2:latest	v2:latest
v1	name@host/v2	host/v2
v1:1	name@host/v2	host/v2
host/v1	name@host/v2	host/v2
host/v1:1	name@host/v2	host/v2
host:port/v1	name@host/v2	host/v2
host:port/v1:1	name@host/v2	host/v2
v1	name@host/v2:2	host/v2:2
v1:1	name@host/v2:2	host/v2:2
host/v1	name@host/v2:2	host/v2:2
host/v1:1	name@host/v2:2	host/v2:2
host:port/v1	name@host/v2:2	host/v2:2
host:port/v1:1	name@host/v2:2	host/v2:2
v1	name@host:port/v2:2	host:port/v2:2
v1:1	name@host:port/v2:2	host:port/v2:2
host/v1	name@host:port/v2:2	host:port/v2:2
host/v1:1	name@host:port/v2:2	host:port/v2:2
host:port/v1	name@host:port/v2:2	host:port/v2:2
host:port/v1:1	name@host:port/v2:2	host:port/v2:2

palantirtech · 2018-10-17T12:04:44Z

Thanks for your interest in palantir/gradle-docker, @higankanshi! Before we can accept your pull request, you need to sign our contributor license agreement - just visit https://cla.palantir.com/ and follow the instructions. Once you sign, I'll automatically update this pull request.

devkanro · 2018-10-30T07:38:16Z

Any update here? @pkoenig10
Not just me, there are still many people who really need this feature.

devkanro · 2018-12-21T06:44:39Z

@uschi2000 @pkoenig10
Could you review this PR? I really need this feature.
if it still no reply for me, I have to publish my fork to plugin repositories.

uschi2000 · 2018-12-21T10:26:27Z

Hej @higankanshi , sorry this slipped through the cracks. I'll take a look now.

devkanro · 2018-12-21T17:41:54Z

@uschi2000 Thanks for your time.

uschi2000 · 2018-12-22T09:11:00Z

Thinking about this change a bit, I wonder if we should take a harder break and rectify some of the config birth defects, in particular the asymmetry between image name and image tags. Should they really be the same thing, and thus naturally allow for multiple hosts etc?
Something like

docker {
  imageName repo:port/foo:1.0
  imageName otherRepo/foo:latest, myTaskName
  ..
}

My rationale is that while your proposed is clearly something we should do (in order to fix all those issues), the implementation makes the config language pretty complicated (as witnessed by constraints like Notice that tag must have a task name if you want replace the repo or image name).

Thoughts?

devkanro · 2018-12-22T15:57:08Z

@uschi2000
We can use the tag command of docker to set repo host for docker, like this:

docker tag httpd fedora/httpd:version1.0
docker tag httpd:test fedora/httpd:version1.0.test

So, i think that tag can be include a repo host, new image name.
But use the @ for marking task name, I think it can be more better, like your example

docker {
  tag 'repo:port/foo:1.0' // Invalid tag, because we can get a valid name for this tag
  tag 'foo:latest' // Maybe can make a dockerFooLatest task ?
  tag 'latest' // Maybe we should not support version only for defined a tag
  tag 'otherRepo/foo:latest', myTaskName // Good, a custom name
}

But if we do this, it would be a breaking change for this plugin...
We should seriously weigh for it.

Using @ for backward compatible
or
Using new dsl for better semantics

pkoenig10 · 2018-12-22T22:58:58Z

I agree with @uschi2000.

It's seems odd to me that tag in this plugin is interpreted differently than arguments to the docker tag. Namely, we try to merge the name and the tag fields to compute an argument for docker tag.

I think we still need the name field to provide a primary identifier for the created image that can be used by other tasks. But the name and tag fields would be interpreted identically. The only difference being that the name tag is applied at build time, while the tag tags are applied in a separate task.

Providing custom task names seems like a step in the right direction, since it seems hopeless for us to try and derive task names if given tags which may or may not contain a registry, repository, image name, and/or tag.

devkanro · 2018-12-23T07:55:54Z

@pkoenig10
Yes, you are right.

How do you think about using @ for marking task name?
I am using this for backward compatible, you do not have to change your existing config for this change.

uschi2000 · 2018-12-24T09:53:35Z

@pkoenig10 if you want to do the DSL break, then we'd also have to think about excavating our internal repos to get them towards the new config. should be relatively simple, but just flagging.

pkoenig10 · 2018-12-27T15:25:06Z

Ideally I'd like to avoid a DSL break here. Using @ for marking task name seems like a reasonable approach. This should be safe since @ cannot be used in tag names.

It looks like the primary blocker to supporting the better DSL is the tags method that takes a variable number of tags. Maybe as part of this change we should also deprecate this method.

Eventually it would be good to push everyone to the better DSL. This probably will take a couple of steps:

Support using @ for marking task names and deprecated the varargs tags method
[break] Remove the varargs tags method, replace it with a two arg (tag, taskName) tags method

And then optionally, to remove custom syntax from this plugin:
3. [break] Remove support for using @ to mark task names. This break will fail loudly since @ cannot be used in tag names.

devkanro · 2018-12-28T03:52:59Z

OK, got it, I will change my PR later

devkanro · 2018-12-28T07:15:41Z

@uschi2000 @pkoenig10
I have added the 'tag' DSL and marked the 'tags' DSL as deprecated.

docker {
  tag 'newTag', 'repo:port/foo:1.0' 
}

However, it doesn't resolve image name for the tag. If you want to retag the version, you must specify the image name too.
This limit due to the design of docker's tag command, it just makes them as same.

docker {
  tag 'latest', 'latest'  // Invalid tag, it must be "tag 'latest', 'name:latest' "
}

pkoenig10

Design looks good, couple of implementation comments

src/test/groovy/com/palantir/gradle/docker/PalantirDockerPluginTests.groovy

src/main/groovy/com/palantir/gradle/docker/DockerExtension.groovy

src/main/groovy/com/palantir/gradle/docker/PalantirDockerPlugin.groovy

pkoenig10

LGTM, couple of minor comments

readme.md

src/main/groovy/com/palantir/gradle/docker/PalantirDockerPlugin.groovy

devkanro · 2019-01-04T04:52:52Z

Fixed

src/main/groovy/com/palantir/gradle/docker/PalantirDockerPlugin.groovy

Support tag task name for 'tags'

153d801

devkanro requested a review from a team as a code owner October 17, 2018 12:04

Add 'dockerTagsPush' and 'dockerAllPush' task

07dc932

This was referenced Oct 19, 2018

Improvement: Allow dockerPush to target multiple repositories #195

Closed

Pushing just the defined tags #181

Open

Ability to easily push all tags for an image #94

Open

devkanro mentioned this pull request Oct 30, 2018

Docker tag should support full retagging #209

Open

devkanro added 2 commits December 28, 2018 14:27

New tag dsl for image tagging

f94e333

Fix error on conflict task name

29f44ab

devkanro mentioned this pull request Dec 29, 2018

Make push task depend on push tasks for each tag if tags are present #95

Closed

pkoenig10 reviewed Dec 30, 2018

View reviewed changes

devkanro added 3 commits January 1, 2019 03:02

Fix some review suggestions

126fc5d

Fix test failed because of the immutable map

86a4ffd

Remove dockerAllPush task

cfa1c4e

devkanro force-pushed the tag-task-name branch from c07a471 to cfa1c4e Compare January 3, 2019 03:23

pkoenig10 reviewed Jan 3, 2019

View reviewed changes

Fix other review suggestions

ad4518f

pkoenig10 approved these changes Jan 4, 2019

View reviewed changes

src/main/groovy/com/palantir/gradle/docker/PalantirDockerPlugin.groovy Outdated Show resolved Hide resolved

Remove unnecessary 'else'

6f45bd3

pkoenig10 merged commit 64853c1 into palantir:develop Jan 7, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support tag task name for 'tags' #205

Support tag task name for 'tags' #205

devkanro commented Oct 17, 2018 •

edited

Loading

palantirtech commented Oct 17, 2018

devkanro commented Oct 30, 2018 •

edited

Loading

devkanro commented Dec 21, 2018 •

edited

Loading

uschi2000 commented Dec 21, 2018

devkanro commented Dec 21, 2018

uschi2000 commented Dec 22, 2018

devkanro commented Dec 22, 2018 •

edited

Loading

pkoenig10 commented Dec 22, 2018

devkanro commented Dec 23, 2018 •

edited

Loading

uschi2000 commented Dec 24, 2018

pkoenig10 commented Dec 27, 2018

devkanro commented Dec 28, 2018

devkanro commented Dec 28, 2018 •

edited

Loading

pkoenig10 left a comment

pkoenig10 left a comment

devkanro commented Jan 4, 2019

Support tag task name for 'tags' #205

Support tag task name for 'tags' #205

Conversation

devkanro commented Oct 17, 2018 • edited Loading

Description

Note

New tasks

Solved issues

Test sample

palantirtech commented Oct 17, 2018

devkanro commented Oct 30, 2018 • edited Loading

devkanro commented Dec 21, 2018 • edited Loading

uschi2000 commented Dec 21, 2018

devkanro commented Dec 21, 2018

uschi2000 commented Dec 22, 2018

devkanro commented Dec 22, 2018 • edited Loading

pkoenig10 commented Dec 22, 2018

devkanro commented Dec 23, 2018 • edited Loading

uschi2000 commented Dec 24, 2018

pkoenig10 commented Dec 27, 2018

devkanro commented Dec 28, 2018

devkanro commented Dec 28, 2018 • edited Loading

pkoenig10 left a comment

Choose a reason for hiding this comment

pkoenig10 left a comment

Choose a reason for hiding this comment

devkanro commented Jan 4, 2019

devkanro commented Oct 17, 2018 •

edited

Loading

devkanro commented Oct 30, 2018 •

edited

Loading

devkanro commented Dec 21, 2018 •

edited

Loading

devkanro commented Dec 22, 2018 •

edited

Loading

devkanro commented Dec 23, 2018 •

edited

Loading

devkanro commented Dec 28, 2018 •

edited

Loading