docker push should allow multiple tags to be pushed

[stevenschlansker]

Currently docker push can either push all layers and tags, or a single tag. In the case where you build multiple layers or tag the same layer with multiple names, pushing each tag must check that every layer is in the registry (even if they are the same) which can dramatically slow down a tag-heavy workflow.

[tuukkamustonen]

+1

We tag each container with output from git describe so for example 0.9.0-6-gbc79b83. When building containers from master in CI, the container is also tagged as latest. Having to do push in two separate invocations slows the process down somewhat...

[andresriancho]

I'm in the same case as @tuukkamustonen , getting unnecessarily slow push because of "double tag" in CI.

[frederik-b]

+1
same here, we tag with a version number and :latest and push both everytime...
with sadly an insane amount of intermediate layers
pushing the :latest onto everything else just eats time by checkin every layer and doing nothing

I want to push my image with all tags on it in 1 go please

[shuber]

👍 Same here!

We'd like to tag our images with latest, a short git sha e.g. afe1c71, and a numerical version e.g. 8. It'd be nice to be able to push all of these tags at once.

[oguzbilgic]

+1. We build our images automatically with every commit, so tagging them with both branch name and sha1 works great.

[evgeny-goldin]

👍 , definitely

[mlehner616]

👍

[paschdan]

👍

[olvesh]

👍

[harto]

👍

[maccman]

👍

[stevenschlansker]

I'm going to have to bake a birthday cake for this issue soon... almost a year and no love from maintainers :(

[cpuguy83]

PR's are welcome.

[stevenschlansker]

Unfortunately I got burned really badly on my last PR - spent months trying to get it through only to be rejected, and still no fix 6 months after that. so I'm not willing to make the same investment again :/

[jconlon]

+1

[lox]

IMO this is a terrible idea and should be avoided at all costs.

[mlehner616]

@lox I'd be interested in learning why you think this is a terrible idea. I don't see why something like docker push myimage:tag1 myimage:tag2 could be a terrible idea if it sped up our build workflows significantly. It's redundant to check every layer for each tag. It would make more sense to check the first one at the layer level. Then check the second one and others based on the image id of the first and just simply throw an extra tag on without the full per layer check. After all, we're not tagging the layers we're simply tagging the collection of layers. As long as the collection of layers is consistent between the tags this should work fine. When it's implemented as a single atomic action this really shouldn't be a problem.

[shanielh]

👍

[salamagd]

👍

[markauskas]

👍

[GordonTheTurtle]

USER POLL

The best way to get notified of updates is to use the Subscribe button on this page.

Please don't use "+1" or "I have this too" comments on issues. We automatically
collect those comments to keep the thread short.

The people listed below have upvoted this issue by leaving a +1 comment:

@amok
@andrask
@AntonioDrusin
@bmdako
@cfontes
@clkao
@commarla
@davidmarquis
@DavidXArnold
@deanrather
@DoffeL
@dustinvanbuskirk
@EliFinkelshteyn
@franklinkim
@gondor
@grachevko
@graingert
@iadknet
@iamianmiller
@jarus
@jasonm-makerbot
@jconlon
@joho
@justin8
@jvhaarst
@KhumpuW
@lauraglasu
@liyaka
@luchaninov
@mabc224
@matthewdiamant
@MiguelMCarrillo
@nicocesar
@Ohadbasan
@pavlo
@phouverneyuff
@pocman
@ramiamar
@shuklanirdesh82
@smebberson
@sylvainmouquet
@themethod
@Trandel
@winstonk
@xnaud
@yuvalbar

[justin8]

Maybe after 13 months with no word from devs people might think showing their interest in such a feature might get some recognition? Many feature requests for docker seem to go totally ignored; not even so much as a "we've thought about this and made it low priority" from someone on the project.

And then the first 'official' response is from a joke account pointing out that people want a feature and are being ignored? Thanks.

[vdemeester]

@justin8 see #15624 (comment). I'm quoting :

I am against adding this functionality without first addressing the major bugs in this section of the code and deciding the future solution to remotes. Specifically, we don't want to extend the push/pull command syntax until we've actually locked down image reference syntax and have made decision about how one might select push location.

There is work to do before doing this, and it's kinda a design decision to make. Pushing multiple image is still possible using a for loop in a bash script (or in the api consumer) — this is nice to have but I think it's definitely not a blocker though..

So yeah, I think it's low priority for now.

[lox]

Hrm, could several of the +1's have been to my comment that it shouldn't be fixed though? It probably throws into question the entire methodology of @GordonTheTurtle's user poll.

[justin8]

@vdemeester that's a comment on an entirely different issue. Should people be expected to read all the comments on all 954 open issues? Or perhaps all 7,000 issues and 8,800 merge requests?

Thanks for the info though.

[vdemeester]

@justin8 note that it's a comment on a pull-request referencing this issue (and appears just on top of the @GordonTheTurtle message even), so people subscribing the issue and that have comment, have been notified of this PR. But yeah, it doesn't cost much to put a comment here too 😉.

[mlehner616]

I think the intention of this ticket was to offer a possible solution to the underlying issue which is that docker much recheck every layer multiple times. Issue #12489 better describes what the problem is with the lack of this. Let's close this one and move the discussion to #12489. I'm in favor of pushing this back until later as it isn't as important as others and a workaround does exist (at the great expense of build and deploy times) but only if we could address some critical bugs that have been open for more than year, some, 2 years with, similarly to this issue, little to no attention from developers. The most frustrating thing about the comment from @stevvooe on your PR is that we're almost at 2.0 and an image reference syntax has not yet been locked down. It just seems like a fundamental decision that should have been made long ago.

The following critical issues must get fixed snarky comments aside. If you run docker in production you have 100% chance of hitting at least one of these. So if we could fix these instead of this one, that's fine by me.

The following important bugs and requests have had ample meaningful discussion but little or no meaningful attention from devs in about a year:
#5684
#8427
#8539 (This critical one was closed with no fix)
#8502

I'm starting to understand why the community might be getting frustrated....

[stevvooe]

@mlehner616 I agree that #12489 appropriately addresses the root cause and discussion should be moved their. For this particular issue, checking layer existence is simply not the primary bottleneck. The proper optimization to address this issue is to make the HEAD check fast on the registry side and ensure that results are properly cached. We are also exploring some fixes to hash stability (see #15177 as an example) that can help make this more efficient. If we make the decision to allow multiple pushes in the same CLI call, we remove the ability to add the nice git-like remote syntax for push/pull. I believe we can address the root cause of this issue without affecting the UI.

As an aside, we are working on the reference package to address the image reference issue in a consistent, type-safe and uniform manner. This will enable a number of features, such as proper remote configuration. More aptly, proper types and data structures will reduce odd behavior and defects. Unfortunately, because of the high number of stakeholders, this process is slow and painstaking.

It would be great if we could get your help on the enumerated issues. The number of contributors working on maintenance is small and the core team is heavily overworked. While these issues seem simple on face value, many require a large amount of time to analyze, reproduce, debug and characterize. A well-done, clear reproduction can do wonders in moving them forward.

[stevenschlansker]

As the original submitter of this ticket, I can confirm that really what it is complaining about is "Pushing and pulling images in a tag-heavy workflow, especially across a WAN, is glacial". Our London team has basically decided that they hate us at this point over this issue and related ones...

I never intended to prescribe a particular fix, we just want to not have Docker be the single bottleneck in our entire workflow. The introduction of v2 which even slowed down some operations, as well as introducing new areas of instability (I've had to report three separate issues against it already) has only burned yet more of our "user happiness buffer" with Docker...

I'm sympathetic to the "number of developers working on this is low, and number of user requests is high" -- but somehow Docker finds time to implement endless streams of complicated features (An entire software defined networking stack? A new differential Tar format?) while simple usability problems that affect every one of our developers every day go without even a peep from maintainers.

[rzachariah]

I just set up a CircleCI pipeline with Docker integration, and I'm having this problem. My pipeline builds a docker image, tags it with $CIRCLE_SHA1, and then pushes the image to our Docker repo. Ideally, I'd like to do a single docker push command, but to update the image tagged as latest, and create the new CIRCLE_SHA1 tag, I'm finding I need to push twice. Each push is about 3-4 minutes, so this is a drag.

Suggestions welcome!

[rzachariah]

Actually, the second push only takes 3s, so this isn't a big deal for me.

[mlehner616]

@stevenschlansker Well said. The biggest gripe I had with the implementation of features issue was the addition of Windows support. Great feature, but thought it was completely unjustified given the critical issues facing the Linux platform as it stands. Docker maintainers can't expect to make decisions like that without blowback from the community especially when people do actually spend time on PRs to have them rejected because it doesn't align with the vision. Community members in the end feel betrayed and move on to other things.
[ End rant ]

Anyway:
@rzachariah Are you using a private repo? If so, are you on docker registry 1.0 or 2.0?

[rzachariah]

@mlehner616 I'm using a private repo hosted on dockerhub. I don't know if it's registry 1.0 or 2.0--it's whatever dockerhub uses. :)

[stevvooe]

@stevenschlansker @mlehner616 I know this can be frustrating but there are a number of factors to take into account when making road map decisions or when accepting PRs. This ticket isn't really the venue for discussing these matters. If you would like to more constructively voice your concerns, I would suggest making this point on the mailing list or via IRC.

[askb]

+1, re-tagging every version of the image to push into the registry is time consuming, something like docker push --all-tags <> would greatly help! when you have a large list of images with different tags or versions. I have added the below lines to my ~/.bashrc (may require some tweaking depending on the name convention used).

# $1 - image with multiple versions
# $2 - registry (ex: 192.168.1.34:5000) 
# usage: dockerpush-alltags devbox 192.168.1.34:500
function dockerpush-alltags() {
    for i in `docker images | grep ^$1 | awk -e 'BEGIN{OFS=":"}{print $1,$2}'`; do docker tag $i $2/$i ; done   
    for i in `docker images | grep ^$1/$2 | awk -e 'BEGIN{OFS=":"}{print $1,$2}'`; do docker push $i; done
}

[ProTip]

I'm bumping up against this now; a significant portion of one of my app builds will be spent doing layer checks for tag pushes. Don't we have Merkle trees now? I thought this would have allowed efficient layer checking/transfer.

[graingert]

really what should happen (according to https://docs.docker.com/registry/spec/api/#pushing-an-image-manifest) is docker should push the image manifest, then for each error upload the relevant layers, then upload the manifest.

[graingert]

ping @stevvooe @aaronlehmann can we get an official view on this ticket?

[stevvooe]

@graingert Introducing this syntax will not make push faster. Remember that push is implemented in a daemon, not the cli, so this implementation will result in the same number of HTTP requests to the daemon as it would be in multiple commands.

There are a number of optimizations on the table that address the concerns in this issue. Specifically, we are not leveraging cache information between push calls, even though the daemon can maintain this state quite easily. It would be great to get some help to get push working as fast as possible before extending command syntax.

We'd really like to reserve extended docker push syntax for remote specification, similar to git:

docker push myremote image:3.4.5

We can't have both due the similarities between image names and remote names, which might be a url. We may be able to add multi-tag push once remotes is figured out.

In the meantime, we would appreciate help from the community in rooting out the actual performance problems in push without introducing more CLI functionality.

[graingert]

Great, can you close this issue?

[Hubbitus]

May be push will not be faster, but command will be clear. Especially in cases we make tag and always move forward latest.

[stevvooe]

@Hubbitus I am not sure that you read my comment in full.

[aaronlehmann]

I noticed an issue with the same topic in the docker/cli repository: docker/cli#267

I'm going to close this issue in favor of that one, since it's a cli topic.

Please don't interpret this as dismissing the feature request - I'm only trying to make sure it's tracked in a single issue in the right place.