Improve Docker Image Pull No-op Performance

[josephearl]

When developing in a large team with an application composed of multiple images, other images may be updated quite frequently so if you want to test your changes against the trunk of the rest of the application, frequent pulling is required.

However, in the case there are no changes docker pull is still quite slow (3-5s per image), which tends to reduce the amount of times developers run the command.

Another place this becomes a problem is in scripts which docker pull to ensure they have the latest images (e.g. for tools like the AWS CLI), even more so when there is a sequence of these scripts. (Yes, in theory it is possible to collect all the different images needed in all the scripts and do one giant docker pull at the start, but in many cases not practical)

I'm not sure to what extent an improvement is possible, so this is a request for comments, but having a very fast no-op case could allow developer productivity tools like compose to have commands that automatically checked for updates to images.

Similar topics have been discussed recently in (other) package management tools:

• Improving performance of brew update (and running it automatically) - Shall we run brew update automatically? Homebrew/legacy-homebrew#38818 (comment)
• Poor NPM install performance for the no-op case - npm@3 wants to be faster npm/npm#8826 (comment)

Results of pulling an image on my machine:

$ time docker pull postman/newman_ubuntu1404
Using default tag: latest
latest: Pulling from postman/newman_ubuntu1404
2de59b831a23: Already exists 
354c3661655e: Already exists 
91930878a2d7: Already exists 
a3ed95caeb02: Already exists 
e2238a1528bc: Already exists 
abe3bf442fa4: Already exists 
Digest: sha256:38db8e9d26ec3b35df1beaf4785462deda6e9fffd9b481a2925dad3ec91d16ed
Status: Image is up to date for postman/newman_ubuntu1404:latest

real    0m4.274s
user    0m0.018s
sys 0m0.026s

[GordonTheTurtle]

If you are reporting a new issue, make sure that we do not have any duplicates already open. You can ensure this by searching the issue list for this repository. If there is a duplicate, please close your issue and add a comment to the existing issue instead.

If you suspect your issue is a bug, please edit your issue description to include the BUG REPORT INFORMATION shown below. If you fail to provide this information within 7 days, we cannot debug your issue and will close it. We will, however, reopen it if you later provide the information.

For more information about reporting issues, see CONTRIBUTING.md.

You don't have to include this information if this is a feature request

(This is an automated, informational response)

---

BUG REPORT INFORMATION

Use the commands below to provide key information from your environment:

docker version:
docker info:

Provide additional environment details (AWS, VirtualBox, physical, etc.):

List the steps to reproduce the issue:
1.
2.
3.

Describe the results you received:

Describe the results you expected:

Provide additional info you think is important:

----------END REPORT ---------

#ENEEDMOREINFO

[josephearl]

I'm going to close this based on docker/compose#1652 - which in theory should solve the issue for large projects by pulling parallel.