Push using Dockerfile not using cache in ADD command

[ntquyen]

I'm deploying my nodejs application using Dockerfile. To avoid having to install the dependencies in every push, I tried to take advantage of docker cache. But with git push deis master, the cache has never been used like when I tried with docker build:

$ git push deis master
remote: -----> Building Docker image
remote: Sending build context to Docker daemon  85.5 kB
remote: Sending build context to Docker daemon
remote: Step 0 : FROM quyennt/nodejs-rediscli
remote:  ---> 7e0babe53d47
remote: Step 1 : MAINTAINER quyennt "quyen@gmail.com"
remote:  ---> Using cache
remote:  ---> 1c67fad0a57c
remote: Step 2 : ENV NODE_ENV development
remote:  ---> Using cache
remote:  ---> da4280ee940f
...
remote: Step 7 : ADD ./.npmrc /
remote:  ---> e2815650d78f
remote: Removing intermediate container f8fce6913055
remote: Step 8 : ADD package.json /tmp/package.json
remote:  ---> 21ed78153e00
remote: Removing intermediate container 57c1c8220859
remote: Step 9 : RUN cd /tmp && npm install
remote:  ---> Running in 555098e1e0ec

The cache is supposed to be used in Step 7 and so on, because there was no modification in .npmrc and package.json.
Could anyone please explain why the behavior is different from using directly docker build? Thanks!!

[bacongobbler]

I agree that it'd be nice for them to work. The issue stems from how our builder is set up:

• first, we create a temp directory
• retrieve the git archive and move it over to temp directory
• build the docker image from the temp directory

So the file could be uncached for a couple reasons possibly:

• git archive master | tar strips file metadata
• running docker build in a different directory busts the cache

Those assumptions are what pop into my mind, but I'm not too sure about the specifics on cache busting.

[gabrtv]

@bacongobbler does Heroku pass build flags through git push? Without that, we'd need to rely on:

1. Special file/metadata in the repository
2. A flag on the App instance used to specify the Docker build behavior

Neither are ideal. I'd much prefer a git push option, if that's possible.

[bacongobbler]

What build flags are you referring to? I think I'm getting lost with what you mean by that term. I honestly think this is more of the fact that we strip file metadata or that we're modifying the file flags like "last updated" without even knowing it.

[dbrock]

This comes up through Google when searching for the keywords "git archive docker build," so I thought I'd just add this cross-reference for the benefit of anyone else who might find this.

Here's a workaround for the cache-busting properties of git archive:

git archive HEAD | { tmpdir=`mktemp -d` &&
  tar x -C"$tmpdir" &&
  tar c -C"$tmpdir" --mtime=1970-01-01T00:00:00Z . &&
  rm -r "$tmpdir"; } | docker build -

And here's a thread on the Git mailing list: http://thread.gmane.org/gmane.comp.version-control.git/257432

[bacongobbler]

Thanks @dbrock! We have a PR going to re-write the builder script in bash so we could definitely investigate using this snippet. 😄

[mwildehahn]

since #1859 has been merged, are there any plans to see if what @dbrock suggested might support caching ADD statements?

[Jim-Holmstroem]

this effects COPY statements also right?

[Jim-Holmstroem]

+1 @mhahn

[Jim-Holmstroem]

Since caching is pretty important for some applications shouldn't this issue be added as a note to http://docs.deis.io/en/latest/using_deis/using-dockerfiles/#using-dockerfiles ?

[mboersma]

shouldn't this issue be added as a note

Yes, I think so. I'll make a PR with the documentation change.

[bacongobbler]

IMO, we're trying to avoid adding bugs in the documentation. Bugs should stay in bug trackers; docs in the documentation. If it's a common problem (read: not a bug; usually a UX issue) that users see often, then we'd add it in Troubleshooting Deis. If it's a limitation with the platform that we'd like to see, but is unsolvable without a major re-write/breaking changes then we'd list it as a note (such as the note about publishing multiple ports in http://docs.deis.io/en/latest/using_deis/using-dockerfiles/#dockerfile-requirements). Otherwise, we direct users here for bug reports as we do in other issues.

[mboersma]

This isn't really appropriate for the Troubleshooting Deis doc, as there is no workaround currently. Although it is really a UX problem. I'm persuaded by your argument @bacongobbler.

[mboersma]

Docker v1.8 release notes say "Make build cache ignore mtime." When we are able to use that newer Docker in builder, this bug should be fixed.

[dbrock]

Horray! :-)

[bacongobbler]

We should verify this now that we're on Docker 1.8.3.

[bacongobbler]

This has been fixed.

><> git ci --allow-empty
[master c706ba3] dummy commit
><> git push deis master
Counting objects: 1, done.
Writing objects: 100% (1/1), 189 bytes | 0 bytes/s, done.
Total 1 (delta 0), reused 0 (delta 0)

-----> Building Docker image
remote: Sending build context to Docker daemon 14.34 kB
Step 0 : FROM alpine:3.1
 ---> cbc72f123e08
Step 1 : RUN apk add -U         bash    nginx   && rm -rf /var/cache/apk*
 ---> Using cache
 ---> f4880d653dc4
Step 2 : RUN ln -sf /dev/stdout /var/log/nginx/access.log
 ---> Using cache
 ---> 66d0adf121c5
Step 3 : RUN ln -sf /dev/stderr /var/log/nginx/error.log
 ---> Using cache
 ---> 592518ae1612
Step 4 : ENV POWERED_BY Deis
 ---> Using cache
 ---> 0db7afdf4edd
Step 5 : COPY rootfs /
 ---> Using cache
 ---> a1af2b7e124b
Step 6 : CMD /bin/boot
 ---> Using cache
 ---> 69fc967fe496
Step 7 : EXPOSE 80
 ---> Using cache
 ---> 8180acface23
Step 8 : ENV GIT_SHA c706ba3ff4d8520ca0efe9bc74e0341275935c93
 ---> Running in 6ee7e0823786
 ---> 8abd7716d390
Removing intermediate container 6ee7e0823786
Successfully built 8abd7716d390
-----> Pushing image to private registry

-----> Launching... 
       done, www:v3 deployed to Deis

       http://www.local3.deisapp.com

       To learn more, use `deis help` or visit http://deis.io

To ssh://git@deis.172.17.8.100.xip.io:2222/www.git
   2e2054e..c706ba3  master -> master