docker build -no-cache does not consistently invalidate the cache for subsequent builds

[vreon]

In this example, docker build -no-cache appears to invalidate the cache for subsequent builds:

$ cat Dockerfile
FROM ubuntu:12.10
RUN date > builddate
$ docker build -t vreon/cachetest .
Uploading context 10240 bytes
Step 1 : FROM ubuntu:12.10
 ---> b750fe79269d
Step 2 : RUN date > builddate
 ---> Running in cb5a9c443588
 ---> 9259bb0663e7
Successfully built 9259bb0663e7
$ docker build -no-cache -t vreon/cachetest .
Uploading context 10240 bytes
Step 1 : FROM ubuntu:12.10
 ---> b750fe79269d
Step 2 : RUN date > builddate
 ---> Running in 411ace38ef1f
 ---> 237318dc283c
Successfully built 237318dc283c
$ docker build -t vreon/cachetest .
Uploading context 10240 bytes
Step 1 : FROM ubuntu:12.10
 ---> b750fe79269d
Step 2 : RUN date > builddate
 ---> Using cache
 ---> 237318dc283c
Successfully built 237318dc283c

tl;dr: 9259bb0663e7, 237318dc283c, 237318dc283c

"Awesome," you'd think, "-no-cache is a nice way to force a full rebuild and account for the occasional RUN that isn't actually deterministic."

But then you try it with a slightly different RUN instruction:

$ cat Dockerfile
FROM ubuntu:12.10
RUN echo `date` > builddate
$ docker build -t vreon/cachetest .
Uploading context 10240 bytes
Step 1 : FROM ubuntu:12.10
 ---> b750fe79269d
Step 2 : RUN echo `date` > builddate
 ---> Running in 0c320266f660
 ---> 1c732f0b8836
Successfully built 1c732f0b8836
$ docker build -no-cache -t vreon/cachetest .
Uploading context 10240 bytes
Step 1 : FROM ubuntu:12.10
 ---> b750fe79269d
Step 2 : RUN echo `date` > builddate
 ---> Running in 67d2a2ae033a
 ---> f59e6459f768
Successfully built f59e6459f768
$ docker build -t vreon/cachetest .
Uploading context 10240 bytes
Step 1 : FROM ubuntu:12.10
 ---> b750fe79269d
Step 2 : RUN echo `date` > builddate
 ---> Using cache
 ---> 1c732f0b8836
Successfully built 1c732f0b8836

tl;dr: 1c732f0b8836, f59e6459f768, 1c732f0b8836

So now -no-cache appears to not invalidate the cache -- the third build used the images generated by the first. Essentially, the first time you use -no-cache, you have to keep using -no-cache on all subsequent builds, or you risk reusing a layer you thought invalidated.

$ docker version
Client version: 0.7.1
Go version (client): go1.2
Git commit (client): e39d35d
Server version: 0.7.1
Git commit (server): e39d35d
Go version (server): go1.2
Last stable version: 0.7.1
$ docker info
Containers: 49
Images: 50
Driver: aufs
 Root Dir: /mnt/archive/docker/aufs
 Dirs: 151
WARNING: No swap limit support

[crosbymichael]

@vreon -no-cache does not invalidate the cache for all builds. It tells docker to not use the cache for THIS build.

[vreon]

@crosbymichael That's fine, but it clearly does invalidate the cache for some builds (see my first example, which should have produced 9259bb0663e7 for the third build, not 237318dc283c). I would argue that the flag should behave consistently regardless of what's in your cache or Dockerfile.

[vieux]

@vreon no-cache doesn't mean don't create cache. it means don't use cache.
You example is perfectly fine.

1st buid produce 9259bb0663e7, so 9259bb0663e7 is in you cache
2nd build you used -no-cache so it will produce a new image 237318dc283c, so 237318dc283c is in you cache
3rd build uses cache so you get 237318dc283c again

[vreon]

@vieux That makes sense... but then, why didn't my second example use f59e6459f768 for the third build, if f59e6459f768 was in my cache?

Sorry, I promise I'm not trying to be obnoxious! :)

[vieux]

@vreon I see f59e6459f768 is produced once, never used. Look above there is ---> Running in... not ---> Using cache

[vreon]

@vieux But that's true for both examples. They both look like this:

Build 1, Step 2: ---> Running in...
Build 2, Step 2: ---> Running in...
Build 3, Step 2: ---> Using cache

The only difference between the two examples is part of the RUN. Why do they do different things?

[borromeotlhs]

Doesn't this raise the fact that if @vreon is correct, that you risk having inconsistent cache results in builds? The only way, then, is to always use -no-cache to ensure you NEVER use a cache at all. When, in fact, I would like to have a command that invalidates a cache entry for a specific RUN command per Dockerfile/image build at the very least.

Can we at least have a mechanism to list how many entries from a Dockerfile/build are in the cache, and allow for invalidating those entries?

[thaJeztah]

@borromeotlhs please see prior discussions on influencing the cache in
#6251, #1996, and a possible solution in #9176 and #15182 (the latest will be in Docker 1.9)

[samistart]

By the way, it's

--no-cache

(with two leading hyphens)

[vreon]

@samistart, Docker used single dashes for long options in 2013.

I haven't used --no-cache since this issue was closed, but I'm pleased to note that the bug is no longer present in modern versions of Docker.