No way to use both --cache-from and a local store?

[asilversempirical]

Using the new --cache-from is a huge improvement most of the time, but it seems to cause docker to avoid looking in its, for lack of a better term, local cache, meaning that a second build of the same image starts from scratch, or at least from where the remote cache diverges.

https://github.com/docker/docker/blob/master/daemon/cache.go#L12 seems to confirm that this is very much intended.

It would be great to look at the local cache first and only hit the --cache-from images if it doesn't find anything. Or possibly have a way of specifying the local cache as an option, say
docker build --cache-from . myimage:v1.0

It would also be great to document this behavior as part of the --cache-from docs. Right now it says "Images to consider as cache sources" but it's unclear that that causes the normal cache to be unconsidered.

Thanks!

Steps to reproduce the issue:

docker pull myimage:v1.0
docker build --cache-from myimage:v1.0 -t myimage:v1.1 .
docker build --cache-from myimage:v1.0 -t myimage:v1.1 .

Describe the results you received:
The second build starts from the latest step it shares with myimage:v1.0

Describe the results you expected:
The second build re-uses the entire cache from the first build.

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

Client:
 Version:      17.03.1-ce
 API version:  1.27
 Go version:   go1.7.5
 Git commit:   c6d412e
 Built:        Fri Mar 24 00:45:26 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.03.1-ce
 API version:  1.27 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   c6d412e
 Built:        Fri Mar 24 00:45:26 2017
 OS/Arch:      linux/amd64
 Experimental: false

Output of docker info:

Containers: 12
 Running: 0
 Paused: 0
 Stopped: 12
Images: 239
Server Version: 17.03.1-ce
Storage Driver: overlay2
 Backing Filesystem: extfs
 Supports d_type: true
 Native Overlay Diff: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins: 
 Volume: local
 Network: bridge host macvlan null overlay
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 4ab9917febca54791c5f071a9d1f404867857fcc
runc version: 54296cf40ad8143b62dbcaa1d90e520a2136ddfe
init version: 949e6fa
Security Options:
 apparmor
 seccomp
  Profile: default
Kernel Version: 4.4.0-72-generic
Operating System: Ubuntu 16.04.2 LTS
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 7.795 GiB
Name: ip-172-31-50-48
ID: H5BG:BAQV:VVQ2:BGUO:5MQG:OXKU:T6MH:ZDXF:ZMXT:TP7G:B6K3:JIZG
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): false
Username: <redacted>
Registry: https://index.docker.io/v1/
WARNING: No swap limit support
Experimental: false
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):

[thaJeztah]

ping @tonistiigi

[tonistiigi]

--cache-from can be used multiple times to use both local and pulled image. I don't think we want to mix them as it creates other weird cases, possibly matching unexpected images. Also, from implementations, the caching strategies are very different, for example local cache is not equal to docker build --cache-from * because that strategy always matches against a specific target and can't switch them in the middle of the build. Always fine to make the docs more clear.

[asilversempirical]

For context, my use case is docker-compose for a stack that runs both locally (for developers) and on travis (for tests).

Travis doesn't maintain a docker cache across runs, so --cache-from is a godsend. But locally we'd really like to use our local caches since most of the time the entire image is cached, rather than just some reasonably recent one (and one we haven't necessarily pulled)

But we'd ideally like to use the same docker-compose.yml for both local and travis runs. Which means our docker-compose.yml looks like (paraphrased):

services:
  server:
    build:
      cache_from: ['dockerhub/server']
  logger:
    build:
      cache_from: ['dockerhub/logger']

Which works as well as possible on Travis, but locally means we lose how great our local cache should run.

My current workaround is to add something like:

services:
  server:
    build:
      cache_from: ['server', 'dockerhub/server']
    image: server
  logger:
    build:
      cache_from: ['logger', 'dockerhub/logger']
    image: logger

which is annoying for two reasons. The minor one is that having to specify yourself feels a bit silly. The more serious one is that you lose any inter-image caching between server and logger. You can go even further and have a whole cartesian explosion of --cache-froms but that's really pushing it.

[Schnitzel]

I'm having the same exact same problem. The workaround with defining an image name within docker-compose and using it as a cache_from itself works, but is tedious and also doesn't work in the case we have multi stage builds, see #34715

[kilianc]

I'm having the same exact same problem. The workaround with defining an image name within docker-compose and using it as a cache_from itself works, but is tedious and also doesn't work in the case we have multi stage builds, see #34715

This works if the tag in the build is static, if it's dynamic (git commit hash for example) caching will only work until your HEAD changes.

I would love a way to specify cache_from + local, for example --cache-from image:*

[thaJeztah]

Let me close this ticket for now, as it looks like it went stale, and this was about the cache-from implementation in the classic builder, which has now been superseded by BuildKit