build --pull fails for local image dependency, works without --pull

[rvernica]

If the docker compose file has two images, where one image is built from the other image, build --pull fails to build the dependent image and stops the build. build without --pull builds the dependent image successfully. Still no pull will be done for images following the dependent image.

Here is an example:

> ls -R a
a:
bar  docker-compose.yml  foo  taz

a/bar:
Dockerfile

a/foo:
Dockerfile

a/taz:
Dockerfile

docker-compose.yml and Dockerfiles:

> cat docker-compose.yml
version: '3'
services:
  foo:
    build: ./foo
  bar:
    build: ./bar
  taz:
    build: ./taz

> cat foo/Dockerfile
FROM alpine:3.7

> cat bar/Dockerfile
FROM a_foo

> cat taz/Dockerfile
FROM alpine:3.7

buld --pull:

> docker-compose build --pull
Building foo
Step 1/1 : FROM alpine:3.7
3.7: Pulling from library/alpine
Digest: sha256:7df6db5aa61ae9480f52f0b3a06a140ab98d427f86d8d5de0bedab9b8df6b1c0
Status: Image is up to date for alpine:3.7
 ---> 3fd9065eaf02
Successfully built 3fd9065eaf02
Successfully tagged a_foo:latest
Building bar
Step 1/1 : FROM a_foo
ERROR: Service 'bar' failed to build: pull access denied for a_foo, repository does not exist or may require 'docker login'

Notice how it never gets to taz. So, it I want to do a pull on taz or any other services follwoing it, I can't.

build without --pull

> docker-compose build
Building foo
Step 1/1 : FROM alpine:3.7
 ---> 3fd9065eaf02
Successfully built 3fd9065eaf02
Successfully tagged a_foo:latest
Building bar
Step 1/1 : FROM a_foo
 ---> 3fd9065eaf02
Successfully built 3fd9065eaf02
Successfully tagged a_bar:latest
Step 1/1 : FROM alpine:3.7
 ---> 3fd9065eaf02
Successfully built 3fd9065eaf02
Successfully tagged a_taz:latest

Version info:

> docker-compose --version
docker-compose version 1.18.0, build 8dd22a9

> docker info
Containers: 1
 Running: 0
 Paused: 0
 Stopped: 1
Images: 271
Server Version: 17.12.0-ce
Storage Driver: devicemapper
 Pool Name: docker-253:1-19922948-pool
 Pool Blocksize: 65.54kB
 Base Device Size: 10.74GB
 Backing Filesystem: xfs
 Udev Sync Supported: true
 Data file: /dev/loop0
 Metadata file: /dev/loop1
 Data loop file: /home/docker/devicemapper/devicemapper/data
 Metadata loop file: /home/docker/devicemapper/devicemapper/metadata
 Data Space Used: 37.92GB
 Data Space Total: 107.4GB
 Data Space Available: 69.45GB
 Metadata Space Used: 54.34MB
 Metadata Space Total: 2.147GB
 Metadata Space Available: 2.093GB
 Thin Pool Minimum Free Space: 10.74GB
 Deferred Removal Enabled: true
 Deferred Deletion Enabled: true
 Deferred Deleted Device Count: 0
 Library Version: 1.02.144 (2017-10-06)
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
 Volume: local
 Network: bridge host macvlan null overlay
 Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: runc
Default Runtime: runc
Init Binary: docker-init
containerd version: 89623f28b87a6004d4b785663257362d1658a729
runc version: b2567b37d7b75eb4cf325b77297b140ea686ce8f
init version: 949e6fa
Security Options:
 seccomp
  Profile: default
Kernel Version: 4.14.14-300.fc27.x86_64
Operating System: Fedora 27 (Workstation Edition)
OSType: linux
Architecture: x86_64
CPUs: 4
Total Memory: 15.58GiB
Name: ...
ID: DFYT:TYDD:VDCC:YVO5:I7CY:LW6O:JRUB:OZ6O:LVLY:TENX:NPUG:IGST
Docker Root Dir: /home/docker
Debug Mode (client): false
Debug Mode (server): false
HTTP Proxy: ...
No Proxy: ...
Registry: https://index.docker.io/v1/
Labels:
Experimental: false
Insecure Registries:
 ...
 127.0.0.0/8
Live Restore Enabled: false

WARNING: devicemapper: usage of loopback devices is strongly discouraged for production use.
         Use `--storage-opt dm.thinpooldev` to specify a custom block storage device.

[shin-]

That's pretty much the expected behavior ; fortunately this can easily be solved by running docker-compose build --pull foo taz && docker-compose build bar instead.

[rvernica]

I see. Could --pull skip the images it cannot pull and continue with the build?

[shin-]

It could, but then you're running into different problems. What if foo isn't built before bar? Now you need to be able to define build-time dependencies, and probably more things as we keep digging to make the whole thing feel robust and coherent.

docker-compose isn't a build management tool, and it's mostly out of scope of what it's trying to do, so we'd rather avoid adding features like that altogether

[rvernica]

Yeah, good point. It seem that docker-compose builds the images in the order they appear in the .yml file (I guess this is not guaranteed, but it works for now). So, if I put dependent services after they parents, images are built in the correct order. Going one step further, I put all the dependent images at the end of the .yml file, so build --pull would do most of its work, except for failing at the end.