Always pulling images even when they're locally available

[sazzer]

I've noticed that the TestContainers code is always attempting to pull images, even when they are locally available. This obviously makes builds slower, and can cause problems when working without internet access.

The problem seems to be in RemoteDockerImage.resolve(), though I've not worked out any more than that yet.

[sazzer]

In particular, it seems to be around this code:

            List<Image> updatedImages = listImagesCmd.exec();
            updatedImages.stream()
                .map(Image::getRepoTags)
                .filter(Objects::nonNull)
                .flatMap(Stream::of)
                .map(DockerImageName::new)
                .collect(Collectors.toCollection(() -> AVAILABLE_IMAGE_NAME_CACHE));

The updatedImages collection does contain the image that I'm working with, but after this the AVAILABLE_IMAGE_NAME_CACHE doesn't get updated to have anything.

However, when AVAILABLE_IMAGE_NAME_CACHE is added to lower down then that does persist and does work correctly, so it seems to literally be the above code not updating the collection correctly.

[AndyMacDroo]

Quite strange. Can you confirm the image you're trying to pull? Might also be worth providing your docker version and OS.

With my local testing I can see that the AVAILABLE_IMAGE_NAME_CACHE gets correctly populated for me when I try with a simple postgres image.

For ref here's my setup:

andy@andy-pc:~$ docker -v
Docker version 18.09.6, build 481bc77

andy@andy-pc:~$ lsb_release -a
No LSB modules are available.
Distributor ID:	Ubuntu
Description:	Ubuntu 19.04
Release:	19.04
Codename:	disco

[sazzer]

I can get exact versions later, but:

• macOS - latest
• Docker for Desktop - latest for macOS
• Java - OpenJDK 13
• Containers: postgres:11.5-alpine, selenium, vncrecorder and sazzer/fakeauth:latest
• Testcontainers - 1.12.2

My app is also written in Kotlin and builds with Maven, but I don't see that being relevant.

Cheers

[sazzer]

More version info:

-> % docker version
Client: Docker Engine - Community
 Version:           19.03.2
 API version:       1.40
 Go version:        go1.12.8
 Git commit:        6a30dfc
 Built:             Thu Aug 29 05:26:49 2019
 OS/Arch:           darwin/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.2
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.8
  Git commit:       6a30dfc
  Built:            Thu Aug 29 05:32:21 2019
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          v1.2.6
  GitCommit:        894b81a4b802e4eb2a91d1ce216b8817763c29fb
 runc:
  Version:          1.0.0-rc8
  GitCommit:        425e105d5a03fabd737a126ad93d62a9eeede87f
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

-> % mvn -v
executing mvnw instead of mvn
Apache Maven 3.6.0 (97c98ec64a1fdfee7767ce5ffb20918da4f719f3; 2018-10-24T19:41:47+01:00)
Maven home: /Users/coxg/.m2/wrapper/dists/apache-maven-3.6.0-bin/2dakv70gp803gtm5ve1ufmvttn/apache-maven-3.6.0
Java version: 13, vendor: Oracle Corporation, runtime: /Users/coxg/.jabba/jdk/openjdk@1.13.0/Contents/Home
Default locale: en_GB, platform encoding: UTF-8
OS name: "mac os x", version: "10.14.6", arch: "x86_64", family: "mac"

[AndyMacDroo]

Are you able to run this with TRACE logging enabled?

I think you can do this by building and running your tests with a logback.xml file similar to the following in your resources folder:

<configuration>

    <appender name="STDOUT" class="ch.qos.logback.core.ConsoleAppender">
        <!-- encoders are assigned the type
             ch.qos.logback.classic.encoder.PatternLayoutEncoder by default -->
        <encoder>
            <pattern>%d{HH:mm:ss.SSS} %-5level %logger - %msg%n</pattern>
        </encoder>
    </appender>

    <root level="TRACE">
        <appender-ref ref="STDOUT"/>
    </root>

    <logger name="org.testcontainers" level="TRACE"/>

    <logger name="com.github.dockerjava" level="TRACE"/>
    <logger name="org.zeroturnaround.exec" level="TRACE"/>
    <logger name="org.testcontainers.shaded" level="TRACE"/>

</configuration>

[sazzer]

I don't quite know what I've done, but it's no longer doing it...

I'd been trying to get a local build of TestContainers that I could depend on - and not worked out how to get my Maven build to depend on a Gradle project :\ - but I've got zero changes outstanding and now it's not trying to pull images every time.

I'll keep an eye on it to see if it happens again, but for now it seems to have resolved itself somehow.

[AndyMacDroo]

No worries - possibly something environmental in your docker setup? If it crops up again then message back 👍

[rnorth]

#1345 might have an effect on this, or at least give options should you find a way to reproduce again.

I'll close this ticket for now, but of course if you do find this happening consistently again please feel free to reopen.

BTW, thanks @AndyMacDroo for your input here 🙇