Testcontainers not working (Gitlab)

[marcelomorgado]

Hi there, I've spent the last few days trying to solve the issue below but I couldn't, can you help me?

My .gitlab-ci.yml file:

stages:
  - test

# DinD service is required for Testcontainers
services:
  - docker:dind
  - postgres:11.4

variables:
  PROFILE: "dev"
  # Instruct Testcontainers to use the daemon of DinD.
  DOCKER_HOST: "tcp://docker:2375"
  # Improve performance with overlayfs.
  DOCKER_DRIVER: overlay2
  DOCKER_TLS_CERTDIR: ""
  DATABASE_URL: postgresql://user:pass@postgres:5432/db
  POSTGRES_USER: user
  POSTGRES_DB: db
  POSTGRES_PASSWORD: pass

test:
  image: openjdk:8u212-jdk-stretch
  stage: test
  script:
    - apt-get update
    - apt-get install -y postgresql postgresql-contrib
    - psql -d $DATABASE_URL < ./docker/services/config/postgres/init.sql
    - ./gradlew test

My test superclass:

@Testcontainers
open class RepositoryTest {

    private val POSTGRES_IMAGE = "postgres:11.4"
    private val DB_NAME = "db"
    private val DB_USER = "user"
    private val DB_PASS = "pass"

    // Note: These files are symlink inside of the `test/resources` dir
    private val INIT_SCRIPT = "init.sql"
    private val PROPERTIES_FILE = "application.properties"


    private var ctx : DSLContext? = null

    private fun startPostgresSQL(): PostgreSQLContainer<Nothing> {
        val postgres = PostgreSQLContainer<Nothing>(POSTGRES_IMAGE).apply {
            withDatabaseName(DB_NAME)
            withUsername(DB_USER)
            withPassword(DB_PASS)
            withInitScript(INIT_SCRIPT)
        }
        postgres.start()
        return postgres
    }

    private fun runMigrations(jdbcUrl : String) {
        Flyway.configure().dataSource(jdbcUrl, DB_USER, DB_PASS).load().migrate()
    }

    fun getDSLContext(): DSLContext {
        return ctx!!
    }

    @Suppress("UNCHECKED_CAST")
    fun <T> getProp(key: String): T {
        val props  = javaClass.classLoader.getResourceAsStream(PROPERTIES_FILE).use {
            Properties().apply { load(it) }
        }
        return (props.getProperty(key) as T) ?: throw RuntimeException("could not find property $key")
    }

    @BeforeEach
    fun setUp() {

        val postgres = startPostgresSQL()
        runMigrations(postgres.jdbcUrl)

        ctx = DSL.using(postgres.jdbcUrl, DB_USER, DB_PASS)
    }
}

Exception received:

06:23:50.632 [DEBUG] [TestEventLogger] 
06:23:50.632 [DEBUG] [TestEventLogger] com.project.modules.user.queries.UserQueryRepositoryTest > should get a user and his roles by email and mid() FAILED
06:23:50.633 [DEBUG] [TestEventLogger]     org.testcontainers.containers.ContainerLaunchException: Container startup failed
06:23:50.633 [DEBUG] [TestEventLogger]         at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:333)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.testcontainers.containers.GenericContainer.start(GenericContainer.java:314)
06:23:50.633 [DEBUG] [TestEventLogger]         at com.project.RepositoryTest.startPostgresSQL(RepositoryTest.kt:34)
06:23:50.633 [DEBUG] [TestEventLogger]         at com.project.RepositoryTest.setUp(RepositoryTest.kt:57)
06:23:50.633 [DEBUG] [TestEventLogger]         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
06:23:50.633 [DEBUG] [TestEventLogger]         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
06:23:50.633 [DEBUG] [TestEventLogger]         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.lang.reflect.Method.invoke(Method.java:498)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:436)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.jupiter.engine.execution.ExecutableInvoker.invoke(ExecutableInvoker.java:115)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.invokeMethodInExtensionContext(ClassTestDescriptor.java:302)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.jupiter.engine.descriptor.ClassTestDescriptor.lambda$synthesizeBeforeEachMethodAdapter$12(ClassTestDescriptor.java:290)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeBeforeEachMethods$2(TestMethodTestDescriptor.java:135)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.jupiter.engine.execution.ThrowableCollector.execute(ThrowableCollector.java:40)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeMethodsOrCallbacksUntilExceptionOccurs(TestMethodTestDescriptor.java:155)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeBeforeEachMethods(TestMethodTestDescriptor.java:134)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:109)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:58)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:112)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$2(HierarchicalTestExecutor.java:120)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.Iterator.forEachRemaining(Iterator.java:116)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:120)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$2(HierarchicalTestExecutor.java:120)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.Iterator.forEachRemaining(Iterator.java:116)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:482)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.lambda$executeRecursively$3(HierarchicalTestExecutor.java:120)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.engine.support.hierarchical.SingleTestExecutor.executeSafely(SingleTestExecutor.java:66)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.executeRecursively(HierarchicalTestExecutor.java:108)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor$NodeExecutor.execute(HierarchicalTestExecutor.java:79)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:55)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:43)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:170)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:154)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:90)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:92)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$100(JUnitPlatformTestClassProcessor.java:77)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:73)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:61)
06:23:50.633 [DEBUG] [TestEventLogger]         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
06:23:50.633 [DEBUG] [TestEventLogger]         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
06:23:50.633 [DEBUG] [TestEventLogger]         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.lang.reflect.Method.invoke(Method.java:498)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:32)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:93)
06:23:50.633 [DEBUG] [TestEventLogger]         at com.sun.proxy.$Proxy2.stop(Unknown Source)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.gradle.api.internal.tasks.testing.worker.TestWorker.stop(TestWorker.java:123)
06:23:50.633 [DEBUG] [TestEventLogger]         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
06:23:50.633 [DEBUG] [TestEventLogger]         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
06:23:50.633 [DEBUG] [TestEventLogger]         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.lang.reflect.Method.invoke(Method.java:498)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:35)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:155)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.gradle.internal.remote.internal.hub.MessageHubBackedObjectConnection$DispatchWrapper.dispatch(MessageHubBackedObjectConnection.java:137)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.gradle.internal.remote.internal.hub.MessageHub$Handler.run(MessageHub.java:404)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
06:23:50.633 [DEBUG] [TestEventLogger]         at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
06:23:50.633 [DEBUG] [TestEventLogger]         at java.lang.Thread.run(Thread.java:748)
06:23:50.633 [DEBUG] [TestEventLogger] 
06:23:50.633 [DEBUG] [TestEventLogger]         Caused by:
06:23:50.633 [DEBUG] [TestEventLogger]         org.testcontainers.containers.ContainerFetchException: Can't get Docker image: RemoteDockerImage(imageName=postgres:11.4, imagePullPolicy=DefaultPullPolicy())
06:23:50.633 [DEBUG] [TestEventLogger]             at org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:1284)
06:23:50.633 [DEBUG] [TestEventLogger]             at org.testcontainers.containers.GenericContainer.logger(GenericContainer.java:618)
06:23:50.633 [DEBUG] [TestEventLogger]             at org.testcontainers.containers.GenericContainer.doStart(GenericContainer.java:323)
06:23:50.633 [DEBUG] [TestEventLogger]             ... 85 more
06:23:50.633 [DEBUG] [TestEventLogger] 
06:23:50.633 [DEBUG] [TestEventLogger]             Caused by:
06:23:50.633 [DEBUG] [TestEventLogger]             java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration
06:23:50.633 [DEBUG] [TestEventLogger]                 at org.testcontainers.dockerclient.DockerClientProviderStrategy.lambda$getFirstValidStrategy$7(DockerClientProviderStrategy.java:214)
06:23:50.633 [DEBUG] [TestEventLogger]                 at java.util.Optional.orElseThrow(Optional.java:290)
06:23:50.633 [DEBUG] [TestEventLogger]                 at org.testcontainers.dockerclient.DockerClientProviderStrategy.getFirstValidStrategy(DockerClientProviderStrategy.java:206)
06:23:50.633 [DEBUG] [TestEventLogger]                 at org.testcontainers.DockerClientFactory.getOrInitializeStrategy(DockerClientFactory.java:131)
06:23:50.633 [DEBUG] [TestEventLogger]                 at org.testcontainers.DockerClientFactory.client(DockerClientFactory.java:170)
06:23:50.633 [DEBUG] [TestEventLogger]                 at org.testcontainers.LazyDockerClient.getDockerClient(LazyDockerClient.java:14)
06:23:50.633 [DEBUG] [TestEventLogger]                 at org.testcontainers.LazyDockerClient.listImagesCmd(LazyDockerClient.java:12)
06:23:50.633 [DEBUG] [TestEventLogger]                 at org.testcontainers.images.LocalImagesCache.maybeInitCache(LocalImagesCache.java:68)
06:23:50.633 [DEBUG] [TestEventLogger]                 at org.testcontainers.images.LocalImagesCache.get(LocalImagesCache.java:32)
06:23:50.633 [DEBUG] [TestEventLogger]                 at org.testcontainers.images.AbstractImagePullPolicy.shouldPull(AbstractImagePullPolicy.java:18)
06:23:50.633 [DEBUG] [TestEventLogger]                 at org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:65)
06:23:50.633 [DEBUG] [TestEventLogger]                 at org.testcontainers.images.RemoteDockerImage.resolve(RemoteDockerImage.java:26)
06:23:50.633 [DEBUG] [TestEventLogger]                 at org.testcontainers.utility.LazyFuture.getResolvedValue(LazyFuture.java:17)
06:23:50.633 [DEBUG] [TestEventLogger]                 at org.testcontainers.utility.LazyFuture.get(LazyFuture.java:39)
06:23:50.633 [DEBUG] [TestEventLogger]                 at org.testcontainers.containers.GenericContainer.getDockerImageName(GenericContainer.java:1282)
06:23:50.633 [DEBUG] [TestEventLogger]                 ... 87 more

Note: I'm using postgres as Gitlab service because it's used by jOOQ to build the table classes.

Thanks

[vcvitaly]

If it's still relevant, could you attach more logs?

[marcelomorgado]

Any specifics? I've pasted all stack of the exception that is thrown.

[marcelomorgado]

Would be nice if you can reproduce the same scenario (what I've shared is enough?).

[marcelomorgado]

Hi @vcvitaly @rnorth @bsideup any clue in how can I solve this?
Thanks!

[rnorth]

Sorry, we'd probably need to see the logs from much earlier than that. Could you show everything from the first testcontainers log line you see? e.g. the first INFO level log line I see from Testcontainers is INFO org.testcontainers.dockerclient.DockerClientProviderStrategy - Loaded org.testcontainers.dockerclient.UnixSocketClientProviderStrategy from ~/.testcontainers.properties, will try it first. Anything starting org.testcontainers would be helpful.

We may not be able to help much as we don't use Gitlab ourselves, but perhaps other users could suggest what might be going wrong.

[rnorth]

Also, what happens if you put a plain docker ps command into your build before the tests are run?

[marcelomorgado]

Hi @rnorth, thanks for the response.
The error is quite silent, the log above is all that I got even using the most verbose setup.
I received this error docker: command not found when trying run docker ps command. Should the docker bin available for testcontainers to work properly?

[mz7mz7mz7]

Hello everyone here -- is there any progress on this ticket, please? We are still facing the same issue.

[rnorth]

@marcelomorgado you'd need to run Gradle with the -i flag to see useful logs...

[kiview]

Should the docker bin available for testcontainers to work properly?

This is not necessary, Testcontainers communicates directly with Docker.

Which Gitlab and Runner version are you using? There have been some changes with regards to DinD in the latest versions, see:
https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#use-the-docker-executor-with-the-docker-image-docker-in-docker

Also, can you verify that DinD works without Testcontainers? Please verify this by running a job similar to the examples from the Gitlab documentation, that runs inside a Docker image, thereby giving you access to the Docker binary:
https://docs.gitlab.com/ee/ci/docker/using_docker_build.html#docker-in-docker-with-tls-disabled

[hfhbd]

If the docker command is not available, how do you login to your private registry?
I tried this, but the content of the config.json is not used, I always get access denied.

services:
  - docker:dind

variables:
  # Instruct Testcontainers to use the daemon of DinD.
  DOCKER_HOST: "tcp://docker:2375"
  # Instruct Docker not to start over TLS.
  DOCKER_TLS_CERTDIR: ""
  # Improve performance with overlayfs.
  DOCKER_DRIVER: overlay2

test:
  image: openjdk:11
  stage: test
  before_script:
    - export login=$(base64 <<< "$CI_REGISTRY_USER:$CI_REGISTRY_PASSWORD")
    - echo "{\"auths\":{\"${CI_REGISTRY}\":{\"auth\":\"$login\"}}}" > ~/.docker/config.json
    - cat ~/.docker/config.json
  script:
    - ./gradlew --build-cache -d check

[kiview]

Testcontainers uses different environment-dependent strategies to resolve credentials, one of them being resolving of the config.json file. For more details, check AuthDelegatingDockerClientConfig.

How did you verify that the config is not used by Testcontainers in your case? Consider configuring appropriate log levels. Testcontainers will log which auth config was used.

[hfhbd]

@kiview Thank you for your fast response.
Using log level debug I don't see any usage of ~/.docker/config.json, but the file exists, even checked using this test:

@Test
fun a() {
    val file = File("${System.getProperty("user.home")}/.docker/config.json")
    println("${file.path} exists: ${file.exists()} and contains: ${file.readText()}")
    val imageName = DockerImageName.parse("registry.gitlab.foo.de/image")
    val tsContainer = GenericContainer<Nothing>(imageName)
    tsContainer.start()
}

debug log file using /.gradlew -d test:

/root/.docker/config.json exists: true and contains: {"auths":{"registry.gitlab.foo.de":{"auth":"someToken"}}}
17:51:42.495 [Test worker @coroutine#1] WARN  o.t.u.TestcontainersConfiguration - Attempted to read Testcontainers configuration file at file:/root/.testcontainers.properties but the file was not found. Exception message: FileNotFoundException: /root/.testcontainers.properties (No such file or directory)
17:51:42.519 [Test worker @coroutine#1] INFO  o.t.d.DockerMachineClientProviderStrategy - docker-machine executable was not found on PATH ([/usr/local/openjdk-11/bin, /usr/local/sbin, /usr/local/bin, /usr/sbin, /usr/bin, /sbin, /bin])
17:51:43.468 [Test worker @coroutine#1] INFO  o.t.d.DockerClientProviderStrategy - Found Docker environment with Environment variables, system properties and defaults. Resolved dockerHost=tcp://docker:2375
17:51:43.477 [Test worker @coroutine#1] INFO  o.testcontainers.DockerClientFactory - Docker host IP address is docker
17:51:43.530 [Test worker @coroutine#1] INFO  o.testcontainers.DockerClientFactory - Connected to docker: 
Server Version: 20.10.7
API Version: 1.41
Operating System: Alpine Linux v3.13 (containerized)
Total Memory: 16012 MB
17:51:43.535 [Test worker @coroutine#1] INFO  o.t.utility.ImageNameSubstitutor - Image name substitution will be performed by: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor')
17:51:45.503 [docker-java-stream-1336992916] INFO  o.testcontainers.DockerClientFactory - Starting to pull image
17:51:45.548 [docker-java-stream-1336992916] INFO  o.testcontainers.DockerClientFactory - Pulling image layers:  0 pending,  0 downloaded,  0 extracted, (0 bytes/0 bytes)
17:51:46.541 [docker-java-stream-1336992916] INFO  o.testcontainers.DockerClientFactory - Pulling image layers:  2 pending,  1 downloaded,  0 extracted, (27 KB/? MB)
17:51:46.568 [docker-java-stream-1336992916] INFO  o.testcontainers.DockerClientFactory - Pulling image layers:  1 pending,  2 downloaded,  0 extracted, (55 KB/? MB)
17:51:46.628 [docker-java-stream-1336992916] INFO  o.testcontainers.DockerClientFactory - Pulling image layers:  0 pending,  3 downloaded,  0 extracted, (55 KB/5 MB)
17:51:46.795 [docker-java-stream-1336992916] INFO  o.testcontainers.DockerClientFactory - Pulling image layers:  0 pending,  3 downloaded,  1 extracted, (2 MB/5 MB)
17:51:46.903 [docker-java-stream-1336992916] INFO  o.testcontainers.DockerClientFactory - Pulling image layers:  0 pending,  3 downloaded,  2 extracted, (2 MB/5 MB)
17:51:47.009 [docker-java-stream-1336992916] INFO  o.testcontainers.DockerClientFactory - Pulling image layers:  0 pending,  3 downloaded,  3 extracted, (5 MB/5 MB)
17:51:47.021 [docker-java-stream-1336992916] INFO  o.testcontainers.DockerClientFactory - Pull complete. 3 layers, pulled in 1s (downloaded 5 MB at 5 MB/s)
17:51:47.842 [Test worker @coroutine#1] INFO  o.testcontainers.DockerClientFactory - Ryuk started - will monitor and terminate Testcontainers containers on JVM exit
17:51:47.842 [Test worker @coroutine#1] INFO  o.testcontainers.DockerClientFactory - Checking the system...
17:51:47.843 [Test worker @coroutine#1] INFO  o.testcontainers.DockerClientFactory - ✔︎ Docker server version should be at least 1.6.0
17:51:48.146 [Test worker @coroutine#1] INFO  o.testcontainers.DockerClientFactory - ✔︎ Docker environment should have more than 2GB free disk space
17:55:48.316 [Test worker] INFO  ?.g.f.de/image:latest] - Pulling docker image: registry.gitlab.f.de/image:latest. Please be patient; this may take some time but only needs to be done once.
17:55:48.378 [docker-java-stream--650640895] ERROR c.g.d.a.async.ResultCallbackTemplate - Error during callback
com.github.dockerjava.api.exception.InternalServerErrorException: Status 500: {"message":"Head https://registry.gitlab.foo.de/image/manifests/latest: unauthorized: HTTP Basic: Access denied"}
	at org.testcontainers.shaded.com.github.dockerjava.core.DefaultInvocationBuilder.execute(DefaultInvocationBuilder.java:247)

[hfhbd]

@kiview FYI: My problem is finally fixed, the login using the $CI_REGISTRY_USER and $CI_REGISTRY_PASSWORD had no access to the registry... I created a new token and it works like a charm! Thanks for the support and of course the working library.

[kiview]

That's great to hear, thanks a lot for sharing your solution!