Could not find a valid Docker environment on MBP M1 (Apple Silicon)

[landsman]

Hello,

we are using this integration and it's amazing! Thank you for it.

We are slowly migrating from Linux to MBP M1 (Apple Silicon), and facing a problem with Docker settings. I would like to ask you for advice, what can I change. Do you have some experience with this?

Env:

a) Docker installed from the link: https://docs.docker.com/docker-for-mac/apple-silicon/

b) docker working in the terminal
c) dependencies:

// database, migrations
implementation("org.postgresql:postgresql:42.2.18")
implementation("org.liquibase:liquibase-core:3.10.3")
implementation("org.hibernate:hibernate-envers:5.4.6.Final")
implementation("org.springframework.boot:spring-boot-starter-data-jpa:2.2.0.RELEASE")

// testing, documentation
testImplementation("org.springframework.restdocs:spring-restdocs-mockmvc:2.0.4.RELEASE")
testImplementation("org.mockito:mockito-core:2.25.0")
testImplementation("io.zonky.test:embedded-database-spring-test:2.0.0")
testImplementation("org.junit.jupiter:junit-jupiter-api:5.4.0")
testRuntimeOnly("org.junit.jupiter:junit-jupiter-engine:5.4.0")

d) firewall is turn off, no vpn etc

---

Exception after I try to run tests:

Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.orm.jpa.LocalContainerEntityManagerFactoryBean]: Factory method 'entityManagerFactory' threw exception; nested exception is io.zonky.test.db.shaded.com.google.common.util.concurrent.UncheckedExecutionException: io.zonky.test.db.provider.ProviderException: Unexpected error when preparing a database cluster; nested exception is java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration

[tomix26]

Hi @landsman, thanks for the report.

I found the following issue testcontainers/testcontainers-java#3834 in the Testcontainers project that we use to integrate with Docker. Could you please take a look at it and let me know if it helped you?

[landsman]

@tomix26 Thank you for the link. I add this mentioned dependency but it had no effect on this error.
implementation("com.amazonaws:aws-java-sdk:1.11.632")

[tomix26]

@landsman I was facing the same issue and adding the net.java.dev.jna:jna:5.7.0 dependency fixed the error.

[ahezzati]

I have the same issue, when I run the test in IntelliJ it works and when I start SpringBoot Application in IntelliJ it works too but when I run my Docker it fails, I added jna dependency but it didn't fix the issue

Here is the stacktrace:
Caused by: io.zonky.test.db.shaded.com.google.common.util.concurrent.UncheckedExecutionException: io.zonky.test.db.provider.ProviderException: Unexpected error when preparing a database cluster; nested exception is java.lang.IllegalStateException: Could not find a valid Docker environment. Please see logs and check configuration at io.zonky.test.db.shaded.com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1571) ~[embedded-database-spring-test-2.0.1.jar:na] at io.zonky.test.db.shaded.com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1556) ~[embedded-database-spring-test-2.0.1.jar:na] at io.zonky.test.db.context.DefaultDatabaseContext.awaitDatabase(DefaultDatabaseContext.java:282) ~[embedded-database-spring-test-2.0.1.jar:na] at io.zonky.test.db.context.DefaultDatabaseContext.refreshDatabase(DefaultDatabaseContext.java:253) ~[embedded-database-spring-test-2.0.1.jar:na] at io.zonky.test.db.context.DefaultDatabaseContext.getDatabase(DefaultDatabaseContext.java:115) ~[embedded-database-spring-test-2.0.1.jar:na] at io.zonky.test.db.context.DatabaseTargetSource.getTarget(DatabaseTargetSource.java:32) ~[embedded-database-spring-test-2.0.1.jar:na] at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:192) ~[spring-aop-5.2.8.RELEASE.jar:5.2.8.RELEASE] at com.sun.proxy.$Proxy105.getConnection(Unknown Source) ~[na:na] at org.springframework.jdbc.datasource.DataSourceTransactionManager.doBegin(DataSourceTransactionManager.java:261) ~[spring-jdbc-5.2.8.RELEASE.jar:5.2.8.RELEASE] ... 54 common frames omitted

Did I miss something in the configurations? Do I need to update my Docker file?

BTW, I use multistage docker, so I run mvn build inside my container.

[tomix26]

@ahezzati I guess your problem is caused by running the build in a docker container in conjunction with using the docker provider. In that case you are actually starting a docker container in another docker container, which is possible but it requires additional configuration. See the instructions here: https://github.com/zonkyio/embedded-database-spring-test#using-docker-provider-inside-a-docker-container

Or the other option is to use a different provider, such as the zonky provider: https://github.com/zonkyio/embedded-database-spring-test#using-zonky-provider-previous-default

[ahezzati]

Thanks @tomix26 for the response and the links. I believe it won't work for my case because we user multistage docker where we run mvn install during Docker build image and default provider that uses TestContainers expect running container which is not possible during build stage, and even when I tried (Zonky and OpenTable) both failed with the same erro Unexpected error when preparing a database cluster; nested exception is java.lang.IllegalStateException: Process [/tmp/embedded-pg/PG-311c3133a5004e71edd221492f23eb46/bin/initdb, -A, trust, -U, postgres, -D, /tmp/epg9279388697672865348, -E, UTF-8] failed I believe this is due to the same issue, I am trying to run it during build stage not container running stage.

[tomix26]

Yes, you're right. It seems that running a docker container during docker build is not supported right now. However, the other provides should work. So could you please provide a logfile with more details for further investigation?

[ahezzati]

My bad, actually I was wrong, when I changed to Zonky provider I needed to run as non-root user as mentioned in the documentation, once I did it fixed the issue and integration test is working fine now

My Dockerfile before

FROM maven:3.6.3-adoptopenjdk-11 AS MAVEN
COPY settings.xml /usr/share/maven/ref/settings-docker.xml
COPY pom.xml /tmp/
WORKDIR /tmp/
RUN mvn -s /usr/share/maven/ref/settings-docker.xml dependency:go-offline

FROM MAVEN AS COMPILE
COPY src /tmp/src
WORKDIR /tmp/
RUN mvn -s /usr/share/maven/ref/settings-docker.xml verify

My Dockerfile after

FROM maven:3.6.3-adoptopenjdk-11 AS MAVEN
COPY settings.xml /usr/share/maven/ref/settings-docker.xml
COPY pom.xml /tmp/
WORKDIR /tmp/
RUN mvn -s /usr/share/maven/ref/settings-docker.xml dependency:go-offline

FROM MAVEN AS COMPILE
RUN adduser --system --group javauser
COPY src /tmp/src
WORKDIR /tmp/
RUN chown -R javauser:javauser /tmp/
USER javauser
RUN mvn -s /usr/share/maven/ref/settings-docker.xml verify

That's good news for mutlistage Docker and Docker in Docker, thanks @tomix26 for your help and response

[landsman]

@landsman I was facing the same issue and adding the net.java.dev.jna:jna:5.7.0 dependency fixed the error.

This is working! Thank you!
Would you please add this dependency to the library it self? To avoid problems for others 🙏

[tomix26]

I will consider it, thanks for the feedback.

[thejeff77]

Seems that I still have this issue. The thread linked above says to upgrade testcontainers to 1.16 and to use the jna dependency.

I'd recommend this project bump the testcontainers version soon and add the jna as well for compatibility.

Ref: testcontainers/testcontainers-java#3834 (comment)

I'm using docker-cli with the colima runtime, and it has this same build error.