Testcontainers integration fails on Docker 29.0.0

[ThomasVitale]

As discussed in #48050, Docker 29.0.0 introduced a breaking change that cause the Buildpacks integration in Spring Boot to fail. That has been fixed.

There is also a problem with the Testcontainers integration. Unfortunately, the Testcontainers project seems not to be releasing a fix for the 1.x release train, only for 2.x (as explained in testcontainers/testcontainers-java#11212 (comment)). That means the integration in Spring Boot 3.5.x will not work (wheres it will work in Spring Boot 4.x which follows the Testcontainers 2.x release train, assuming the patch is released before Spring Boot 4.0.0 is released).

I'm aware the issue is not in Spring Boot, but in Testcontainers. But if they don't release a fix for 1.x, I thought of suggesting having a workaround in Spring Boot 3.5.x.

It seems like the issue is solved if setting this system property when using Testcontainers:

System.setProperty("api.version", "1.44");

That could be set temporarily when running Testcontainers, as suggested here: testcontainers/testcontainers-java#11212 (comment)

If it's not something desired to be addressed in Spring Boot, I would like to suggest at least documenting this problem and the workaround.

Thanks a lot!

[wilkinsona]

@eddumelendez would you consider doing a final Testcontainers 1.x patch release to tolerate the breaking change in Docker? No worries at all if not, we'll figure out what to do on the Spring Boot side for 3.x.

[eddumelendez]

Hi @wilkinsona, no release for version 1.x unfortunately.

[philwebb]

There's not a great way for us to fix this assuming that we don't want to upgrade to Testcontainers 2 in Boot 3.x. Users can add docker-java.properties to their project and specify the version, but that's a real pain.

The best option I've found so far is to abuse the DockerClientProviderStrategy service loading code to do the upgrade behind the scenes. A prototype is in https://github.com/philwebb/spring-boot/tree/gh-48104.

The advantage is that you now only need a docker-java.properties if you don't want to automatically upgrade. The downside is a new public class.

[philwebb]

Flagging to see what the team think of my "fix". I'll add tests if we want to continue in this direction.

[bmorris591]

The downstream library - Docker Java - has patched this. So I think you can just bump that library in Spring Boot and that should address the issue.

https://github.com/docker-java/docker-java/releases/tag/3.7.0

[wilkinsona]

Thanks for the suggestion, @bmorris591. Unfortunately, we can't do that as we don't manage the version of Docker Java. We also wouldn't want to upgrade to a new minor version of it in a Spring Boot maintenance release.

[vpavic]

For those affected, if you control environment Docker daemon runs in, perhaps the easiest workaround is to set DOCKER_MIN_API_VERSION=1.24 env variable, as mentioned in Docker Engine v29 Release blog post.

This way you don't have to apply workarounds for each piece of tooling (libs like Testcontainers, your IDE, and so on) broken by minimum API version update, and gives you some time until the broader ecosystem to adapts to this change.

[bmorris591]

There's not a great way for us to fix this assuming that we don't want to upgrade to Testcontainers 2 in Boot 3.x. Users can add docker-java.properties to their project and specify the version, but that's a real pain.

The best option I've found so far is to abuse the DockerClientProviderStrategy service loading code to do the upgrade behind the scenes. A prototype is in https://github.com/philwebb/spring-boot/tree/gh-48104.

The advantage is that you now only need a docker-java.properties if you don't want to automatically upgrade. The downside is a new public class.

My main issue with this is that System.setProperty is obviously JVM global - whilst I imagine there is a way to undo the change after the test is finished, it's still prone to races and bugs in concurrent test scenarios etc.

Maybe there's a hook somewhere in Docker Java to inject config somehow?

[philwebb]

The downstream library - Docker Java - has patched this. So I think you can just bump that library in Spring Boot and that should address the issue.

...

we don't manage the version of Docker Java

It's even more complicated than that I'm afraid since Docker Java is shaded into testcontainers. We can't just bump Docker Java.

[philwebb]

Maybe there's a hook somewhere in Docker Java to inject config somehow?

I couldn't find an obvious one. The version is set in a static method in DockerClientProviderStrategy when DefaultDockerClientConfig.Builder doesn't provide one. I don't think we can hook anything in there and there are multiple subclasses of DockerClientProviderStrategy that can call that method.

The DefaultDockerClientConfig supports a number of strategies to configure builder, but none are pluggable.

[philwebb]

My main issue with this is that System.setProperty is obviously JVM global - whilst I imagine there is a way to undo the change after the test is finished, it's still prone to races and bugs in concurrent test scenarios etc.

That's a valid concern, made a little more problematic by the system property name which doesn't even include docker in it.

My guess is that in most situations you wont want a different API version once it's been set so pollution isn't so much of a concern.

It's a hard call to make with this problem because I'd like it if most users could just continue to have their apps work when Docker Desktop upgrades itself. We could always do nothing in Boot and recommend that users adopt one of the existing workarounds. I think I prefer trying to do something so that 90% of users don't notice the problem and those that do have test pollution issues opt-out of the auto fix.

[GeorgEchterling]

The upstream issue from docker-java also affects the Spring Boot Gradle plugin, not just Testcontainers:

Execution failed for task ':bootBuildImage'.
> Docker API call to '/var/run/docker.sockdocker:///var/run/docker.sock/v1.24/images/create?fromImage=docker.io%2Fpaketobuildpacks%2Fbuilder-noble-java-tiny%3Alatest' failed with status code 400 "Bad Request" and message "client version 1.24 is too old. Minimum supported API version is 1.44, please upgrade your client to a newer version"

(I didn't see anyone mention this. Sorry if this has already been reported elsewhere.)