Timed out waiting for Ryuk container to start - Ubuntu 20.04.3, Docker 20.10.12, 1.16.3

[pieninck]

Hi,
I am using Webdriver Containers for Selenium test.

When the tests are running on my local machine everything works fine.

However when I run the same tests on a jenkins agent following errors are thrown:

00:12:41.410 [main] INFO  o.t.d.DockerClientProviderStrategy - Loaded org.testcontainers.dockerclient.UnixSocketClientProviderStrategy from ~/.testcontainers.properties, will try it first
00:12:41.621 [main] INFO  o.t.d.DockerClientProviderStrategy - Found Docker environment with local Unix socket (unix:///var/run/docker.sock)
00:12:41.622 [main] INFO  o.testcontainers.DockerClientFactory - Docker host IP address is localhost
00:12:41.639 [main] INFO  o.testcontainers.DockerClientFactory - Connected to docker:
  Server Version: 20.10.12
  API Version: 1.41
  Operating System: Ubuntu 20.04.3 LTS
  Total Memory: 64220 MB
00:12:41.641 [main] INFO  o.t.utility.ImageNameSubstitutor - Image name substitution will be performed by: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor')
00:13:12.080 [main] ERROR o.t.utility.ResourceReaper - Timed out waiting for Ryuk container to start. Ryuk's logs:
2022/01/27 00:12:42 Pinging Docker...
2022/01/27 00:12:42 Docker daemon is available!
2022/01/27 00:12:42 Starting on port 8080...
2022/01/27 00:12:42 Started! 

As of the start of each test following error is thrown:

FAILED: java.lang.IllegalStateException: Could not connect to Ryuk at localhost:49210

Specification of the server:
Ubuntu 20.04.3
Docker 20.10.12
Testcontainers 1.16.3
OpenJDK 16.0.1

It seems that the container starts correctly:

CONTAINER ID   IMAGE                       COMMAND   CREATED         STATUS         PORTS                                         NAMES
3f263aa0e454   testcontainers/ryuk:0.3.3   "/app"    5 seconds ago   Up 4 seconds   0.0.0.0:49209->8080/tcp, :::49209->8080/tcp   testcontainers-ryuk-3cb3f86e-546d-4ae0-9d37-c0ce13659bf1

Disabling ryuk with TESTCONTAINERS_RYUK_DISABLED=true doesn't` solve the problem. When disabling ryuk selenium and vnc-recorder are started but there are also errors thrown:

00:35:10.486 [main] INFO  o.t.d.DockerClientProviderStrategy - Loaded org.testcontainers.dockerclient.UnixSocketClientProviderStrategy from ~/.testcontainers.properties, will try it first
00:35:10.697 [main] INFO  o.t.d.DockerClientProviderStrategy - Found Docker environment with local Unix socket (unix:///var/run/docker.sock)
00:35:10.698 [main] INFO  o.testcontainers.DockerClientFactory - Docker host IP address is localhost
00:35:10.715 [main] INFO  o.testcontainers.DockerClientFactory - Connected to docker:
  Server Version: 20.10.12
  API Version: 1.41
  Operating System: Ubuntu 20.04.3 LTS
  Total Memory: 64220 MB
00:35:10.715 [main] INFO  o.testcontainers.DockerClientFactory - Checking the system...
00:35:10.715 [main] INFO  o.testcontainers.DockerClientFactory - ✔︎ Docker server version should be at least 1.6.0
00:35:10.717 [main] INFO  o.t.utility.ImageNameSubstitutor - Image name substitution will be performed by: DefaultImageNameSubstitutor (composite of 'ConfigurationFileImageNameSubstitutor' and 'PrefixingImageNameSubstitutor')
00:35:11.193 [main] INFO  o.testcontainers.DockerClientFactory - ✔︎ Docker environment should have more than 2GB free disk space
00:35:11.537 [main] INFO  o.t.containers.SeleniumUtils - Selenium API version 3.141.59 detected on classpath
00:35:11.553 [main] INFO  ?.141.59] - Creating container for image: selenium/standalone-chrome-debug:3.141.59
00:35:12.372 [main] INFO  ?.141.59] - Container selenium/standalone-chrome-debug:3.141.59 is starting: c8d62a89d143f988ea7d82d21517583aa84fb0d0b635eef4c4a1d4549e0d6ed8
00:35:14.950 [main] INFO  ?.141.59] - Container selenium/standalone-chrome-debug:3.141.59 started in PT3.404032216S
00:35:14.951 [main] INFO  ?.2.0] - Creating container for image: testcontainers/vnc-recorder:1.2.0
00:35:14.976 [main] INFO  ?.2.0] - Container testcontainers/vnc-recorder:1.2.0 is starting: 223d9db7a3401544ae0ea9d624f849d2c36bf7ffe18367daa9e8139df8511d25
00:35:15.330 [main] INFO  ?.2.0] - Container testcontainers/vnc-recorder:1.2.0 started in PT0.379303582S

org.rnorth.ducttape.TimeoutException: Timeout waiting for result with exception
Caused by: java.lang.RuntimeException:
org.openqa.selenium.remote.UnreachableBrowserException: Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03'
System info: host: 'hostname', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-96-generic', java.version: '16.0.1'
Driver info: driver.version: RemoteWebDriver
Caused by: org.openqa.selenium.remote.UnreachableBrowserException:
Could not start a new session. Possible causes are invalid address of the remote server or browser start-up failure.
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03'
System info: host: 'hostname', ip: '127.0.1.1', os.name: 'Linux', os.arch: 'amd64', os.version: '5.4.0-96-generic', java.version: '16.0.1'
Driver info: driver.version: RemoteWebDriver
Caused by: java.net.SocketException: Connection reset

Any ideas what could be the cause?
Thanks in advance for the support!

[ger-benjamin]

Hi,
Today I ran into exactly the same issue, but on my own machine (on Ubuntu 20.04.03). To solve that, I've... simply restarted docker - Following the idea here: #3609. Maybe one of these solution could help you.

But it would be nice if the testcontainers teams could do something to fix this issue. If they can reproduce it... It looks happening to some people regularly.

[kiview]

The issue described in #3609 is a different one and is related to Docker for Windows.

The issue described here seems to be related to the specific CI and networking setup, something where it is hard for us to give generic advice without understanding the CI infrastructure of the user better.

@pieninck Since TESTCONTAINERS_RYUK_DISABLED=true also does not help, it seems to me that this is a networking or firewall issue. Does this work on your CI if run as part of the Jenkins job?

docker run -d -p 9090:80 httpd
curl localhost:9090

[qingwli]

Hi, I have encountered this problem before. And after research, I finally fix this problem.

First, I have use Jenkins Clouds model. All new task will create a pod on K8s cluster and are destroyed when the task is complete.

And the solution is enable Host Network in Jenkins, the way is configureClouds -> Pod Template details... and it works.

[luchomarfil]

Hi, in my case my VPN connection overlap the docker bip network (172.17.0.0/16).
I just move docker bip to anothar range. That range wasn't take (yet) by my VPN.

Run route -n | grep 172.17 and watch deafult gateway for docker bip. Default gw should be docker0 for 172.17.0.0