[Bug]: Ryuk v0.6.0 terminates immediately after startup - MacOS with colima #2074

tompaz3 · 2024-01-04T11:12:32Z

Testcontainers version

v0.27.0

Using the latest Testcontainers version?

Yes

Host OS

MacOS

Host arch

ARM

Go version

1.21

Docker version

Client: Docker Engine - Community
 Version:           24.0.7
 API version:       1.43
 Go version:        go1.21.3
 Git commit:        afdd53b4e3
 Built:             Thu Oct 26 07:06:42 2023
 OS/Arch:           darwin/arm64
 Context:           colima

Server: Docker Engine - Community
 Engine:
  Version:          24.0.7
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.10
  Git commit:       311b9ff
  Built:            Thu Oct 26 09:08:29 2023
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.25
  GitCommit:        d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f
 runc:
  Version:          1.1.10
  GitCommit:        v1.1.10-0-g18a0cb0
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Docker info

Client: Docker Engine - Community
 Version:    24.0.7
 Context:    colima
 Debug Mode: false
 Plugins:
  compose: Docker Compose (Docker Inc.)
    Version:  2.23.3
    Path:     /Users/tomasz/.docker/cli-plugins/docker-compose

Server:
 Containers: 6
  Running: 4
  Paused: 0
  Stopped: 2
 Images: 13
 Server Version: 24.0.7
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: d8f198a4ed8892c764191ef7b3b06d8a2eeb5c7f
 runc version: v1.1.10-0-g18a0cb0
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 6.5.0-14-generic
 Operating System: Ubuntu 23.10
 OSType: linux
 Architecture: aarch64
 CPUs: 2
 Total Memory: 7.737GiB
 Name: colima
 ID: 4e9f6a1b-9237-4356-9220-2d2d70d4785c
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

What happened?

Description

I tried to add Testcontainers to my Go project. Been using Testcontainers without any bigger issues in my Java projects for some time already.

Error occurs when Ryuk container is being started.

Error is returned here:

testcontainers-go/docker.go

Line 1209 in 8d0f53a

return c, fmt.Errorf("%w: could not start container", err)

When reaper is created:

testcontainers-go/reaper.go

Line 252 in 8d0f53a

c, err := provider.RunContainer(ctx, req)

https://github.com/testcontainers/testcontainers-go/blob/8d0f53aea823b460a30e243ca47fccd379b1cbd1/reaper.go#L171C24-L171C24

When I run my Java project ( org.testcontainers:testcontainers:1.19.3), I noticed it used testcontainers/ryuk:0.5.1 image while testcontainers-go uses testcontainers/ryuk:0.6.0 image.

Looking at the execution logs, it looks like the Ryuk container was created, started and immediately destroyed.

NOTE:

If I disable Ryuk TESTCONTAINERS_RYUK_DISABLED=true, everything works fine. I just have to manage resources myself.

Relevant log output

=== RUN   Test_SetUpContainers
2024/01/04 11:52:02 github.com/testcontainers/testcontainers-go - Connected to docker: 
  Server Version: 24.0.7
  API Version: 1.43
  Operating System: Ubuntu 23.10
  Total Memory: 7922 MB
  Resolved Docker Host: unix:///Users/tomasz/.docker/run/docker.sock
  Resolved Docker Socket Path: /Users/tomasz/.docker/run/docker.sock
  Test SessionID: 5828dc74a3761850df556239b512ea38cacd69eb8bf59bb60116bf2c30cd0f75
  Test ProcessID: d820c540-630c-4efb-8425-fb44c03e1a6f
2024/01/04 11:52:02 🐳 Creating container for image testcontainers/ryuk:0.6.0
2024/01/04 11:52:02 ✅ Container created: 26908c7e1ba1
2024/01/04 11:52:02 🐳 Starting container: 26908c7e1ba1
2024/01/04 11:52:02 ✅ Container started: 26908c7e1ba1
2024/01/04 11:52:02 🚧 Waiting for container id 26908c7e1ba1 image: testcontainers/ryuk:0.6.0. Waiting for: &{Port:8080/tcp timeout:<nil> PollInterval:100ms}
2024/01/04 11:52:02 failed accessing container logs: Error response from daemon: No such container: 26908c7e1ba1380560164797589e38c554d99bf245a429aa2241b5de7ee31d5f
    other_test.go:17: 
        	Error Trace:	/Users/tomasz/Projects/some_project/some_test.go:17
        	Error:      	Received unexpected error:
        	            	Error response from daemon: No such container: 26908c7e1ba1380560164797589e38c554d99bf245a429aa2241b5de7ee31d5f: could not start container: creating network reaper failed: failed to create network
        	Test:       	Test_SetUpContainers
        	Messages:   	failed to create network
--- FAIL: Test_SetUpContainers (0.47s)

Additional information

Minimal reproducible example:

import (
	"context"
	"testing"

	"github.com/stretchr/testify/assert"
	"github.com/testcontainers/testcontainers-go/network"
)

func Test_SetUpContainers(t *testing.T) {
	// if I disable Ryuk, everything works correctly, but I have to clean up the resources myself.
	// os.Setenv("TESTCONTAINERS_RYUK_DISABLED", "true")
	
	net, err := network.New(context.Background(),
		network.WithCheckDuplicate(),
	)
	if err == nil {
		defer net.Remove(context.Background())
	}

	assert.NoError(t, err, "failed to create network")
	assert.NotEmpty(t, net.Name, "network name is empty")
}

Docker events output when the test above is run:

2024-01-04T11:52:02.581178298+01:00 container create 26908c7e1ba1380560164797589e38c554d99bf245a429aa2241b5de7ee31d5f (image=testcontainers/ryuk:0.6.0, name=reaper_5828dc74a3761850df556239b512ea38cacd69eb8bf59bb60116bf2c30cd0f75, org.testcontainers=true, org.testcontainers.lang=go, org.testcontainers.reaper=true, org.testcontainers.ryuk=true, org.testcontainers.sessionId=5828dc74a3761850df556239b512ea38cacd69eb8bf59bb60116bf2c30cd0f75, org.testcontainers.version=0.27.0)
2024-01-04T11:52:02.601870149+01:00 network connect a3ba414323cf0f6d22692298fa1d6b184da1b6e3daf457095234cc1265d3cbe9 (container=26908c7e1ba1380560164797589e38c554d99bf245a429aa2241b5de7ee31d5f, name=bridge, type=bridge)
2024-01-04T11:52:02.743223717+01:00 container start 26908c7e1ba1380560164797589e38c554d99bf245a429aa2241b5de7ee31d5f (image=testcontainers/ryuk:0.6.0, name=reaper_5828dc74a3761850df556239b512ea38cacd69eb8bf59bb60116bf2c30cd0f75, org.testcontainers=true, org.testcontainers.lang=go, org.testcontainers.reaper=true, org.testcontainers.ryuk=true, org.testcontainers.sessionId=5828dc74a3761850df556239b512ea38cacd69eb8bf59bb60116bf2c30cd0f75, org.testcontainers.version=0.27.0)
2024-01-04T11:52:02.863724099+01:00 network disconnect a3ba414323cf0f6d22692298fa1d6b184da1b6e3daf457095234cc1265d3cbe9 (container=26908c7e1ba1380560164797589e38c554d99bf245a429aa2241b5de7ee31d5f, name=bridge, type=bridge)
2024-01-04T11:52:02.871380137+01:00 container die 26908c7e1ba1380560164797589e38c554d99bf245a429aa2241b5de7ee31d5f (execDuration=0, exitCode=2, image=testcontainers/ryuk:0.6.0, name=reaper_5828dc74a3761850df556239b512ea38cacd69eb8bf59bb60116bf2c30cd0f75, org.testcontainers=true, org.testcontainers.lang=go, org.testcontainers.reaper=true, org.testcontainers.ryuk=true, org.testcontainers.sessionId=5828dc74a3761850df556239b512ea38cacd69eb8bf59bb60116bf2c30cd0f75, org.testcontainers.version=0.27.0)
2024-01-04T11:52:02.874163275+01:00 container destroy 26908c7e1ba1380560164797589e38c554d99bf245a429aa2241b5de7ee31d5f (image=testcontainers/ryuk:0.6.0, name=reaper_5828dc74a3761850df556239b512ea38cacd69eb8bf59bb60116bf2c30cd0f75, org.testcontainers=true, org.testcontainers.lang=go, org.testcontainers.reaper=true, org.testcontainers.ryuk=true, org.testcontainers.sessionId=5828dc74a3761850df556239b512ea38cacd69eb8bf59bb60116bf2c30cd0f75, org.testcontainers.version=0.27.0)

The text was updated successfully, but these errors were encountered:

vincentserpoul · 2024-01-08T03:06:25Z

same setup as you
I can observe the same issue, and the setting the TESTCONTAINERS_RYUK_DISABLED to true also "fixes" the issue.

also, if I sleep 2s on my local before trying to connect to the container, it also works

tompaz3 · 2024-01-08T16:51:09Z

I also don't think it has anything to do with testcontainers/ryuk version, as I retagged 0.6.0 as 0.5.1, ran with my Java project with org.testcontainers:testcontainers:1.19.3 and everything worked perfectly fine.

I tried adding any sleeps, but it didn't help, since Ryuk is immediately destroyed.

I tried executing test testcontainers-go/docker_exec_test.go - TestExecWithMultiplexedResponse, but it failed.
I added sleep in testcontainers-go/docker.go#L197, but the problem was that at this moment, Ryuk was already destroyed (container die and container destroy events had already been fired in docker).

mdelapenya · 2024-02-14T20:43:47Z

Hey folks, could you verify if this issue has disappeared with main? We added some improvements for how Ryuk is created here: #2084

tompaz3 · 2024-02-26T13:36:15Z

Hey,

Sorry, took me some time as I have already uninstalled colima nad had to reinstall my docker engine to verify this once again.

Just reinstalled colima and works like a charm. ✅

Testcontainers version

v0.28.0

Host OS

MacOS

Host arch

ARM

Go version

1.22

Docker version

Client: Docker Engine - Community
 Version:           25.0.3
 API version:       1.43 (downgraded from 1.44)
 Go version:        go1.21.6
 Git commit:        4debf411d1
 Built:             Tue Feb  6 20:42:40 2024
 OS/Arch:           darwin/arm64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          24.0.9
  API version:      1.43 (minimum version 1.12)
  Go version:       go1.20.13
  Git commit:       fca702d
  Built:            Thu Feb  1 00:48:55 2024
  OS/Arch:          linux/arm64
  Experimental:     false
 containerd:
  Version:          1.6.28
  GitCommit:        ae07eda36dd25f8a1b98dfbf587313b99c0190bb
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

Colima version

colima version
colima version 0.6.8
git commit: 9b0809d0ed9ad3ff1e57c405f27324e6298ca04f

runtime: docker
arch: aarch64
client: v25.0.3
server: v24.0.9

tompaz3 · 2024-03-07T12:55:15Z

@mdelapenya I suppose we could close this?

I've been using Testcontainers Go v0.28.0 with colima for the past week with absolutely no issues at all.

mdelapenya · 2024-03-07T13:49:28Z

Closing as per @tompaz3 comment. Thanks for confirming it!

pckhoi · 2024-06-13T07:55:48Z

The error is still there for me folks.

Testcontainers version

v0.30.0

Host OS

MacOS

Host arch

x86_64

Go version

1.22

Colima version

colima version 0.6.9
git commit: c3a31ed05f5fab8b2cdbae835198e8fb1717fd0f

runtime: docker
arch: x86_64
client: v26.1.1
server: v26.1.1

Docker version

Client:
 Cloud integration: v1.0.35+desktop.13
 Version:           26.1.1
 API version:       1.45
 Go version:        go1.21.9
 Git commit:        4cf5afa
 Built:             Tue Apr 30 11:44:56 2024
 OS/Arch:           darwin/amd64
 Context:           default

Server: Docker Engine - Community
 Engine:
  Version:          26.1.1
  API version:      1.45 (minimum version 1.24)
  Go version:       go1.21.9
  Git commit:       ac2de55
  Built:            Tue Apr 30 11:47:54 2024
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.6.31
  GitCommit:        e377cd56a71523140ca6ae87e30244719194a521
 runc:
  Version:          1.1.12
  GitCommit:        v1.1.12-0-g51d5e94
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

mdelapenya · 2024-06-13T08:17:10Z

@pckhoi I precisely updated the colima docs here #2578. Could you double check with those instructions please?

tompaz3 added the bug An issue with the library label Jan 4, 2024

mdelapenya closed this as completed Mar 7, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Bug]: Ryuk v0.6.0 terminates immediately after startup - MacOS with colima #2074

[Bug]: Ryuk v0.6.0 terminates immediately after startup - MacOS with colima #2074

tompaz3 commented Jan 4, 2024

vincentserpoul commented Jan 8, 2024 •

edited

Loading

tompaz3 commented Jan 8, 2024 •

edited

Loading

mdelapenya commented Feb 14, 2024

tompaz3 commented Feb 26, 2024 •

edited

Loading

tompaz3 commented Mar 7, 2024

mdelapenya commented Mar 7, 2024

pckhoi commented Jun 13, 2024

mdelapenya commented Jun 13, 2024

[Bug]: Ryuk v0.6.0 terminates immediately after startup - MacOS with colima #2074

[Bug]: Ryuk v0.6.0 terminates immediately after startup - MacOS with colima #2074

Comments

tompaz3 commented Jan 4, 2024

Testcontainers version

Using the latest Testcontainers version?

Host OS

Host arch

Go version

Docker version

Docker info

What happened?

Description

NOTE:

Relevant log output

Additional information

vincentserpoul commented Jan 8, 2024 • edited Loading

tompaz3 commented Jan 8, 2024 • edited Loading

mdelapenya commented Feb 14, 2024

tompaz3 commented Feb 26, 2024 • edited Loading

Testcontainers version

Host OS

Host arch

Go version

Docker version

Colima version

tompaz3 commented Mar 7, 2024

mdelapenya commented Mar 7, 2024

pckhoi commented Jun 13, 2024

Testcontainers version

Host OS

Host arch

Go version

Colima version

Docker version

mdelapenya commented Jun 13, 2024

vincentserpoul commented Jan 8, 2024 •

edited

Loading

tompaz3 commented Jan 8, 2024 •

edited

Loading

tompaz3 commented Feb 26, 2024 •

edited

Loading