Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] docker compose up not returning shell and erroneously detecting error #10661

Closed
snspinn opened this issue Jun 6, 2023 · 4 comments · Fixed by #11718
Closed

[BUG] docker compose up not returning shell and erroneously detecting error #10661

snspinn opened this issue Jun 6, 2023 · 4 comments · Fixed by #11718

Comments

@snspinn
Copy link

snspinn commented Jun 6, 2023

Description

I want to get this issue down now before I forget some details, so apologies if it's a little light on detail. I will update as I reproduce this error for myself.
I'm running a fairly large project with docker compose and am running an issue where docker compose up -d and docker compose up are not returning the shell properly.

In the case of detached, one of the services is showing an error when I bring it up. When this happenss, the shell is not returned, even with a CTRL+C. The only way to exit is the kill the service directly w/ kill -HUP <PID>. When I get the shell back, I can attach to the logs and can confirm that the service that showed error has no errors (although this happens if service genuinely errors or now).

In the non-detached case, the services all come up without error and I attach to the output.log correctly. However, if I do a CTRL-C here, the services are stopped but I do not get my shell back automatically. I must then repeatedly do CTRL-C which results in

^Cno container to kill
^CERRO[0026] got 3 SIGTERM/SIGINTs, forcing shutdown  

I tried this in in zsh and bash shells, and each of these in xterm and gnome-terminal (xterm-256color).

Three things that might be causing it/ that are different from similar compose files:

  • A lot of services in the compose (35)
  • Some sections in the yaml are commented out
  • Long file paths for volume mounts

Steps To Reproduce

  1. Environment: Ubuntu 20.04.2 LTS, xterm
    • Docker version 24.0.2, build cb74dfc
  2. Run docker compose up -d (see psuedo-compose below)**

** I will add to this as I try reproduce with simpler compose files

version: '3'

services:

  myproject_pgsql:
    container_name: pgsql_myproject
    image: postgres:9.6-alpine
    restart: always
    environment:
      - POSTGRES_PASSWORD=root
    volumes:
      - ./subdir1/subdir2/configuration/postgresql/initdb:/docker-entrypoint-initdb.d
      - ./subdir1/subdir2/data/pgsql:/var/lib/postgresql/data
        #healthcheck:
        #test: ["CMD-SHELL", "pg_isready"]
        #interval: 10s
        #timeout: 5s
        #retries: 5
    networks:
      myproject_network:
        ipv4_address: 172.0.0.200


networks:
  myproject_network:
    external: true

Compose Version

Docker Compose version v2.18.1

Docker Environment

Client: Docker Engine - Community
 Version:    24.0.2
 Context:    default
 Debug Mode: false
 Plugins:
  buildx: Docker Buildx (Docker Inc.)
    Version:  v0.10.5
    Path:     /usr/libexec/docker/cli-plugins/docker-buildx
  compose: Docker Compose (Docker Inc.)
    Version:  v2.18.1
    Path:     /usr/libexec/docker/cli-plugins/docker-compose

Server:
 Containers: 35
  Running: 34
  Paused: 0
  Stopped: 1
 Images: 22
 Server Version: 24.0.2
 Storage Driver: overlay2
  Backing Filesystem: extfs
  Supports d_type: true
  Using metacopy: false
  Native Overlay Diff: true
  userxattr: false
 Logging Driver: json-file
 Cgroup Driver: systemd
 Cgroup Version: 2
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: io.containerd.runc.v2 runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 3dce8eb055cbb6872793272b4f20ed16117344f8
 runc version: v1.1.7-0-g860f061
 init version: de40ad0
 Security Options:
  apparmor
  seccomp
   Profile: builtin
  cgroupns
 Kernel Version: 5.19.0-43-generic
 Operating System: Ubuntu 22.04.2 LTS
 OSType: linux
 Architecture: x86_64
 CPUs: 8
 Total Memory: 31.22GiB
 Name: user
 ID: 1e21a719-ab98-48a2-ae2d-154f3e2cd0e2
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Experimental: false
 Insecure Registries:
  127.0.0.0/8
 Live Restore Enabled: false

Anything else?

No response

@TracKer
Copy link

TracKer commented Jun 10, 2023

I also started experiencing this issue after the latest update

@ndeloof
Copy link
Contributor

ndeloof commented Jun 12, 2023

I wonder this is the same as #10648

@snspinn
Copy link
Author

snspinn commented Jun 14, 2023

Thanks @ndeloof, looks like the same issue alright.
Come to think of it, I was experiencing some issue with builds also. After posting the original post the up also worked, though I haven't been able to conclude why yet.

Had a look at through #10648 and #10634. The work-arounds are good for now. Unfortunately, it means that builds are failing regularly.

Other colleagues seeing this issue. Will update if I find a fix or it goes away for good.

jsoriano added a commit to jsoriano/compose that referenced this issue Apr 11, 2024
Kill executed on the second Ctrl-C of docker compose up  was
filtering containers depending on its state. In some cases the
containers can reach an state for what these filters don't get
any container, and the command keeps reporting `no container to
kill`.

Remove this filtering, so it tries to kill any container it
finds, independently of its state.

Fixes docker#10661

Signed-off-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
@jsoriano
Copy link
Contributor

I found a similar issue, that I could reproduce by hitting Ctrl-C quickly a couple of times on a container that is capturing signals and doesn't die with compose stop.
In my case the output also printed this no container to kill error, and the command hanged.

I think the problem is that kill only tries to kill running containers, but in these cases the pending containers seem to have other states. I have opened a PR to make this kill attempt to kill any container, independently of their state: #11718

It'd be great if anyone else having this issue could give a try to my fix.

ndeloof pushed a commit that referenced this issue Apr 14, 2024
Kill executed on the second Ctrl-C of docker compose up  was
filtering containers depending on its state. In some cases the
containers can reach an state for what these filters don't get
any container, and the command keeps reporting `no container to
kill`.

Remove this filtering, so it tries to kill any container it
finds, independently of its state.

Fixes #10661

Signed-off-by: Jaime Soriano Pastor <jaime.soriano@elastic.co>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants