s6-svscanctl: fatal: unable to control /var/run/s6/services: supervisor not listening

[sbaydush]

Checklist

• Have you pulled and found the error with jc21/nginx-proxy-manager:latest docker image? Yes
• Are you sure you're not using someone else's docker image? Yes
• If having problems with Lets Encrypt, have you made absolutely sure your site is accessible from outside of your network? N/A

Describe the bug

• A clear and concise description of what the bug is.
• What version of Nginx Proxy Manager is reported on the login page?

Upon standing up a new NPM instance i get a few warnings about database communication issues) but I can login. As soon as i tell the stack to stop and then start it again, the database starts fine but the app container fails to start. (see screenshot)

To Reproduce
Steps to reproduce the behavior:
Deploy latest version using docker stack in example. Once everything is running fine, stop the stack and then start it again.

Expected behavior
The app container should be able to be re-created pointing to the same volumes and function properly

Screenshots
App container:

Database container:

Operating System
Using centos 7 with docker swarm on top. I am also using Storidge for the storage plugin.

Additional context
Add any other context about the problem here, docker version, browser version if applicable to the problem. Too much info is better than too little.

The only way I can get it to be fixed is to delete everything about the stack and have it re-create.

Could I be missing a volume mount?

version: "3"
services:
  app:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: always
    ports:
      # Public HTTP Port:
      - '80:80'
      # Public HTTPS Port:
      - '443:443'
      # Admin Web Port:
      - '81:81'
    environment:
      # These are the settings to access your db
      DB_MYSQL_HOST: "db"
      DB_MYSQL_PORT: 3306
      DB_MYSQL_USER: "npm"
      DB_MYSQL_PASSWORD: "Password123"
      DB_MYSQL_NAME: "npm"
      # If you would rather use Sqlite uncomment this
      # and remove all DB_MYSQL_* lines above
      # DB_SQLITE_FILE: "/data/database.sqlite"
      # Uncomment this if IPv6 is not enabled on your host
      # DISABLE_IPV6: 'true'
    volumes:
      - app:/data
      - letsencrypt:/etc/letsencrypt
    depends_on:
      - db
  db:
    image: jc21/mariadb-aria:10.4
    restart: always
    environment:
      MYSQL_ROOT_PASSWORD: 'Password123'
      MYSQL_DATABASE: 'npm'
      MYSQL_USER: 'npm'
      MYSQL_PASSWORD: 'Password123'
    volumes:
      - db:/var/lib/mysql
      
volumes:
  db:
    driver: cio
    driver_opts:
      profile: "MYSQL"
  app:
    driver: cio
    driver_opts:
      profile: "SMALL"
  letsencrypt:
    driver: cio
    driver_opts:
      profile: "SMALL"

[sbaydush]

I messed around some more and sometimes the container starts successfully and some times it doesnt...

[damntourists]

I am also encountering this issue while trying to run this on a swarm of raspberry pi 4's.

[pixel3design-hub]

I am also encountering this issue while trying to run this on a swarm of raspberry pi 4's.

Same here. Using Docker Swarm in Portainer on a RPi4-4GB

[NozomiKimot]

I am in the same boat, starting up a fresh instance is completely fine but any following startup attempts of the container falls flat on its face and recreating the entire instance with a blank slate is the only way to get it running again, super frustrating!

[NozomiKimot]

Although upon further testing I'm not entirely sure I'm encountering the exact same problem
I only started encountering the issue today and the only thing I had changed was changing some hosts to start using DNS challenges for the SSL certificates.
I removed those entries from my database and it starts up fine again
So, for me at least, my failed startup issues are related to using DNS challenges for my SSL certs
Not sure if DNS provider for the challenges make a difference but I use Cloudflare

[cestnickell]

Same issue here, I'm also on an RPi4 using DNS challenge (Cloudflare) for my SSL certs.

[wieluk]

Same Issue for me on boot up this happens but if I manually restart afterwards I don't have any problems

[ggantn]

Same issue for me I cannot create the container on my raspberry pi4

[0x5ECF4ULT]

Same issue.
4x RPi 4B Swarm
Deployment using Portainer
This issue does not arise if the stack is started via docker-compose. Very weird...
I dug into the code a little bit and it seems this issue is related to the supervisor on http://127.0.0.1:3000/. At least this is what my logs are saying. I think the timing is kind of screwed. The container crashes after the JWT keys are generated and NGINX is restarted.
My theory is that the supervisor service on http://127.0.0.1:3000/ is not running yet when restarting NGINX and that causes s6 to panic. But why would it work when starting the stack via compose then?

[0x5ECF4ULT]

Okay let me clear up my nonsense:
The error log indicates that the service on http://127.0.0.1:3000/ is not available... this has nothing to do with s6, however I've found this which means if the frontend terminates, the s6 service in /var/run/s6/services is also terminated which causes s6 to panic. @jc21 could you please take a look at this? I'm by no means a professional user of s6 and could just talk bs. Hope my theory helps tho!

[Riiskyy]

I am having this issue when trying to run in a single node RPi swam even when deploying using compose

[MaleNurse]

Occurs for me as well, as varying points in the startup. RaspPi4 as a manager in a swarm, with the stack running through Portainer. Pulled 2.7.3 instead of latest (no other changes to the compose) and it's running fine.

ETA: nevermind. After one reboot it decided that it didn't want to start again. That version must be happy only w/ an empty config and DB.

[serenewaffles]

Has anyone found a workaround for this?

[oMadMartigaNo]

Same issue for me RPi4x2 (Ubuntu 20.10 x64) swarm and running npm through Portainer v2.1.1.

s6-svscanctl: fatal: unable to control /var/run/s6/services: supervisor not listening,
finish: applet not found,
[cont-finish.d] executing container finish scripts...,
[cont-finish.d] done.,
[s6-finish] waiting for services.,
[s6-finish] sending all processes the TERM signal.,
[s6-finish] sending all processes the KILL signal and exiting.

version: "3"
services:
  npm:
    image: 'jc21/nginx-proxy-manager:latest'
    restart: unless-stopped
    ports:
      # Public HTTP Port:
      - '80:80'
      # Public HTTPS Port:
      - '443:443'
      # Admin Web Port:
      - '81:81'
    environment:
      DB_MYSQL_HOST: 'xxx.xxx.xxx.xxx'
      DB_MYSQL_PORT: '3306'
      DB_MYSQL_USER: 'x'
      DB_MYSQL_PASSWORD: 'x'
      DB_MYSQL_NAME: 'x'
      DISABLE_IPV6: 'true'
    volumes:
      - '/mnt/docker/npm/data:/data'
      - '/mnt/docker/npm/letsencrypt:/etc/letsencrypt'