[BUG] Container startup fails due to abc crontab

[florian-h05]

Is there an existing issue for this?

• I have searched the existing issues

Current Behavior

Container startup fails with this log message:

crontab: can't open '/config/crontabs/abc': Permission denied
s6-rc: warning: unable to start service init-crontabs-config: command exited 1

Expected Behavior

No response

Steps To Reproduce

1. On Synology NAS Docker
2. With a user named nginx-swag that owns the full config local directory that is mapped into the container
3. UID and GID for the container set up
4. nginx-swag versions after 2.2.0-ls175

Environment

- OS: Synology DSM 7.1.1-42962 Update 4
- How docker service was installed: from the Synology app store
- Docker version: Docker version 20.10.3, build 55f0773

CPU architecture

x86-64

Docker creation

docker-compose.yml:


version: "3.3"

networks:
  traefik:
    external: true

services:
  swag:
    image: linuxserver/swag:2.2.0-ls175
    container_name: swag
    cap_add:
      - NET_ADMIN
    networks:
      - traefik
    environment:
      - PUID=1046
      - PGID=100
      - TZ=Europe/Berlin
      - URL=<redacted>
      - SUBDOMAINS=wildcard # DuckDNS certs support subdomain wildcadrs
      - VALIDATION=duckdns
      - DNSPLUGIN=duckdns # optional
      - DUCKDNSTOKEN=<redacted>
      - ONLY_SUBDOMAINS=true # optional
      - DOCKER_MODS=linuxserver/mods:swag-dashboard|linuxserver/mods:swag-dbip|linuxserver/mods:swag-auto-reload
      - WATCHLIST="/config/nginx|/config/nginx/proxy-confs"
    volumes:
      - ./config:/config
    ports:
      - 8443:443
    restart: always

Container logs

docker-compose logs:


Starting swag ... done
Attaching to swag
swag    | [mod-init] Attempting to run Docker Modification Logic
swag    | [mod-init] Applying linuxserver/mods:swag-dashboard files to container
swag    | [mod-init] linuxserver/mods:swag-dashboard at sha256:d50df4493919a652c74a4b0570d7822ccc31f06c2bf0abc28825b55deae10146 has been previously applied skipping
swag    | [mod-init] Applying linuxserver/mods:swag-dbip files to container
swag    | [mod-init] linuxserver/mods:swag-dbip at sha256:8d4d9cec2795ce0ad2636a05aabe93fe261226568f57f4def3d372c64ce28385 has been previously applied skipping
swag    | [mod-init] Applying linuxserver/mods:swag-auto-reload files to container
swag    | [mod-init] linuxserver/mods:swag-auto-reload at sha256:70f83cfcb537678558870a5f59660c367df6a914f4b5face582648ebd01c01b7 has been previously applied skipping
swag    | [migrations] started
swag    | [migrations] 01-nginx-site-confs-default: skipped
swag    | [migrations] done
swag    | usermod: no changes
swag    | ───────────────────────────────────────
swag    | 
swag    |       ██╗     ███████╗██╗ ██████╗ 
swag    |       ██║     ██╔════╝██║██╔═══██╗
swag    |       ██║     ███████╗██║██║   ██║
swag    |       ██║     ╚════██║██║██║   ██║      
swag    |       ███████╗███████║██║╚██████╔╝
swag    |       ╚══════╝╚══════╝╚═╝ ╚═════╝ 
swag    | 
swag    |    Brought to you by linuxserver.io
swag    | ───────────────────────────────────────
swag    | 
swag    | To support the app dev(s) visit:
swag    | Certbot: https://supporters.eff.org/donate/support-work-on-certbot
swag    | 
swag    | To support LSIO projects visit:
swag    | https://www.linuxserver.io/donate/
swag    | 
swag    | ───────────────────────────────────────
swag    | GID/UID
swag    | ───────────────────────────────────────
swag    | 
swag    | User UID:    1046
swag    | User GID:    100
swag    | ───────────────────────────────────────
swag    | 
swag    | using keys found in /config/keys
swag    | crontab: can't open '/config/crontabs/abc': Permission denied
swag    | s6-rc: warning: unable to start service init-crontabs-config: command exited 1
swag    | /run/s6/basedir/scripts/rc.init: warning: s6-rc failed to properly bring all the services up! Check your logs (in /run/uncaught-logs/current if you have in-container logging) for more information.
swag    | prog: fatal: stopping the container.

[github-actions]

Thanks for opening your first issue here! Be sure to follow the relevant issue templates, or risk having this issue marked as invalid.

[nemchik]

This should be resolved in #346
Can you pull the latest image and check?

[florian-h05]

The latest image (:latest in the compose file) and docker-compose pull, docker-compose up unfortunately doesn't help.

[nemchik]

I cannot recreate, even with a clean config.

Can you post the output of

ls -lah /config/crontabs

[florian-h05]

florian@diskstation:/volume1/docker/nginx-swag$ ls -lah config/crontabs
total 4.0K
drwxrwxrwx  1 root       root   14 Mar  5 15:37 .
drwxrwxrwx+ 1 nginx-swag users 158 Jan  7 17:19 ..
-rw-r--r--  1 nginx-swag users   0 Mar  5 15:37 abc
-rwxrwxrwx  1 root       root  571 Jan  7 17:19 root

The container is running with uid and gid from the nginx-swag user in the users group.

[nemchik]

Can you try to pin your image at

2.3.0-ls186

And see if it makes a difference?

[florian-h05]

It does not make a difference. I have tried this both in my existing Container directory and in a new one on the Synology, both don‘t work.
On my PC (Fedora 36) it works fine.

[nemchik]

Can you try adding

    environment:
      - STAGING=true

and removing

    volumes:
      - ./config:/config

this should start a container with no /config volume, forcing the full init to run from scratch in a temporary fashion.

Please post the full container logs of this temporary run here.

After that, you can set your compose back to how it was, and I would recommend deleting (or renaming if you want) /config/crontabs and then starting the container.

If the error returns in the logs, include that information in your post.

[florian-h05]

Container logs of the temporary run:

swag    | Certbot: https://supporters.eff.org/donate/support-work-on-certbot
swag    | 
swag    | To support LSIO projects visit:
swag    | https://www.linuxserver.io/donate/
swag    | 
swag    | ───────────────────────────────────────
swag    | GID/UID
swag    | ───────────────────────────────────────
swag    | 
swag    | User UID:    1046
swag    | User GID:    100
swag    | ───────────────────────────────────────
swag    | 
swag    | Setting resolver to  127.0.0.11
swag    | Setting worker_processes to 4
swag    | generating self-signed keys in /config/keys, you can replace these with your own keys if required
swag    | ........+.................+....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*...+....+...+..+.+..+....+.....+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.+..+............+.+.................+.+..+.......+........+..........+..+...+...+....+...+........+.+....................+.............+..+.+...............+...+..+.........+....+......+...............+.....+...+.+......+........+.+.....+.........+..........+..+.+.....+............+................+......+...+...............+..+...............+......+..........+........+....+........+............+.+......+..............+.+.........+..+...+.+........+.......+...........+.......+............+......+...+.....+.......+............+......+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
swag    | .........+..+...+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*........+.......+..+.+...+..+.......+.....+......+.+...+.....+.+.....+.+.....+...............+...+.+..+...+.+...+.....+.......+.....+.........+....+.....+...+....+...+...........+.......+............+..+....+......+......+...............+..+............+.+...............+...+..+.......+..+......+...+...............+.+.........+.....+.......+..+......+.+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++*.......+....................................+..+.......+.....+......+.+.....+.........+......+.+.........+...+.....+......+.+......+..+.+.....+.........+......+......+.+...+......+...+........+.........+.........+....+..+.+...+..+....+........+.......+..............+.+...+..+...+.............+...............+..+......+.+......+........+.........+.+......+...+............+.........+............+..+.............+...+......+...........+....+.....+.+............+.....+.+.........+..+......+....+........+...+.............+.....+.......+..+.+.....+....+........+...+....+..+....+.....+....+......+......+.....+......+....+......+.....+.........+...+.......+.....+....+..+...+....+..+......+...+...............................+.....+...+...+..................+.+...+..+.+...+.....................+...+..+.+.....+...+......................+..+.......+...+..+......+....+...+........+....+...+..+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
swag    | -----
swag    | # do daily/weekly/monthly maintenance
swag    | # min   hour    day     month   weekday command
swag    | */15    *       *       *       *       run-parts /etc/periodic/15min
swag    | 0       *       *       *       *       run-parts /etc/periodic/hourly
swag    | 0       2       *       *       *       run-parts /etc/periodic/daily
swag    | 0       3       *       *       6       run-parts /etc/periodic/weekly
swag    | 0       5       1       *       *       run-parts /etc/periodic/monthly
swag    | # renew letsencrypt certs
swag    | 8       2       *       *       *       /app/le-renew.sh >> /config/log/letsencrypt/letsencrypt.log 2>&1
swag    | Variables set:
swag    | PUID=1046
swag    | PGID=100
swag    | TZ=Europe/Berlin
swag    | URL=redacted
swag    | SUBDOMAINS=wildcard
swag    | EXTRA_DOMAINS=
swag    | ONLY_SUBDOMAINS=true
swag    | VALIDATION=duckdns
swag    | CERTPROVIDER=
swag    | DNSPLUGIN=duckdns
swag    | EMAIL=
swag    | STAGING=true
swag    | 
swag    | Created .donoteditthisfile.conf
swag    | the resulting certificate will only cover the subdomains due to a limitation of duckdns, so it is advised to set the root location to use www.subdomain.duckdns.org
swag    | Different validation parameters entered than what was used before. Revoking and deleting existing certificate, and an updated one will be created
swag    | NOTICE: Staging is active
swag    | Using Let's Encrypt as the cert provider
swag    | SUBDOMAINS entered, processing
swag    | Wildcard cert for only the subdomains of hotze.duckdns.org will be requested
swag    | No e-mail address entered or address invalid
swag    | dns validation via duckdns plugin is selected
swag    | Generating new certificate
swag    | Saving debug log to /var/log/letsencrypt/letsencrypt.log
swag    | Account registered.
swag    | Requesting a certificate for *.hotze.duckdns.org
swag    | Unsafe permissions on credentials configuration file: /config/dns-conf/duckdns.ini
swag    | Unsafe permissions on credentials configuration file: /config/dns-conf/duckdns.ini
swag    | Waiting 30 seconds for DNS changes to propagate
swag    | 
swag    | Successfully received certificate.
swag    | Certificate is saved at: /etc/letsencrypt/live/hotze.duckdns.org/fullchain.pem
swag    | Key is saved at:         /etc/letsencrypt/live/hotze.duckdns.org/privkey.pem
swag    | This certificate expires on 2023-06-08.
swag    | These files will be updated when the certificate renews.
swag    | NEXT STEPS:
swag    | - The certificate will need to be renewed before it expires. Certbot can automatically renew the certificate in the background, but you may need to take steps to enable that functionality. See https://certbot.org/renewal-setup for instructions.
swag    | 
swag    | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
swag    | If you like Certbot, please consider supporting our work by:
swag    |  * Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
swag    |  * Donating to EFF:                    https://eff.org/donate-le
swag    | - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
swag    | New certificate generated; starting nginx
swag    | The cert does not expire within the next day. Letting the cron script handle the renewal attempts overnight (2:08am).
swag    | [custom-init] No custom files found, skipping...
swag    | [ls.io-init] done.
swag    | 2023-03-10 22:51:56,087 fail2ban.configreader   [387]: WARNING 'allowipv6' not defined in 'Definition'. Using default one: 'auto'
swag    | Server ready

Unfortunately, the error returns after deleting the `config/crontabs` directory:

swag    | [migrations] started
swag    | [migrations] 01-nginx-site-confs-default: skipped
swag    | [migrations] done
swag    | ───────────────────────────────────────
swag    | 
swag    |       ██╗     ███████╗██╗ ██████╗ 
swag    |       ██║     ██╔════╝██║██╔═══██╗
swag    |       ██║     ███████╗██║██║   ██║
swag    |       ██║     ╚════██║██║██║   ██║      
swag    |       ███████╗███████║██║╚██████╔╝
swag    |       ╚══════╝╚══════╝╚═╝ ╚═════╝ 
swag    | 
swag    |    Brought to you by linuxserver.io
swag    | ───────────────────────────────────────
swag    | 
swag    | To support the app dev(s) visit:
swag    | Certbot: https://supporters.eff.org/donate/support-work-on-certbot
swag    | 
swag    | To support LSIO projects visit:
swag    | https://www.linuxserver.io/donate/
swag    | 
swag    | ───────────────────────────────────────
swag    | GID/UID
swag    | ───────────────────────────────────────
swag    | 
swag    | User UID:    1046
swag    | User GID:    100
swag    | ───────────────────────────────────────
swag    | 
swag    | using keys found in /config/keys
swag    | # do daily/weekly/monthly maintenance
swag    | # min   hour    day     month   weekday command
swag    | */15    *       *       *       *       run-parts /etc/periodic/15min
swag    | 0       *       *       *       *       run-parts /etc/periodic/hourly
swag    | 0       2       *       *       *       run-parts /etc/periodic/daily
swag    | 0       3       *       *       6       run-parts /etc/periodic/weekly
swag    | 0       5       1       *       *       run-parts /etc/periodic/monthly
swag    | # renew letsencrypt certs
swag    | 8       2       *       *       *       /app/le-renew.sh >> /config/log/letsencrypt/letsencrypt.log 2>&1
swag    | crontab: can't open '/config/crontabs/abc': Permission denied
swag    | s6-rc: warning: unable to start service init-crontabs-config: command exited 1
swag    | /run/s6/basedir/scripts/rc.init: warning: s6-rc failed to properly bring all the services up! Check your logs (in /run/uncaught-logs/current if you have in-container logging) for more information.
swag    | prog: fatal: stopping the container.

[florian-h05]

That‘s weird:

1. I have removed config/crontabs
2. Ran chown -R nginx-swag:users config
3. Ran chmod -R 777 nginx-swag:users config

Server successfully starts up:

swag    | [migrations] started
swag    | [migrations] 01-nginx-site-confs-default: skipped
swag    | [migrations] done
swag    | usermod: no changes
swag    | ───────────────────────────────────────
swag    | 
swag    |       ██╗     ███████╗██╗ ██████╗ 
swag    |       ██║     ██╔════╝██║██╔═══██╗
swag    |       ██║     ███████╗██║██║   ██║
swag    |       ██║     ╚════██║██║██║   ██║      
swag    |       ███████╗███████║██║╚██████╔╝
swag    |       ╚══════╝╚══════╝╚═╝ ╚═════╝ 
swag    | 
swag    |    Brought to you by linuxserver.io
swag    | ───────────────────────────────────────
swag    | 
swag    | To support the app dev(s) visit:
swag    | Certbot: https://supporters.eff.org/donate/support-work-on-certbot
swag    | 
swag    | To support LSIO projects visit:
swag    | https://www.linuxserver.io/donate/
swag    | 
swag    | ───────────────────────────────────────
swag    | GID/UID
swag    | ───────────────────────────────────────
swag    | 
swag    | User UID:    1046
swag    | User GID:    100
swag    | ───────────────────────────────────────
swag    | 
swag    | using keys found in /config/keys
swag    | Variables set:
swag    | PUID=1046
swag    | PGID=100
swag    | TZ=Europe/Berlin
swag    | URL=redactedt
swag    | SUBDOMAINS=wildcard
swag    | EXTRA_DOMAINS=
swag    | ONLY_SUBDOMAINS=true
swag    | VALIDATION=duckdns
swag    | CERTPROVIDER=
swag    | DNSPLUGIN=duckdns
swag    | EMAIL=
swag    | STAGING=
swag    | 
swag    | the resulting certificate will only cover the subdomains due to a limitation of duckdns, so it is advised to set the root location to use www.subdomain.duckdns.org
swag    | Using Let's Encrypt as the cert provider
swag    | SUBDOMAINS entered, processing
swag    | Wildcard cert for only the subdomains of hotze.duckdns.org will be requested
swag    | No e-mail address entered or address invalid
swag    | dns validation via duckdns plugin is selected
swag    | Certificate exists; parameters unchanged; starting nginx
swag    | The cert does not expire within the next day. Letting the cron script handle the renewal attempts overnight (2:08am).
swag    | **** The following active confs have different version dates than the samples that are shipped. ****
swag    | **** This may be due to user customization or an update to the samples. ****
swag    | **** You should compare the following files to the samples in the same folder and update them. ****
swag    | **** Use the link at the top of the file to view the changelog. ****
swag    | ┌────────────┬────────────┬────────────────────────────────────────────────────────────────────────┐
swag    | │  old date  │  new date  │ path                                                                   │
swag    | ├────────────┼────────────┼────────────────────────────────────────────────────────────────────────┤
swag    | │ 2022-09-22 │ 2023-02-09 │ /config/nginx/authelia-server.conf                                     │
swag    | │ 2022-08-20 │ 2023-02-09 │ /config/nginx/authelia-location.conf                                   │
swag    | │ 2022-09-01 │ 2023-02-09 │ /config/nginx/proxy.conf                                               │
swag    | │ 2022-10-03 │ 2023-02-09 │ /config/nginx/site-confs/default.conf                                  │
swag    | │ 2022-09-08 │ 2023-02-05 │ /config/nginx/proxy-confs/adguard.subdomain.conf                       │
swag    | │ 2022-09-08 │ 2023-02-12 │ /config/nginx/proxy-confs/portainer.subdomain.conf                     │
swag    | │ 2022-09-08 │ 2023-02-12 │ /config/nginx/proxy-confs/grafana.subdomain.conf                       │
swag    | └────────────┴────────────┴────────────────────────────────────────────────────────────────────────┘
swag    | [custom-init] No custom files found, skipping...
swag    | [ls.io-init] done.
swag    | 2023-03-10 23:05:35,662 fail2ban.configreader   [418]: WARNING 'allowipv6' not defined in 'Definition'. Using default one: 'auto'
swag    | Server ready

[nemchik]

This section:

swag    | # do daily/weekly/monthly maintenance
swag    | # min   hour    day     month   weekday command
swag    | */15    *       *       *       *       run-parts /etc/periodic/15min
swag    | 0       *       *       *       *       run-parts /etc/periodic/hourly
swag    | 0       2       *       *       *       run-parts /etc/periodic/daily
swag    | 0       3       *       *       6       run-parts /etc/periodic/weekly
swag    | 0       5       1       *       *       run-parts /etc/periodic/monthly
swag    | # renew letsencrypt certs
swag    | 8       2       *       *       *       /app/le-renew.sh >> /config/log/letsencrypt/letsencrypt.log 2>&1

in the temporary run being followed by additional init steps tells me that it worked correctly in the temporary run.

This looks like a permission issue on your host. You can try stopping the swag container, deleting /config/crontabs, run:

chown -R 1046:100 /path/to/swag/config

Start the swag container, check for errors.

If that still does not work, try moving /path/to/swag/config (which you have as ./config in your compose) to another location (I assume you are storing the file on a disk from your NAS, try storing it on a different disk, or on the host filesystem).

ex:

    volumes:
      - /tmp/swag/config:/config

I actually do recommend trying it with /tmp/swag/config but note that this will be a temporary location that gets wiped when you reboot, so don't plan on running it there aside from just to test. This kind of run is a bit different than not having a volume at all (the last temporary test) because the volume being mounted to a folder on your NAS makes the files "permanent" as far as the container is concerned (but being in /tmp makes them temporary as far as the OS is concerned).

If this works, try saving it to a permanent location on a different disk on your nas.

[nemchik]

3. Ran chmod -R 777 nginx-swag:users config

Avoid 777. It is not a solution to the problem, it just confirms that you have a permission issue that needs a proper fix.
I would chmod -R a=,a+rX,u+w,g+w config (this sets folders to 775 and files to 664) and ensure the correct owner is set on the files with chown.

[florian-h05]

I would chmod -R a=,a+rX,u+w,g+w config (this sets folders to 775 and files to 664) and ensure the correct owner is set on the files with chown.

Doing this after the 777, it still works fine.

So the problem was solved by:

1. Remove config/crontabs
2. chown -R nginx-swag:users config
3. chmod -R a=,a+rX,u+w,g+w config

Thank you very much for your help!!

[florian-h05]

I already performed 2 and 3 several times, but I think 1 made the difference for me.