letsencrypt-nginx-proxy-companion is removing out of its context symlinks in /etc/nginx/certs

[ejbp]

I'm not able to have a mixed configuration with domains that are under letsencrypt-nginx-proxy-companion management and domains that are not, having in mind that jwilder/nginx-proxy is the same and using the same certs dir for all domains. What's happening is that letsencrypt-nginx-proxy-companion is removing my custom certificates since it considers that they are unsed.

log:

letsencrypt-nginx-proxy-companion    | Symlinked domains: yyy.com xxx.com
letsencrypt-nginx-proxy-companion    | Enabled domains: xxx.com
letsencrypt-nginx-proxy-companion    | Disabled domains: yyy.com
letsencrypt-nginx-proxy-companion    | Some domains are disabled. Check them to remove unused symlinks.
letsencrypt-nginx-proxy-companion    | 
letsencrypt-nginx-proxy-companion    | Checking domain yyy.com: 
letsencrypt-nginx-proxy-companion    | Checking yyy.com.crt - removing.
letsencrypt-nginx-proxy-companion    | Checking yyy.com.key - removing.
letsencrypt-nginx-proxy-companion    | Checking yyy.com.dhparam.pem
letsencrypt-nginx-proxy-companion    | Checking yyy.com.chain.pem - removing.
letsencrypt-nginx-proxy-companion    | Unused domains checking is finished.

docker-compose.yml:

services:
  letsencrypt_nginx_proxy_companion:
    image: jrcs/letsencrypt-nginx-proxy-companion
    container_name: letsencrypt-nginx-proxy-companion
    volumes_from:
      - load_balancer
    volumes:
      - "/var/run/docker.sock:/var/run/docker.sock:ro"
      - "${LETSENCRYPT_CERTS_PATH}:/etc/nginx/certs:rw"
    environment:
      - NGINX_DOCKER_GEN_CONTAINER=load_balancer
  load_balancer:
    image: jwilder/nginx-proxy
    container_name: load_balancer
    ports:
      - 80:80
      - 443:443
    volumes:
      - ./docker/load-balancer/nginx-proxy-vhost.d:/etc/nginx/vhost.d
      - /var/run/docker.sock:/tmp/docker.sock:ro
      - /usr/share/nginx/html
      - "${LETSENCRYPT_CERTS_PATH:/etc/nginx/certs:ro"
    labels:
      com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy: "true"
    environment:
      - ENABLE_IPV6=true
    restart: always
  website:
    container_name: website
    image:  ${IMAGE_WEBSITE}
    ports:
      - 5555:80
    environment:
      - VIRTUAL_HOST=xxx.com
      - LETSENCRYPT_HOST=xxx.com
      - LETSENCRYPT_EMAIL=production@xxx.com
  custom_certificates_website:
    container_name: custom-certificates-website
    image:  ${IMAGE_CUSTOM_CERTIFICATES_WEBSITE}
    ports:
      - 5556:80
    environment:
      - VIRTUAL_HOST=yyy.com,*.yyy.com

Originally posted by @ejbp in #319 (comment)

UPDATE:

Can you describe how you organize your custom certificates in /etc/nginx/certs ? Exactly like the LE companion does (a folder named after the certificate's base domain and symlinks pointing to the cert and private key inside) ? Please try to be as precise and detailed as possible.

Are they obtained through Let's Encrypt ? I guess they are.

This is happening when I organize the certificates exactly how letsencrypt_nginx_proxy_companion.nginx does, meaning:

ls -laht (before launching letsencrypt_nginx_proxy_companion container):

user@machine:~/data/certificates$ ls -laht
total 36K
drwxr-xr-x 6 root root 4.0K Nov 12 18:53 .
drwxr-xr-x 2 root root 4.0K Nov 12 18:12 yyy.com
lrwxrwxrwx 1 root root   32 Nov 12 18:53 yyy.com.key -> yyy.com/privkey2.pem
lrwxrwxrwx 1 root root   30 Nov 12 18:53 yyy.com.chain.pem -> yyy.com/chain2.pem
lrwxrwxrwx 1 root root   34 Nov 12 18:53 yyy.com.crt -> yyy.com/fullchain2.pem
drwxr-xr-x 2 root root 4.0K Nov 12 02:23 xxx.com
lrwxrwxrwx 1 root root   29 Nov  9 18:24 xxx.com.chain.pem -> ./xxx.com/chain.pem
lrwxrwxrwx 1 root root   33 Nov  9 18:24 xxx.com.crt -> ./xxx.com/fullchain.pem
lrwxrwxrwx 1 root root   13 Nov  9 18:24 xxx.com.dhparam.pem -> ./dhparam.pem
lrwxrwxrwx 1 root root   27 Nov  9 18:24 xxx.com.key -> ./xxx.com/key.pem

ls -laht (after launching letsencrypt_nginx_proxy_companion container):

user@machine:~/data/certificates$ ls -laht
total 36K
drwxr-xr-x 6 root root 4.0K Nov 12 18:53 .
drwxr-xr-x 2 root root 4.0K Nov 12 18:12 yyy.com
drwxr-xr-x 2 root root 4.0K Nov 12 02:23 xxx.com
lrwxrwxrwx 1 root root   29 Nov  9 18:24 xxx.com.chain.pem -> ./xxx.com/chain.pem
lrwxrwxrwx 1 root root   33 Nov  9 18:24 xxx.com.crt -> ./xxx.com/fullchain.pem
lrwxrwxrwx 1 root root   13 Nov  9 18:24 xxx.com.dhparam.pem -> ./dhparam.pem
lrwxrwxrwx 1 root root   27 Nov  9 18:24 xxx.com.key -> ./xxx.com/key.pem

note: Custom certificates are generated with letsencrypt.

[buchdag]

Can you describe how you organize your custom certificates in /etc/nginx/certs ? Exactly like the LE companion does (a folder named after the certificate's base domain and symlinks pointing to the cert and private key inside) ? Please try to be as precise and detailed as possible.

Are they obtained through Let's Encrypt ? I guess they are.

[buchdag]

Okay.

So the issue at hand is that the LE companion can't really discriminate the certificate it creates from other obtained through another method when they use a similar structure (symlinks to files in a subfolder).

The best solution I can think of right away would be to include a dot file next to the certs handled by the companion during creation / renewal (most importantly before the symlink cleanup function). Then the symlink cleanup function check for the presence of this dot file. If it's not there, those files are not handled by the companion, don't touch the symlinks.

Does that seem sound to you ?

[ejbp]

I can't think in a better solution. Probably is what you have in mind but that dot file should be inside each directory not in the root (near the sym link) for better user visualization of the root dir.

Though, in the moment of transition from current version to this one, some symlinks may stay zombies if:

• Docker with current version is running
• Stop the docker and update it
• Remove an existent companion managed host
• Launch the docker with the new image

In my opinion is not a reason to implement a more complex and eventually worst solution so I would advance with this one.