Permission denied: search permissions are missing on a component of the path #6872

minj · 2019-03-20T20:43:28Z

I am sorry but there are likely two unrelated issues in this report. The important one is the second (see title).

My operating system is (include version):

Distributor ID: Ubuntu
Description: Ubuntu 16.04.6 LTS
Release: 16.04
Codename: xenial

I installed Certbot with (certbot-auto, OS package manager, pip, etc):

certbot-auto probably, it's a self-contained executable (not inside package)

I ran this command and it produced this output:

I found the first issue in cron logs

cron: /root/certbot-auto renew --no-self-upgrade --quiet

Attempting to renew cert (%HOST) from /etc/letsencrypt/renewal/%HOST.conf produced an unexpected error: Failed authorization procedure. %HOST (http-01): urn:ietf:params:acme:error:unauthorized :: The client lacks sufficient authorization :: Invalid response from http://%HOST/.well-known/acme-challenge/DSfQ67Jn1-rvINrFTPU_BcaxKCGIu_KmDXrvSoYwaGA [194.135.85.123]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML 2.0//EN\">\n<html><head>\n<title>404 Not Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p". Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/%HOST/fullchain.pem (failure)
1 renew failure(s), 0 parse failure(s)

I figured I need to upgrade so I ran this manually: /root/certbot-auto renew

Upgrading certbot-auto 0.31.0 to 0.32.0...
Replacing certbot-auto...
Creating virtual environment...
Installing Python packages...

Installation succeeded.
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/%HOST.conf
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator apache, Installer apache
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for %HOST
Waiting for verification...
Challenge failed for domain %HOST
http-01 challenge for %HOST
Cleaning up challenges
Attempting to renew cert (%HOST) from /etc/letsencrypt/renewal/%HOST.conf produced an unexpected error: Some challenges have failed.. Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/%HOST/fullchain.pem (failure)

ll renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/%HOST/fullchain.pem (failure)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: %HOST
   Type:   unauthorized
   Detail: Invalid response from
   http://%HOST/.well-known/acme-challenge/RpuDQr0DETVT0UzVYUU33d29Oi36IbpW1ZQEzWNgiEY
   [194.135.85.123]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>404 Not
   Found</title>\n</head><body>\n<h1>Not Found</h1>\n<p"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.

I have a fancy ServerAlias/ServerName/DocumentRoot setup so I figured certbot got confused and disabled everything. This lead to the important issue:

/root/certbot-auto renew --dry-run
Saving debug log to /var/log/letsencrypt/letsencrypt.log

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Processing /etc/letsencrypt/renewal/%HOST
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Cert is due for renewal, auto-renewing...
Plugins selected: Authenticator apache, Installer apache
Renewing an existing certificate
Performing the following challenges:
http-01 challenge for %HOST
Waiting for verification...
Challenge failed for domain %HOST
http-01 challenge for %HOST
Cleaning up challenges
Attempting to renew cert (%HOST) from /etc/letsencrypt/renewal/%HOST.conf produced an unexpected error: Some challenges have failed.. Skipping.
All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/%HOST/fullchain.pem (failure)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates below have not been saved.)

All renewal attempts failed. The following certs could not be renewed:
  /etc/letsencrypt/live/%HOST/fullchain.pem (failure)
** DRY RUN: simulating 'certbot renew' close to cert expiry
**          (The test certificates above have not been saved.)
- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
1 renew failure(s), 0 parse failure(s)

IMPORTANT NOTES:
 - The following errors were reported by the server:

   Domain: %HOST
   Type:   unauthorized
   Detail: Invalid response from
   http://%HOST/.well-known/acme-challenge/kW76RHKK3ZF3gKQ4OUIK52jEdmj_GCqr_q690NFLGTs
   [194.135.85.123]: "<!DOCTYPE HTML PUBLIC \"-//IETF//DTD HTML
   2.0//EN\">\n<html><head>\n<title>403
   Forbidden</title>\n</head><body>\n<h1>Forbidden</h1>\n<p"

   To fix these errors, please make sure that your domain name was
   entered correctly and the DNS A/AAAA record(s) for that domain
   contain(s) the right IP address.

The apache error log contained this:

[core:error] [pid 22837] (13)Permission denied: [client 52.29.173.72:54462] AH00035: access to /.well-known/acme-challenge/kW76RHKK3ZF3gKQ4OUIK52jEdmj_GCqr_q690NFLGTs denied (filesystem path '/var/lib/letsencrypt/http_challenges') because search permissions are missing on a component of the path

The error message indicated how to solve the actual problem:

ll /var/lib/letsencrypt/                      
total 16K
drwxr-x---  4 root root 4.0K 2019-03-20 22:04 ./
drwxr-xr-x 44 root root 4.0K 2019-02-13 06:39 ../
drwxr-x---  3 root root 4.0K 2017-01-24 11:54 backups/
drwxr-xr-x  2 root root 4.0K 2019-03-20 22:04 http_challenges/

chmod o+x /var/lib/letsencrypt/ => profit

So... you might want to fix it on your end.

The text was updated successfully, but these errors were encountered:

minj · 2019-03-20T20:48:25Z

I may file a separate issue later for the 404 problem since I see it even now, after I restored my aliases.

schoen · 2019-03-26T02:17:45Z

@bmw @joohoi, we do know about some form of this problem, don't we? Where Apache doesn't have proper permissions to read /var/lib/letsencrypt? (but I kind of thought that a fix for that had already landed)

bmw · 2019-03-26T17:22:20Z

My understanding is that we don't have any code that helps users who get into this state doing something like checking for it and then warning about it and/or fixing it, but we're not sure how it happens. When we create /var/lib/letsencrypt, we create it with 755 permissions.

jchkn · 2020-03-03T09:36:17Z

is this problem possible because of a more restrictive umask setting? I had umask 077 and /var/lib/letsencrypt was "drwx------" after certbot installation. Certificate request worked after I fixed the rights (added rx for others).

bmw · 2020-03-03T22:25:02Z

Could be! We actually fixed the umask problem in our 1.3.0 release today. The PR with that change is #7742.

Since I think this could have fixed the problem and we haven't seen any other updates here, I'm going to close this issue for now but please comment or open a new issue if the problems persist with an up-to-date version of Certbot.

twkonefal · 2021-04-19T16:59:52Z

I have a more restrictive umask on my system (027) and I still got hit with this error today. Not sure whether a patch was supposed to land for this, but it's still broken.

bmw · 2021-04-19T18:23:51Z

@twkonefal, check your Certbot version with certbot --version and if it's newer than 1.3.0, please open a new issue.

twkonefal · 2021-04-20T21:39:02Z

# certbot --version
certbot 1.14.0

My umask is set in /etc/profiles. The fix was "chmod o+x /var/lib/letsencrypt/".

schoen added the area: apache label Mar 26, 2019

bmw closed this as completed Mar 3, 2020

alexzorin mentioned this issue Oct 4, 2022

Creation of /var/lib/letsencrypt (and other core directories) is affected by umask #9423

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Permission denied: search permissions are missing on a component of the path #6872

Permission denied: search permissions are missing on a component of the path #6872

minj commented Mar 20, 2019

minj commented Mar 20, 2019

schoen commented Mar 26, 2019

bmw commented Mar 26, 2019

jchkn commented Mar 3, 2020

bmw commented Mar 3, 2020

twkonefal commented Apr 19, 2021

bmw commented Apr 19, 2021

twkonefal commented Apr 20, 2021 •

edited

Permission denied: search permissions are missing on a component of the path #6872

Permission denied: search permissions are missing on a component of the path #6872

Comments

minj commented Mar 20, 2019

My operating system is (include version):

I installed Certbot with (certbot-auto, OS package manager, pip, etc):

I ran this command and it produced this output:

minj commented Mar 20, 2019

schoen commented Mar 26, 2019

bmw commented Mar 26, 2019

jchkn commented Mar 3, 2020

bmw commented Mar 3, 2020

twkonefal commented Apr 19, 2021

bmw commented Apr 19, 2021

twkonefal commented Apr 20, 2021 • edited

twkonefal commented Apr 20, 2021 •

edited