.well-known ACME challenge files blocked 403 Forbidden in some Nginx configurations

[centminmod]

Hi this is related to Letsencrypt manual authenticator mode with the ACME challenge file having a dot prefix certbot/certbot#730. This can be blocked with 403 Forbidden access by some Nginx configurations which block dot prefix files/folders from web access by default.

i.e.

location ~ /\.          { access_log off; log_not_found off; deny all; }

For a workaround for me, I had to add this to my Nginx vhost

location ~ /.well-known {
      location ~ /.well-known/acme-challenge/(.*) {
              more_set_headers    "Content-Type: application/jose+json";
      }
}   

Probably need to document this for folks as to requirements needed for Nginx to allow dot prefix file for .well-known requests

not sure if you just add a curl check of the ACME challenge file for the status code so if it's anything other than 200 status, you can show a more detailed explanation ? i.e. if it's 403 status for the curl header check, say

ensure nginx vhost allows dot prefix directory access to /.well-known

[riobard]

You should use the following in nginx config

location ^~ /.well-known/acme-challenge/ {
  # the usual settings
}

[mnpenner]

Thank you @riobard. I was trying with

location '/.well-known/acme-challenge' { # <-- does not work
    default_type "text/plain";
    root        /tmp/letsencrypt-auto;
}

But that didn't do it... I guess nginx keeps processing the other restrictions. Yours seems to do the trick though.

[kb-]

Hi, my free webhost is blocking the " .well-known too", the dot being the problem. They only give very limited control with htaccess files and the apache configuration is of course locked... So I can't find a workaround.

Is there a way to change the challenge URL to something else who doesn't contain a dot?

[augnustin]

After some struggle, here's my working config:

server {
  listen                80;
  server_name           example.com;
  root /vagrant/www/current/public;

  # Necessary for Let's Encrypt Domain Name ownership validation
  location /.well-known/acme-challenge/ {
    try_files $uri /dev/null =404;
  }
  location / {
    return 301 https://$host$request_uri;
  }
}

[wturrell]

Despite plenty of servers/configs where this works fine…

location ~ /.well-known {
  allow all;
}

on one where /.well-known is an alias, I have to add an ^ otherwise I get 403 Forbidden. (I can't understand why – I commented out all the other location blocks in turn, none of which match anyway, and it made no difference…)

location ^~ /.well-known {
  allow all;
  auth_basic off;
  alias /path/to/.well-known/;
}

[rowanthorpe]

The reason @riobard's version works is because of the order and manner in which nginx works through the location matches. I personally find the way it is documented at nginx.org adds an unnecessary layer of temporal convolution (unless there is some nuance that I have missed when reordering the guts of points 3 and 4 below), so this is my version:

1. If there is an exact path-match (with location = XXX {})
   • this block is processed
   • match-searching stops
2. If there are one or more preferential prefix-matches (with location ^~ XXX {})
   • the block of the longest (most explicit) of those matches is processed
   • match-searching stops
3. If there are one or more case-sensitive regex-matches (with location ~ XXX {}), or case-insensitive regex-matches (with location ~* XXX {})
   • the block of the first matching regex found (when scanning the config-file top-to-bottom) is processed
   • match-searching stops
4. If there are one or more non-preferential prefix-matches (with location XXX {})
   • the block of the longest (most explicit) of those matches is processed
   • match-searching stops

The three biggest sources of confusion I have seen relevant to these kind of issues with Nginx configs are:

• ^~ does not signify a form of regex-match despite including the ~ symbol. This surprises people coming from languages like Perl, etc.
• The non-regex match-types are fully declarative - order of definition in the config doesn't matter - but the winning regex-match (if processing even gets that far) is entirely based on its order of entry in the config file. These two very different methods of evaluation combined within the same file raises the cognitive load a lot.
• the files checked by try_files, error_page, etc are constructed based on the path in the context of any given root and alias directives, which sometimes become "action at a distance" gotchas

I find the following example useful (good to leave all of /.well-known open, as acme isn't the only protocol to use that directory), and of course add things like @centminmod's added header for an acme-challenge sub-match within the .well-known block if you want:

root /var/www/my-server-directory;
index index.php;  # or whatever your index files are...

location ^~ /.well-known/ {
    limit_req            [tighter per-ip settings here];
    access_log           off;
    log_not_found        off;
    root                 /var/www/html;
    autoindex            off;
    index                index.html; # "no-such-file.txt",if expected protos don't need it
    try_files            $uri $uri/ =404;
}

location / {
    limit_req            [looser per-ip settings here];
    ...
}

location ~ /\. {
    return               403;
}

[cpu]

This repository is deprecated & un-maintained. Closing this issue. If applicable, please move discussion to the replacement IETF owned repo and the mailing list.

[helok]

location ^~ /.well-known/acme-challenge/ {
allow all;
default_type "text/plain";
}