certbot with nginx fails if directives are already defined #5199

pwaring · 2017-10-21T18:10:18Z

My operating system is (include version):

Ubuntu 16.04.3 LTS (GNU/Linux 4.4.0-96-generic x86_64)

nginx version: nginx/1.10.3 (Ubuntu)
certbot version: 0.17.0

I installed Certbot with (certbot-auto, OS package manager, pip, etc):

Package manager (apt) using the certbot PPA (ppa:certbot/certbot). The package I installed was: python-certbot-nginx.

I ran this command and it produced this output:

# certbot certonly --nginx -d baldursgate.rixort.com
Saving debug log to /var/log/letsencrypt/letsencrypt.log
Obtaining a new certificate
Performing the following challenges:
tls-sni-01 challenge for baldursgate.rixort.com
nginx: [emerg] "server_names_hash_bucket_size" directive is duplicate in /etc/nginx/conf.d/01-server-names.conf:1
Cleaning up challenges
nginx restart failed:

Certbot's behavior differed from what I expected because:

I expected the certificate to be created. I've run the same command (certbot certonly --nginx -d [domain]) on another machine without any problems.

The issue seems to be that certbot adds a server_names_hash_bucket_size 128; entry to the nginx.conf file that it tries to generate (I'm assuming it shuts down nginx, starts a new instance, gets the certificates and then restarts the system nginx?). However, I've already defined server_names_hash_bucket_size 64; in /etc/nginx/conf.d/01-server-names.conf. As a result, nginx fails to restart because there are two definitions of the same directive.

If I run nginx -t on my configuration without certbot then it returns:

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

So I think the problem is with certbot, rather than my configuration. All my sites load correctly as well when I'm not using certbot.

Here is a Certbot log showing the issue (if available):

Logs are stored in `/var/log/letsencrypt` by default. Feel free to redact domains, e-mail and IP addresses as you see fit.

Contents of /var/log/letsencrypt/letsencrypt.log:

2017-10-21 17:37:51,497:DEBUG:certbot.main:certbot version: 0.17.0
2017-10-21 17:37:51,497:DEBUG:certbot.main:Arguments: ['--nginx', '-d', 'baldursgate.rixort.com']
2017-10-21 17:37:51,497:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#nginx,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2017-10-21 17:37:51,515:DEBUG:certbot.log:Root logging level set at 20
2017-10-21 17:37:51,516:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2017-10-21 17:37:51,517:DEBUG:certbot.plugins.selection:Requested authenticator nginx and installer nginx
2017-10-21 17:37:51,704:DEBUG:certbot.plugins.selection:Single candidate plugin: * nginx
Description: Nginx Web Server plugin - Alpha
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: nginx = certbot_nginx.configurator:NginxConfigurator
Initialized: <certbot_nginx.configurator.NginxConfigurator object at 0x7f5e6b4df990>
Prep: True
2017-10-21 17:37:51,706:DEBUG:certbot.plugins.selection:Single candidate plugin: * nginx
Description: Nginx Web Server plugin - Alpha
Interfaces: IAuthenticator, IInstaller, IPlugin
Entry point: nginx = certbot_nginx.configurator:NginxConfigurator
Initialized: <certbot_nginx.configurator.NginxConfigurator object at 0x7f5e6b4df990>
Prep: True
2017-10-21 17:37:51,706:DEBUG:certbot.plugins.selection:Selected authenticator <certbot_nginx.configurator.NginxConfigurator object at 0x7f5e6b4df990> and installer <certbot_nginx.configurator.NginxConfigurator object at 0x7f5e6b4df990>
2017-10-21 17:37:51,710:DEBUG:certbot.main:Picked account: <Account(RegistrationResource(body=Registration(status=None, contact=(u'mailto:paul@xk7.net',), agreement=u'https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf', key=JWKRSA(key=<ComparableRSAKey(<cryptography.hazmat.backends.openssl.rsa._RSAPublicKey object at 0x7f5e6ff22150>)>)), uri=u'https://acme-v01.api.letsencrypt.org/acme/reg/23041370', new_authzr_uri=u'https://acme-v01.api.letsencrypt.org/acme/new-authz', terms_of_service=u'https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf'), 530e68ede5c3d15b8b84c8aabba88044, Meta(creation_host=u'quark.default.pwaring.uk0.bigv.io', creation_dt=datetime.datetime(2017, 10, 21, 16, 43, 23, tzinfo=<UTC>)))>
2017-10-21 17:37:51,712:DEBUG:acme.client:Sending GET request to https://acme-v01.api.letsencrypt.org/directory.
2017-10-21 17:37:51,732:DEBUG:requests.packages.urllib3.connectionpool:Starting new HTTPS connection (1): acme-v01.api.letsencrypt.org
2017-10-21 17:37:51,952:DEBUG:requests.packages.urllib3.connectionpool:https://acme-v01.api.letsencrypt.org:443 "GET /directory HTTP/1.1" 200 561
2017-10-21 17:37:51,953:DEBUG:acme.client:Received response:
HTTP 200
Server: nginx
Content-Type: application/json
Content-Length: 561
Replay-Nonce: ifQQwU3PMlcfrWLFBkQwHdprHrxewWEK5DyfHLzixI8
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Sat, 21 Oct 2017 17:37:51 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Sat, 21 Oct 2017 17:37:51 GMT
Connection: keep-alive

{
  "key-change": "https://acme-v01.api.letsencrypt.org/acme/key-change",
  "meta": {
    "terms-of-service": "https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf"
  },
  "new-authz": "https://acme-v01.api.letsencrypt.org/acme/new-authz",
  "new-cert": "https://acme-v01.api.letsencrypt.org/acme/new-cert",
  "new-reg": "https://acme-v01.api.letsencrypt.org/acme/new-reg",
  "revoke-cert": "https://acme-v01.api.letsencrypt.org/acme/revoke-cert",
  "w3n51rRvSp4": "https://community.letsencrypt.org/t/adding-random-entries-to-the-directory/33417"
}
2017-10-21 17:37:51,956:INFO:certbot.main:Obtaining a new certificate
2017-10-21 17:37:51,957:DEBUG:acme.client:Requesting fresh nonce
2017-10-21 17:37:51,957:DEBUG:acme.client:Sending HEAD request to https://acme-v01.api.letsencrypt.org/acme/new-authz.
2017-10-21 17:37:52,148:DEBUG:requests.packages.urllib3.connectionpool:https://acme-v01.api.letsencrypt.org:443 "HEAD /acme/new-authz HTTP/1.1" 405 0
2017-10-21 17:37:52,149:DEBUG:acme.client:Received response:
HTTP 405
Server: nginx
Content-Type: application/problem+json
Content-Length: 91
Allow: POST
Replay-Nonce: aVafyNZChyGzVxP43hqhZ1d9pb7zoo1wTRTSXRR4pkM
Expires: Sat, 21 Oct 2017 17:37:52 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Sat, 21 Oct 2017 17:37:52 GMT
Connection: keep-alive


2017-10-21 17:37:52,149:DEBUG:acme.client:Storing nonce: aVafyNZChyGzVxP43hqhZ1d9pb7zoo1wTRTSXRR4pkM
2017-10-21 17:37:52,150:DEBUG:acme.client:JWS payload:
{
  "identifier": {
    "type": "dns", 
    "value": "baldursgate.rixort.com"
  }, 
  "resource": "new-authz"
}
2017-10-21 17:37:52,153:DEBUG:acme.client:Sending POST request to https://acme-v01.api.letsencrypt.org/acme/new-authz:
{
  "protected": "eyJub25jZSI6ICJhVmFmeU5aQ2h5R3pWeFA0M2hxaFoxZDlwYjd6b28xd1RSVFNYUlI0cGtNIiwgImFsZyI6ICJSUzI1NiIsICJqd2siOiB7ImUiOiAiQVFBQiIsICJrdHkiOiAiUlNBIiwgIm4iOiAiMFVaWUM3U0E2S0VhTFpjTGlHSUwtZlZPUlQtNjlTdVFGaGdGTDNDajkyUGM0MFh4UzNrMm8ya2lONUtwaTdQREJwVmQ5djBfSXljZFFGZTVxQ1BCV1hHSk90d2xxOUhxOVNERXN1QXN6M00wSkY0NThMS1B2UDhMVW9ETUJ4NWZaZXo3NkxubGtPeU5CODZlM3lXWVN1SVB4VG4wdU5UWDZoS3JjZHBnaG5UYjNjVU9IUl9MV1R3X3RoTVVpWHVrbVFtYzJ2TE9JYlNrU3ZMSmh6dWVaZDh3VW9vLXJHQTVmUV82My02UW0xYlluWEtFVF9aRllIY3BYNW9oNE9WSjhNVkpmbTdOa0hHLWl4RG81bDUyNnBHU2NCckZEcUgzQTM0Y1pmNjkzTW1rRHVuQVdLVUF5TkZaXzNDNDZ4TGpoclkyZi1hekFkM3ZZVHFGa3JSaTJRIn19", 
  "payload": "ewogICJpZGVudGlmaWVyIjogewogICAgInR5cGUiOiAiZG5zIiwgCiAgICAidmFsdWUiOiAiYmFsZHVyc2dhdGUucml4b3J0LmNvbSIKICB9LCAKICAicmVzb3VyY2UiOiAibmV3LWF1dGh6Igp9", 
  "signature": "xG9PLFkO48jtJ0rWDrjyaHoIVv7WpreoeLeBTbvzDC6Mt5AiENBNoEMtMQYn1URMaDq50pdUw_ciyvHt0-H1BSb8LD0Aj0Ee7y5hkhZj7uyhrfrMWLY7zxDQ-SMT254ACBle8xFT-7rJOq6ueCYh9eMKQ8yXUmM9ZqbPn0B1-nB1LuwQ3NVq7UPhFRb7UP2pmZwlzkAsR7aCGKovIqEQ2O79tVQBi9rS5sKlbWNecJNbvEe-0MtUNIg5Z2QvCZAMJFmpZ9jaMCXfHDb5eNBmH087kl1fYhjbe9jcq4_DPFBrIuhfRGWWnW9Na72PIA4B6rdqSeHmVM_5Ps-gkxez7Q"
}
2017-10-21 17:37:52,359:DEBUG:requests.packages.urllib3.connectionpool:https://acme-v01.api.letsencrypt.org:443 "POST /acme/new-authz HTTP/1.1" 201 1000
2017-10-21 17:37:52,360:DEBUG:acme.client:Received response:
HTTP 201
Server: nginx
Content-Type: application/json
Content-Length: 1000
Boulder-Requester: 23041370
Link: <https://acme-v01.api.letsencrypt.org/acme/new-cert>;rel="next"
Location: https://acme-v01.api.letsencrypt.org/acme/authz/3Iiab4sxjF2BZaG8CwDvMZRtBnJMLsQgs7ItqVqhCvo
Replay-Nonce: yxYML9unpcpmcoENrOHaKJmyIs2RzoOz5_upaBzT7H8
X-Frame-Options: DENY
Strict-Transport-Security: max-age=604800
Expires: Sat, 21 Oct 2017 17:37:52 GMT
Cache-Control: max-age=0, no-cache, no-store
Pragma: no-cache
Date: Sat, 21 Oct 2017 17:37:52 GMT
Connection: keep-alive

{
  "identifier": {
    "type": "dns",
    "value": "baldursgate.rixort.com"
  },
  "status": "pending",
  "expires": "2017-10-28T17:14:19Z",
  "challenges": [
    {
      "type": "tls-sni-01",
      "status": "pending",
      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/3Iiab4sxjF2BZaG8CwDvMZRtBnJMLsQgs7ItqVqhCvo/2267292169",
      "token": "Ox6GxItCCa7iOc8HBjvdRXDEZOZ3QhdGlYsOa-2CWAw"
    },
    {
      "type": "dns-01",
      "status": "pending",
      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/3Iiab4sxjF2BZaG8CwDvMZRtBnJMLsQgs7ItqVqhCvo/2267292170",
      "token": "iVM9nHEuOQFMTf90Ut8gbylaIvFUNrFxyKmvCMRcvKc"
    },
    {
      "type": "http-01",
      "status": "pending",
      "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/3Iiab4sxjF2BZaG8CwDvMZRtBnJMLsQgs7ItqVqhCvo/2267292171",
      "token": "xF1zb9ypIKh5rjwNTHBLtT5nhIh_-xNdBWWCzFytCSU"
    }
  ],
  "combinations": [
    [
      0
    ],
    [
      1
    ],
    [
      2
    ]
  ]
}
2017-10-21 17:37:52,360:DEBUG:acme.client:Storing nonce: yxYML9unpcpmcoENrOHaKJmyIs2RzoOz5_upaBzT7H8
2017-10-21 17:37:52,361:INFO:certbot.auth_handler:Performing the following challenges:
2017-10-21 17:37:52,362:INFO:certbot.auth_handler:tls-sni-01 challenge for baldursgate.rixort.com
2017-10-21 17:37:52,422:DEBUG:certbot.crypto_util:Generating key (1024 bits): /var/lib/letsencrypt/snakeoil/0005_key.pem
2017-10-21 17:37:52,669:DEBUG:certbot.reverter:Creating backup of /etc/nginx/sites-enabled/quark.xk7.net
2017-10-21 17:37:52,669:DEBUG:certbot.reverter:Creating backup of /etc/nginx/nginx.conf
2017-10-21 17:37:52,669:DEBUG:certbot.reverter:Creating backup of /etc/nginx/mime.types
2017-10-21 17:37:52,670:DEBUG:certbot.reverter:Creating backup of /etc/nginx/sites-enabled/baldursgate.rixort.com
2017-10-21 17:37:52,670:DEBUG:certbot.reverter:Creating backup of /etc/nginx/conf.d/01-server-names.conf
2017-10-21 17:37:52,670:DEBUG:certbot.reverter:Creating backup of /etc/nginx/sites-enabled/default
2017-10-21 17:37:52,670:DEBUG:certbot.reverter:Creating backup of /etc/letsencrypt/options-ssl-nginx.conf
2017-10-21 17:37:52,671:DEBUG:certbot_nginx.parser:Writing nginx conf tree to /etc/nginx/sites-enabled/baldursgate.rixort.com:
server {
  listen 80;
  listen [::]:80;
  server_name baldursgate.rixort.com;

  #return 301 https://$host$request_uri;

    listen 443 ssl; # managed by Certbot
    ssl_certificate /var/lib/letsencrypt/snakeoil/0005_cert.pem; # managed by Certbot
    ssl_certificate_key /var/lib/letsencrypt/snakeoil/0005_key.pem; # managed by Certbot
    include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
}

#server {
#  listen 443 ssl;
#  listen [::]:443 ssl;
#  server_name baldursgate.rixort.com;

#  ssl_certificate /etc/letsencrypt/live/baldursgate.rixort.com/fullchain.pem;
#  ssl_certificate_key /etc/letsencrypt/live/baldursgate.rixort.com/privkey.pem;
#  ssl_dhparam /etc/nginx/dhparam.pem;
#  include /etc/letsencrypt/options-ssl-nginx.conf;

#  root /var/www/baldursgate.rixort.com/public_html;

#  location / {
#    try_files $uri $uri/ =404;
#  }
#}

2017-10-21 17:37:52,673:DEBUG:certbot_nginx.parser:Writing nginx conf tree to /etc/nginx/nginx.conf:
user www-data;
worker_processes auto;
pid /run/nginx.pid;

events {
	worker_connections 768;
	# multi_accept on;
}

http {
include /etc/letsencrypt/le_tls_sni_01_cert_challenge.conf;
server_names_hash_bucket_size 128;

	##
	# Basic Settings
	##

	sendfile on;
	tcp_nopush on;
	tcp_nodelay on;
	keepalive_timeout 65;
	types_hash_max_size 2048;
	# server_tokens off;

	# server_names_hash_bucket_size 64;
	# server_name_in_redirect off;

	include /etc/nginx/mime.types;
	default_type application/octet-stream;

	##
	# SSL Settings
	##

	ssl_protocols TLSv1 TLSv1.1 TLSv1.2; # Dropping SSLv3, ref: POODLE
	ssl_prefer_server_ciphers on;

	##
	# Logging Settings
	##

	access_log /var/log/nginx/access.log;
	error_log /var/log/nginx/error.log;

	##
	# Gzip Settings
	##

	gzip on;
	gzip_disable "msie6";

	# gzip_vary on;
	# gzip_proxied any;
	# gzip_comp_level 6;
	# gzip_buffers 16 8k;
	# gzip_http_version 1.1;
	# gzip_types text/plain text/css application/json application/javascript text/xml application/xml application/xml+rss text/javascript;

	##
	# Virtual Host Configs
	##

	include /etc/nginx/conf.d/*.conf;
	include /etc/nginx/sites-enabled/*;
}


#mail {
#	# See sample authentication script at:
#	# http://wiki.nginx.org/ImapAuthenticateWithApachePhpScript
# 
#	# auth_http localhost/auth.php;
#	# pop3_capabilities "TOP" "USER";
#	# imap_capabilities "IMAP4rev1" "UIDPLUS";
# 
#	server {
#		listen     localhost:110;
#		protocol   pop3;
#		proxy      on;
#	}
# 
#	server {
#		listen     localhost:143;
#		protocol   imap;
#		proxy      on;
#	}
#}

2017-10-21 17:37:52,693:DEBUG:certbot.error_handler:Encountered exception:
Traceback (most recent call last):
  File "/usr/lib/python2.7/dist-packages/certbot/auth_handler.py", line 115, in _solve_challenges
    resp = self.auth.perform(self.achalls)
  File "/usr/lib/python2.7/dist-packages/certbot_nginx/configurator.py", line 806, in perform
    self.restart()
  File "/usr/lib/python2.7/dist-packages/certbot_nginx/configurator.py", line 590, in restart
    nginx_restart(self.conf('ctl'), self.nginx_conf)
  File "/usr/lib/python2.7/dist-packages/certbot_nginx/configurator.py", line 853, in nginx_restart
    "nginx restart failed:\n%s\n%s" % (out.read(), err.read()))
MisconfigurationError: nginx restart failed:



2017-10-21 17:37:52,693:DEBUG:certbot.error_handler:Calling registered functions
2017-10-21 17:37:52,693:INFO:certbot.auth_handler:Cleaning up challenges
2017-10-21 17:37:53,943:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==0.17.0', 'console_scripts', 'certbot')()
  File "/usr/lib/python2.7/dist-packages/certbot/main.py", line 753, in main
    return config.func(config, plugins)
  File "/usr/lib/python2.7/dist-packages/certbot/main.py", line 692, in certonly
    lineage = _get_and_save_cert(le_client, config, domains, certname, lineage)
  File "/usr/lib/python2.7/dist-packages/certbot/main.py", line 82, in _get_and_save_cert
    lineage = le_client.obtain_and_enroll_certificate(domains, certname)
  File "/usr/lib/python2.7/dist-packages/certbot/client.py", line 357, in obtain_and_enroll_certificate
    certr, chain, key, _ = self.obtain_certificate(domains)
  File "/usr/lib/python2.7/dist-packages/certbot/client.py", line 318, in obtain_certificate
    self.config.allow_subset_of_names)
  File "/usr/lib/python2.7/dist-packages/certbot/auth_handler.py", line 74, in get_authorizations
    resp = self._solve_challenges()
  File "/usr/lib/python2.7/dist-packages/certbot/auth_handler.py", line 115, in _solve_challenges
    resp = self.auth.perform(self.achalls)
  File "/usr/lib/python2.7/dist-packages/certbot_nginx/configurator.py", line 806, in perform
    self.restart()
  File "/usr/lib/python2.7/dist-packages/certbot_nginx/configurator.py", line 590, in restart
    nginx_restart(self.conf('ctl'), self.nginx_conf)
  File "/usr/lib/python2.7/dist-packages/certbot_nginx/configurator.py", line 853, in nginx_restart
    "nginx restart failed:\n%s\n%s" % (out.read(), err.read()))
MisconfigurationError: nginx restart failed:

Here is the relevant nginx server block or Apache virtualhost for the domain I am configuring:

I'm not sure if this is relevant as the failure is before nginx gets to the server block, but here it is (I've commented out the HTTPS block as I'm generating the certificate first, then enabling HTTPS, rather than letting certbot update the configuration files for me. This is intentional as I don't want certbot to overwrite files which are in my configuration management system).

server {
  listen 80;
  listen [::]:80;
  server_name baldursgate.rixort.com;

  #return 301 https://$host$request_uri;
}

#server {
#  listen 443 ssl;
#  listen [::]:443 ssl;
#  server_name baldursgate.rixort.com;

#  ssl_certificate /etc/letsencrypt/live/baldursgate.rixort.com/fullchain.pem;
#  ssl_certificate_key /etc/letsencrypt/live/baldursgate.rixort.com/privkey.pem;
#  ssl_dhparam /etc/nginx/dhparam.pem;
#  include /etc/letsencrypt/options-ssl-nginx.conf;

#  root /var/www/baldursgate.rixort.com/public_html;

#  location / {
#    try_files $uri $uri/ =404;
#  }
#}

The text was updated successfully, but these errors were encountered:

bmw · 2017-10-23T19:47:35Z

@ohemorange, can you take a look at this?

ohemorange · 2017-10-23T20:34:50Z

Thanks for getting in touch! This is a design decision that we've made to most gracefully handle interacting with the configuration options people have already set; we don't know if we can safely overwrite your existing directive, but we need the directive to be set to 128. If you would like to use Certbot's Nginx installer, you can change your directive to say 128 as well, then Certbot will work as expected. Otherwise, you'll want to use a different authenticator and installer.

pwaring · 2017-10-24T07:42:48Z

The problem isn't that cerbot is setting the directive to something other than what I have used, it's that certbot is duplicating an existing directive. Even with server_names_hash_bucket_size 128; in my nginx.conf, certbot will crash nginx with the same error message.

As a simple test, if I forget certbot for a moment and add the following two lines to nginx.conf:

server_names_hash_bucket_size 128;
server_names_hash_bucket_size 128;

Then I check the configuration syntax:

# nginx -t
nginx: [emerg] "server_names_hash_bucket_size" directive is duplicate in /etc/nginx/conf.d/01-server-names.conf:2

So nginx will fail a configuration check if the same directive is defined twice regardless of the value of that directive. This means that if you need to set server_names_hash_bucket_size to any value, even the same as used by certbot, you cannot use the nginx plugin.

ohemorange · 2017-10-24T21:26:02Z

That's true, because of the way we currently handling needing to push updates regarding server directives for certificates that have already been installed. We're currently working to update our update mechanism, which should remove the current restriction, making the set of configs we can handle automatically more expansive. For now, if you're ok with having Certbot set server_names_hash_bucket_size for you, feel free to delete that line from your config.

pwaring · 2017-10-25T07:07:34Z

I can't remove that line from my configuration though because nginx won't start without it being increased from the default, which is why I added it in the first place (before I started using certbot).

If I'm understanding this correctly, the situation is that the nginx plugin won't work if the following are true:

The user wants to use certonly so that certbot fetches the certificates but does not touch existing nginx configurations.
One or more of the directives used by certbot are already defined.

Unfortunately that is true for my setup.

bmw · 2017-11-02T00:57:37Z

I think there has been some misunderstanding here. When performing domain validation challenges required by the server, Certbot's Nginx plugin may need to set server_names_hash_bucket_size so it can configure Nginx to respond to abnormally long domain names. The logic for determining if this value should be set is it looks in the http block of your main Nginx configuration file (/etc/nginx/nginx.conf unless you've explicitly set it to something else). If it finds an existing server_names_hash_bucket_size directive but it is set to a number smaller than 128, our plugin temporarily increases the value, otherwise, it ignores it. The Nginx plugin does not currently check if the value has been set in any included files.

Because of only checking the main nginx.conf file, the value you set in /etc/nginx/conf.d/01-server-names.conf isn't noticed, our plugin adds a duplicate directive, and Nginx becomes unhappy. Setting this value in your main Nginx configuration file should have the same effect as where you've defined it, so while our plugin should do better, you should be able to work around the problem by moving the directive there rather than the included file.

In an earlier post, you said:

Even with server_names_hash_bucket_size 128; in my nginx.conf, certbot will crash nginx with the same error message.

If you've moved this directive to your main Nginx config file and nginx -t works but Certbot does not, please provide a copy of your main Nginx config file.

I hope this helps! I'm reopening this issue as I think there was a misunderstanding about the part of the plugin causing the problem here, but if I'm mistaken, please tell me I'm wrong and close the issue again.

pwaring · 2017-11-02T14:54:35Z

The bug is that the plugin logic is simply doing a search on the nginx.conf file, rather than parsing it and expanding any include directives.

Whilst moving everything to nginx.conf might be a workaround, that breaks the whole model of packaged versions of nginx, whereby the distribution (in this case Debian or Ubuntu) ships a default file and then you override any directives in different files. If you edit nginx.conf instead, you risk getting a configuration conflict each time nginx is updated, because the package manager will notice that your configuration differs from the one supplied by the package (this isn't unique to nginx, it happens with PHP and lots of other software).

bmw · 2017-11-02T18:51:37Z

I agree that is the bug. Unfortunately, setting this value in your main nginx configuration file or not setting it all are the two workarounds until this is fixed.

innocode-devops · 2018-01-19T14:37:20Z

This happened to us while moving from apt-installed certbot to certbot-auto.
Somewhere down the road certbot moved this directive:

ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;

From this file:

/etc/letsencrypt/options-ssl-nginx.conf

And now it appends it inside virtualhost file after the line:

include /etc/letsencrypt/options-ssl-nginx.conf;

But during the upgrade certbot-auto didn't update contents of this file: /etc/letsencrypt/options-ssl-nginx.conf and it still contains same line ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
Solution was commenting line out in include file and then retrofit all legacy virtualhosts installed by old certbot version with ssl_dhparam line (bit of a hussle).

I guess certbot-auto doesn't overwrite old files in /etc/, but is there a way to force update of configs or fix missing? I.e. I rename old config file, run update and certbot-auto grabs up-to-date version of config file?

pwaring · 2018-06-12T09:17:49Z

I don't know if this bug is still being worked on or is still relevant, but for the benefit of anyone who runs into it my workaround is:

Generate initial certificate using: certbot certonly --webroot -w /path/to/document_root -d example.org

Create a cronjob with the following content to automatically renew certificates and then reload the web server so that it picks up the new certs:

#!/bin/bash

set -u
set -e

/usr/bin/certbot renew
/usr/sbin/service nginx reload

The other change you need to make is for your HTTP server block to contain:

location /.well-known {
    root /path/to/document_root;
  }

I found that without the above, the renewal would fail because I would be auto-redirecting from HTTP to HTTPS and that seemed to break LetsEncrypt (not sure why).

I've been running the above for several months now, including through multiple renewal cycles, without any problems.

artfulrobot · 2018-10-11T10:16:47Z

I have the same, or similar issue. If it's a different issue I can re-post.

nginx: [emerg] duplicate listen options for 0.0.0.0:443 in /etc/nginx/sites-enabled/000-default:61

This is caused because I have, in my default config, used a listen option (backlog=1024).

nginx does not allow two listen directives with options specified, even if they're the same.

The only way I could get it to work was by removing this server optimisation.

ohemorange · 2018-10-16T00:34:41Z

@artfulrobot, this is a different issue which was fixed in #6223, so look for it in upcoming versions of certbot.

stale · 2019-06-06T08:28:31Z

To help us better see what issues are still affecting our users, this issue has been automatically marked as stale. If you still have this issue with an up-to-date version of Certbot and are interested in seeing it resolved, please add a comment letting us know. If there is no further activity, this issue will be automatically closed.

ohemorange · 2019-06-07T00:50:41Z

Fixing this should be easier once the Nginx parser rewrite is done.

houdini69 · 2020-02-09T08:31:15Z

I have got the same problem with a "server_names_hash_bucket_size 64" directive in one of my nginx configuration file and my letsencrypt certificates were due to renewal.

houdini69 · 2020-02-09T08:46:46Z

In my case I solved this issue by moving the "server_names_hash_bucket_size 64" directive from my nginx configuration file in /etc/nginx/sites-available to /etc/nginx/nginx.conf file.

GwynethLlewelyn · 2020-05-10T19:13:47Z

Today, by mere chance, I came across this issue (I would call it a bug, in spite of the developers above not really considering it a bug), as 'mysteriously' a certificate was due to renewal and, well, didn't renew — while it had done so consistently for many, many months.

IMFNSHO, there is a design concept failure here: while understandably certbot requires server_names_hash_bucket_size to be at least 128, which means that it does have to check /etc/nginx/nginx.conf, it's not really user-friendly to 'imply' that somehow the user's configuration is wrong at some level, and simply fail, leaving the user to figure out where the issue is — when the issue was with certbot, not with the user's configuration!

A much better alternative, assuming that you don't want to have certbot spending a lot of time figuring out where all included files are and parse them one by one, is simply to use

nginx -T | grep server_names_hash_bucket_size

This should return 1 if the command fails; or return 0 and be empty if that configuration directive is not present (thus making it safe for certbot to add); the rest of the cases has to be parsed manually (i.e. set with a value under 128, commented out, etc.). But at least it should validate all included files without the need to do a lot of coding on your part. Of course, I haven't checked the Python code myself (I'm not 'fluent' enough in Python) to see how exactly you check for server_names_hash_bucket_size so maybe you have good reasons to avoid calling anything directly from within Python and that would mean that my 'quick & dirty' solution would certainly not be viable.

In any case, a much better alternative would be just testing out if setting server_names_hash_bucket_size breaks nginx during the initial configuration of certbot, and if it does, tell the user that there is some kind of compatibility issue. Granted, this will not fix the issue when the user, at some point in time, long after having configured certbot, starts including new files (which was my case!... I was surprised that certbot, all of a sudden, simply stopped working — and because I use HSTS, this means that 'losing' a certificate will wreak havoc)...

Anyway, thanks to @pwaring for raising this issue; I would be utterly confused about how to 'fix' things if nobody had ever reported it before...

maximillianh · 2020-05-14T02:30:12Z

Thanks @GwynethLlewelyn . As mentioned above, this is indeed a bug which we intend to address following the rewrite of the Nginx parser. Until then, setting the value in nginx.confor not setting the it at all are the two available workarounds.

Moving forward, please be sure that your comments on this project adhere to the EFF Public Projects Code of Conduct.

pwaring · 2021-04-24T08:48:31Z

I'm going through some of my outstanding issues and noticed that this one has been open for 3+ years now without a resolution (which isn't a criticism, it's an open source project at the end of the day and I certainly don't have the time or skills to fix it). Is it worth closing this issue to keep the number of open issues manageable?

GwynethLlewelyn · 2021-04-24T13:02:24Z

Speaking strictly for myself, since the workaround is so simple, I guess that this issue could be closed...

pwaring · 2021-07-09T10:07:23Z

Closing this issue as it hasn't received any updates.

yarcowong · 2021-11-12T08:13:06Z

Ha... I'm sorry... But I met the issue today, and found it is really historical.
I would say, maybe the error messsage should be improved, that cause me confused and spend hours(em?minutes) to find out the duplicated directive: try find . -type f -exec grep server_names_hash_bucket_size... but got no lucky. ( For I can find only one in hash.conf which is provided by the ubuntu distribution and another in nginx.conf but commented out.)

Failed to renew certificate xxxx with error: nginx restart failed:
nginx: [emerg] "server_names_hash_bucket_size" directive is duplicate in /etc/nginx/conf.d/hash.conf:1

So, after reading above posts, I realized certbot will add this directive also.
Maybe the error message should be certbot will add server_names_hash_bucket_size directive, but you defined it somewhere, so duplicate -- that would be more clear 😄

alexd2580 · 2022-01-20T11:53:35Z

This issue is still relevant, especially if you're doing Certbot on ElasticBeanstalk without a load balancer to handle TLS/SSL.

The EB way to nginx configuration does not really cope well with modifying nginx.conf,
the config file is modified/reset in the automated (!) setup process multiple times.
It is not an option to modify the root-config manually, and I don't think that modifying it via script is a good idea either,
given that I don't know what else EB decides to put in there.

So there is no other option than putting it into conf.d/....

That produces the dilemma seen above, where certbot fails, because the config is invalid (double directive),
or nginx fails because, in fact, you do have a domain name exceeding the default 64.

I solved this issue by running the deploy script twice, once without server_names_hash_bucket_size: 128 for certbot to succeed,
and then a second time including the setting, (the second time certbot does not validate the domain) for the app to start properly.

syoder · 2022-02-06T20:36:47Z

Thanks, @alexd2580 - I have run into this exact problem on ElasticBeanstalk and am unable to use the workaround mentioned above since nginx.conf is a file that is generated/managed by ElasticBeanstalk itself. I think I will try your approach of running certbot twice - first without my server_names_hash_bucket_size config, and then the second time with it. But there must be a better way...

joel-foo · 2022-08-04T06:30:24Z

It works after I remove the server_names_hash_bucket_size directive from /etc/nginx/conf.d/*.conf and then add it to the main /etc/nginx/nginx.conf.

boutell · 2023-03-07T14:38:15Z

Would scanning /etc/nginx/nginx.conf for include statements be considered if submitted as a PR?

montanaviking · 2024-07-05T22:37:19Z

Still a problem for me. I just intalled Certbot on Ubuntu 22.04 using nginx and I get the errors when getting the cert for the first time:

certbot --nginx

Saving debug log to /var/log/letsencrypt/letsencrypt.log

Which names would you like to activate HTTPS for?

1: myurl.ddns.net

Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel):
Requesting a certificate for myurl.ddns.net
nginx restart failed:
nginx: [emerg] "server_names_hash_bucket_size" directive is duplicate in /etc/nginx/sites-enabled/myurl.ddns.net.conf:1

nginx -t

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Any advice is appreciated Thanks

montanaviking · 2024-07-05T22:59:03Z

I have a similar problem:
using certbot with nginx on Ubuntu 22.04

nginx -t

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
[1]+ Done gedit nginx.conf
root@kismet:/etc/nginx#
root@kismet:/etc/nginx#
root@kismet:/etc/nginx#
root@kismet:/etc/nginx# nginx -t
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
root@kismet:/etc/nginx# certbot --nginx
Saving debug log to /var/log/letsencrypt/letsencrypt.log

Which names would you like to activate HTTPS for?

1: myurl.ddns.net

Select the appropriate numbers separated by commas and/or spaces, or leave input
blank to select all options shown (Enter 'c' to cancel):
Requesting a certificate for myurl.ddns.net
nginx restart failed:
nginx: [emerg] "server_names_hash_bucket_size" directive is duplicate in /etc/nginx/sites-enabled/myurlddns.net.conf:1

Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.

nginx -t

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Any advice much appreciated.
Thanks

montanaviking · 2024-07-05T23:07:21Z

Hi again,
It appears my problem is solved.
Apparently commenting out the line:
server_names_hash_bucket_size 64;
(set to #server_names_hash_bucket_size 64;)
from /etc/nginx/sites-available/myurl.dns.net.conf
now apparently allows Let's encrypt to run the command certbot --nginx correctly. Of course, I also had to open my firewalls to expose my server's port 80.
Thanks and best.

bmw added the area: nginx label Oct 23, 2017

ohemorange closed this as completed Oct 23, 2017

bmw reopened this Nov 2, 2017

stale bot added the needs-update label Jun 6, 2019

stale bot removed the needs-update label Jun 7, 2019

mc0e mentioned this issue Jun 13, 2019

Certbot not recognizing individual .conf files from include directive in nginx.conf #6325

Closed

obfusk mentioned this issue Jun 22, 2020

certbot bug triggered by jitsi nginx config jitsi/jitsi-meet#7119

Closed

alexzorin mentioned this issue Sep 19, 2020

Nginx restart failed #8309

Closed

pwaring closed this as completed Jul 9, 2021

Mar01 mentioned this issue Sep 5, 2021

NOTE: certbot renew --nginx will fail after Jitsi setup LukeSmithxyz/landchad#127

Closed

aumouth mentioned this issue Oct 27, 2021

LetsEncrypt certbot fails with conf.d/botblocker-nginx-settings.conf mitchellkrogza/nginx-ultimate-bad-bot-blocker#449

Open

setpill mentioned this issue Feb 7, 2022

Certbot sets conflicting server_names_hash_bucket_size #9193

Closed

ohemorange mentioned this issue Feb 9, 2022

Search included files for nginx server_names_hash_bucket_size directive #9198

Merged

ohemorange self-assigned this Feb 10, 2022

veale mentioned this issue Jan 24, 2023

Allow Certbot --nginx to function by altering ssowat.conf YunoHost/issues#2135

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

certbot with nginx fails if directives are already defined #5199

certbot with nginx fails if directives are already defined #5199

pwaring commented Oct 21, 2017

bmw commented Oct 23, 2017

ohemorange commented Oct 23, 2017

pwaring commented Oct 24, 2017

ohemorange commented Oct 24, 2017

pwaring commented Oct 25, 2017

bmw commented Nov 2, 2017

pwaring commented Nov 2, 2017

bmw commented Nov 2, 2017

innocode-devops commented Jan 19, 2018

pwaring commented Jun 12, 2018 •

edited

Loading

artfulrobot commented Oct 11, 2018

ohemorange commented Oct 16, 2018 •

edited

Loading

stale bot commented Jun 6, 2019

ohemorange commented Jun 7, 2019

houdini69 commented Feb 9, 2020

houdini69 commented Feb 9, 2020

GwynethLlewelyn commented May 10, 2020

maximillianh commented May 14, 2020

pwaring commented Apr 24, 2021

GwynethLlewelyn commented Apr 24, 2021

pwaring commented Jul 9, 2021

yarcowong commented Nov 12, 2021

alexd2580 commented Jan 20, 2022

syoder commented Feb 6, 2022

joel-foo commented Aug 4, 2022

boutell commented Mar 7, 2023

montanaviking commented Jul 5, 2024

montanaviking commented Jul 5, 2024

montanaviking commented Jul 5, 2024

certbot with nginx fails if directives are already defined #5199

certbot with nginx fails if directives are already defined #5199

Comments

pwaring commented Oct 21, 2017

My operating system is (include version):

I installed Certbot with (certbot-auto, OS package manager, pip, etc):

I ran this command and it produced this output:

Certbot's behavior differed from what I expected because:

Here is a Certbot log showing the issue (if available):

Logs are stored in /var/log/letsencrypt by default. Feel free to redact domains, e-mail and IP addresses as you see fit.

Here is the relevant nginx server block or Apache virtualhost for the domain I am configuring:

bmw commented Oct 23, 2017

ohemorange commented Oct 23, 2017

pwaring commented Oct 24, 2017

ohemorange commented Oct 24, 2017

pwaring commented Oct 25, 2017

bmw commented Nov 2, 2017

pwaring commented Nov 2, 2017

bmw commented Nov 2, 2017

innocode-devops commented Jan 19, 2018

pwaring commented Jun 12, 2018 • edited Loading

artfulrobot commented Oct 11, 2018

ohemorange commented Oct 16, 2018 • edited Loading

stale bot commented Jun 6, 2019

ohemorange commented Jun 7, 2019

houdini69 commented Feb 9, 2020

houdini69 commented Feb 9, 2020

GwynethLlewelyn commented May 10, 2020

maximillianh commented May 14, 2020

pwaring commented Apr 24, 2021

GwynethLlewelyn commented Apr 24, 2021

pwaring commented Jul 9, 2021

yarcowong commented Nov 12, 2021

alexd2580 commented Jan 20, 2022

syoder commented Feb 6, 2022

joel-foo commented Aug 4, 2022

boutell commented Mar 7, 2023

montanaviking commented Jul 5, 2024

certbot --nginx

nginx -t

montanaviking commented Jul 5, 2024

nginx -t

nginx -t

montanaviking commented Jul 5, 2024

Logs are stored in `/var/log/letsencrypt` by default. Feel free to redact domains, e-mail and IP addresses as you see fit.

pwaring commented Jun 12, 2018 •

edited

Loading

ohemorange commented Oct 16, 2018 •

edited

Loading