Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

unable to run certbot --nginx - UnicodeDecodeError: 'ascii' #5236

Closed
logol opened this Issue Nov 12, 2017 · 11 comments

Comments

Projects
None yet
9 participants
@logol
Copy link

logol commented Nov 12, 2017

My operating system is (include version):

ArchLinux (Kernel 4.13.11-1)

I installed Certbot with (certbot-auto, OS package manager, pip, etc):

pacman -S cerbot-nginx

I ran this command and it produced this output:

cerbot --nginx

Certbot's behavior differed from what I expected because:

Saving debug log to /var/log/letsencrypt/letsencrypt.log
An unexpected error occurred:
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 330: ordinal not in range(128)
Please see the logfiles in /var/log/letsencrypt for more details.

Here is a Certbot log showing the issue (if available):

2017-11-12 15:15:18,497:DEBUG:certbot.main:certbot version: 0.19.0
2017-11-12 15:15:18,499:DEBUG:certbot.main:Arguments: ['--nginx']
2017-11-12 15:15:18,500:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#nginx,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2017-11-12 15:15:18,529:DEBUG:certbot.log:Root logging level set at 20
2017-11-12 15:15:18,531:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2017-11-12 15:15:18,532:DEBUG:certbot.plugins.selection:Requested authenticator nginx and installer nginx
2017-11-12 15:15:18,557:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
  File "/usr/bin/certbot", line 11, in <module>
    load_entry_point('certbot==0.19.0', 'console_scripts', 'certbot')()
  File "/usr/lib/python3.6/site-packages/certbot/main.py", line 861, in main
    return config.func(config, plugins)
  File "/usr/lib/python3.6/site-packages/certbot/main.py", line 685, in run
    installer, authenticator = plug_sel.choose_configurator_plugins(config, plugins, "run")
  File "/usr/lib/python3.6/site-packages/certbot/plugins/selection.py", line 189, in choose_configurator_plugins
    authenticator = installer = pick_configurator(config, req_inst, plugins)
  File "/usr/lib/python3.6/site-packages/certbot/plugins/selection.py", line 25, in pick_configurator
    (interfaces.IAuthenticator, interfaces.IInstaller))
  File "/usr/lib/python3.6/site-packages/certbot/plugins/selection.py", line 77, in pick_plugin
    verified.prepare()
  File "/usr/lib/python3.6/site-packages/certbot/plugins/disco.py", line 248, in prepare
    return [plugin_ep.prepare() for plugin_ep in six.itervalues(self._plugins)]
  File "/usr/lib/python3.6/site-packages/certbot/plugins/disco.py", line 248, in <listcomp>
    return [plugin_ep.prepare() for plugin_ep in six.itervalues(self._plugins)]
  File "/usr/lib/python3.6/site-packages/certbot/plugins/disco.py", line 130, in prepare
    self._initialized.prepare()
  File "/usr/lib/python3.6/site-packages/certbot_nginx/configurator.py", line 156, in prepare
    self.parser = parser.NginxParser(self.conf('server-root'))
  File "/usr/lib/python3.6/site-packages/certbot_nginx/parser.py", line 35, in __init__
    self.load()
  File "/usr/lib/python3.6/site-packages/certbot_nginx/parser.py", line 42, in load
    self._parse_recursively(self.config_root)
  File "/usr/lib/python3.6/site-packages/certbot_nginx/parser.py", line 53, in _parse_recursively
    trees = self._parse_files(filepath)
  File "/usr/lib/python3.6/site-packages/certbot_nginx/parser.py", line 203, in _parse_files
    parsed = nginxparser.load(_file)
  File "/usr/lib/python3.6/site-packages/certbot_nginx/nginxparser.py", line 122, in load
    return loads(_file.read())
  File "/usr/lib/python3.6/encodings/ascii.py", line 26, in decode
    return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 330: ordinal not in range(128)
2017-11-12 15:15:18,560:ERROR:certbot.log:An unexpected error occurred:

Here is the relevant nginx server block or Apache virtualhost for the domain I am configuring:

user http;
worker_processes auto;
worker_cpu_affinity auto;
pcre_jit on;

events {
    worker_connections 2048;
}


http {
    include mime.types;
    default_type application/octet-stream;
    sendfile on;
    tcp_nopush on;
    aio threads;
    server_tokens off; # Security: Disables nginx version in error messages and in the “Server” response header field.
    charset utf-8; # Force usage of UTF-8
    index index.php index.html index.htm;
    server {
      listen 80;
      listen [::]:80;
      server_name domain.tld;
      root /usr/share/nginx/html;
      location / {
        index index.htm index.html;
      }

      # ACME challenge
      location ^~ /.well-known/acme-challenge/ {
        default_type "text/plain";
        root /var/lib/letsencrypt;
      }
    }
}
@joohoi

This comment has been minimized.

Copy link
Member

joohoi commented Nov 13, 2017

We will look into this, as it is certainly something that we should handle, but to work around this, replace the double quotes in comment of server_tokens directive: “Server”

@shiroorg

This comment has been minimized.

Copy link

shiroorg commented May 5, 2018

@joohoi

2018-05-05 12:28:44,181:DEBUG:certbot.main:certbot version: 0.22.2
2018-05-05 12:28:44,181:DEBUG:certbot.main:Arguments: ['--nginx']
2018-05-05 12:28:44,181:DEBUG:certbot.main:Discovered plugins: PluginsRegistry(PluginEntryPoint#manual,PluginEntryPoint#nginx,PluginEntryPoint#null,PluginEntryPoint#standalone,PluginEntryPoint#webroot)
2018-05-05 12:28:44,193:DEBUG:certbot.log:Root logging level set at 20
2018-05-05 12:28:44,193:INFO:certbot.log:Saving debug log to /var/log/letsencrypt/letsencrypt.log
2018-05-05 12:28:44,194:DEBUG:certbot.plugins.selection:Requested authenticator nginx and installer nginx
2018-05-05 12:28:44,343:DEBUG:certbot.log:Exiting abnormally:
Traceback (most recent call last):
File "/usr/bin/certbot", line 11, in
load_entry_point('certbot==0.22.2', 'console_scripts', 'certbot')()
File "/usr/lib/python3/dist-packages/certbot/main.py", line 1266, in main
return config.func(config, plugins)
File "/usr/lib/python3/dist-packages/certbot/main.py", line 1018, in run
installer, authenticator = plug_sel.choose_configurator_plugins(config, plugins, "run")
File "/usr/lib/python3/dist-packages/certbot/plugins/selection.py", line 189, in choose_configurator_plugins
authenticator = installer = pick_configurator(config, req_inst, plugins)
File "/usr/lib/python3/dist-packages/certbot/plugins/selection.py", line 25, in pick_configurator
(interfaces.IAuthenticator, interfaces.IInstaller))
File "/usr/lib/python3/dist-packages/certbot/plugins/selection.py", line 77, in pick_plugin
verified.prepare()
File "/usr/lib/python3/dist-packages/certbot/plugins/disco.py", line 245, in prepare
return [plugin_ep.prepare() for plugin_ep in six.itervalues(self._plugins)]
File "/usr/lib/python3/dist-packages/certbot/plugins/disco.py", line 245, in
return [plugin_ep.prepare() for plugin_ep in six.itervalues(self._plugins)]
File "/usr/lib/python3/dist-packages/certbot/plugins/disco.py", line 126, in prepare
self._initialized.prepare()
File "/usr/lib/python3/dist-packages/certbot_nginx/configurator.py", line 137, in prepare
self.parser = parser.NginxParser(self.conf('server-root'))
File "/usr/lib/python3/dist-packages/certbot_nginx/parser.py", line 38, in init
self.load()
File "/usr/lib/python3/dist-packages/certbot_nginx/parser.py", line 45, in load
self._parse_recursively(self.config_root)
File "/usr/lib/python3/dist-packages/certbot_nginx/parser.py", line 66, in _parse_recursively
self._parse_recursively(subentry[1])
File "/usr/lib/python3/dist-packages/certbot_nginx/parser.py", line 56, in _parse_recursively
trees = self._parse_files(filepath)
File "/usr/lib/python3/dist-packages/certbot_nginx/parser.py", line 206, in _parse_files
parsed = nginxparser.load(_file)
File "/usr/lib/python3/dist-packages/certbot_nginx/nginxparser.py", line 123, in load
return loads(_file.read())
File "/usr/lib/python3.4/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xd1 in position 31: ordinal not in range(128)
2018-05-05 12:28:44,346:ERROR:certbot.log:An unexpected error occurred:

@mardiros

This comment has been minimized.

Copy link

mardiros commented Jul 5, 2018

For instance, the mime type files contains this lines:

application/vnd.geocube+xml                     g3 g³;

g³ will make raise the UnicodeDecodeError

> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(62)_parse_recursively()
-> elif entry[0] == ['http'] or entry[0] == ['server']:
(Pdb) l
 57             for tree in trees:
 58                 for entry in tree:
 59                     if _is_include_directive(entry):
 60                         # Parse the top-level included file
 61                         self._parse_recursively(entry[1])
 62  ->                 elif entry[0] == ['http'] or entry[0] == ['server']:
 63                         # Look for includes in the top-level 'http'/'server' context
 64                         for subentry in entry[1]:
 65                             if _is_include_directive(subentry):
 66                                 self._parse_recursively(subentry[1])
 67                             elif entry[0] == ['http'] and subentry[0] == ['server']:
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(64)_parse_recursively()
-> for subentry in entry[1]:
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(65)_parse_recursively()
-> if _is_include_directive(subentry):
(Pdb) subentry
['include', 'mime.types']
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(66)_parse_recursively()
-> self._parse_recursively(subentry[1])
(Pdb) s
--Call--
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(47)_parse_recursively()
-> def _parse_recursively(self, filepath):
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(55)_parse_recursively()
-> filepath = self.abs_path(filepath)
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(56)_parse_recursively()
-> trees = self._parse_files(filepath)
(Pdb) s
--Call--
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(189)_parse_files()
-> def _parse_files(self, filepath, override=False):
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(198)_parse_files()
-> files = glob.glob(filepath) # nginx on unix calls glob(3) for this
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(201)_parse_files()
-> trees = []
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(202)_parse_files()
-> for item in files:
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(203)_parse_files()
-> if item in self.parsed and not override:
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(205)_parse_files()
-> try:
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(206)_parse_files()
-> with open(item) as _file:
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/parser.py(207)_parse_files()
-> parsed = nginxparser.load(_file)
(Pdb) s
--Call--
> /usr/lib/python3.6/site-packages/certbot_nginx/nginxparser.py(115)load()
-> def load(_file):
(Pdb) n
> /usr/lib/python3.6/site-packages/certbot_nginx/nginxparser.py(123)load()
-> return loads(_file.read())
(Pdb) loads
<function loads at 0x7faacac85048>
(Pdb) s
--Call--
> /usr/lib/python3.6/encodings/ascii.py(25)decode()
-> def decode(self, input, final=False):
(Pdb) --KeyboardInterrupt--
(Pdb) n
> /usr/lib/python3.6/encodings/ascii.py(26)decode()
-> return codecs.ascii_decode(input, self.errors)[0]
(Pdb) n
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc2 in position 10453: ordinal not in range(128)
> /usr/lib/python3.6/encodings/ascii.py(26)decode()
-> return codecs.ascii_decode(input, self.errors)[0]
@joohoi

This comment has been minimized.

Copy link
Member

joohoi commented Jul 6, 2018

Thanks for the additional information!

@TommyZG

This comment has been minimized.

Copy link

TommyZG commented Sep 2, 2018

After reading a comment on another thread by user @egberts, I ran the following command:

grep -r -P '[^\x00-\x7f]' /etc/apache2 /etc/letsencrypt /etc/nginx

That command found the offending character "´" in one .conf file in the comment. After removing it (you can edit comments as you wish) and reloading nginx, everything worked again.

@Fretice

This comment has been minimized.

Copy link

Fretice commented Sep 7, 2018

@TommyZG can u tell me change which file ?

@Fretice

This comment has been minimized.

Copy link

Fretice commented Sep 13, 2018

i found that when i use certbot-auto in windows remote, it throw me this error ,but when i use it in linux remote ,it works great

@Finesse

This comment has been minimized.

Copy link

Finesse commented Sep 15, 2018

I had the same problem. I've found the following workaround (on Ubuntu):

  1. Run in a console:
    sudo locale-gen "en_US.UTF-8"
    sudo dpkg-reconfigure locales
  2. Add line LC_ALL="en_US.UTF-8" to the file /etc/default/locale.
  3. Reconnect SSH.

After doing it, running cerbot --nginx successes.

@da3020

This comment has been minimized.

Copy link

da3020 commented Sep 18, 2018

Hi! The reason is that if you have some non ascii letters in nginx config (even in comments!!!) it will not work.

@ohemorange

This comment has been minimized.

Copy link
Contributor

ohemorange commented Sep 18, 2018

Closing in favor of duplicate #5337, which has a PR at #5341.

@ohemorange ohemorange closed this Sep 18, 2018

@TommyZG

This comment has been minimized.

Copy link

TommyZG commented Oct 29, 2018

@TommyZG can u tell me change which file ?

It was in one of my .conf files. You have others. It was in the comment line.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.