Provision a certificate - LetsEncrypt

[henningwerner]

Hi there,

my LE SSL Certs doesn't get refreshed automatically.
Now they will expire in about 12 days.

So I tried to manually reprovision the certificates, but then I become following error:

Something went wrong, sorry.

Which log files should I provide you and where can I find them?

[JoshData]

/var/log/syslog

[henningwerner]

Thanks @JoshData
Can't find any regarding LetsEncrypt.
I rechecked the provision button after running tail -f /var/log/syslog

[JoshData]

You can also try running management/ssl_certificates.py from the command-line.

[henningwerner]

./ssl_certificates.py
Traceback (most recent call last):
  File "./ssl_certificates.py", line 793, in <module>
    provision_certificates_cmdline()
  File "./ssl_certificates.py", line 436, in provision_certificates_cmdline
    status = provision_certificates(env, agree_to_tos_url=agree_to_tos_url, logger=my_logger, force_domains=force_domains, show_extended_problems=show_extended_problems)
  File "./ssl_certificates.py", line 323, in provision_certificates
    logger=my_logger)
  File "/usr/local/lib/python3.4/dist-packages/free_tls_certificates/client.py", line 153, in issue_certificate
    cert_pem = cert_to_pem(cert_response.body)
  File "/usr/local/lib/python3.4/dist-packages/free_tls_certificates/client.py", line 152, in cert_to_pem
    return OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, cert)
TypeError: must be X509, not ComparableX509

[JoshData]

Could you run

 pip3 freeze | egrep "cryptography|pyOpenSSL|free-tls"

and paste the output?

[henningwerner]

Exception:
Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/pip/basecommand.py", line 122, in main
    status = self.run(options, args)
  File "/usr/lib/python3/dist-packages/pip/commands/freeze.py", line 74, in run
    req = pip.FrozenRequirement.from_dist(dist, dependency_links, find_tags=find_tags)
  File "/usr/lib/python3/dist-packages/pip/__init__.py", line 236, in from_dist
    assert len(specs) == 1 and specs[0][0] == '=='
AssertionError

Storing debug log for failure in /root/.pip/pip.log

pip.log provides the same output.

Edit: I tried with pip install --upgrade pip

Now I get following output of the pip3 freeze ... command:

Traceback (most recent call last):
  File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 635, in _build_master
    ws.require(__requires__)
  File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 943, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 834, in resolve
    raise VersionConflict(dist, req).with_context(dependent_req)
pkg_resources.VersionConflict: (pip 8.1.2 (/usr/local/lib/python3.4/dist-packages), Requirement.parse('pip==1.5.4'))

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/bin/pip3", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 2927, in <module>
    @_call_aside
  File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 2913, in _call_aside
    f(*args, **kwargs)
  File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 2940, in _initialize_master_working_set
    working_set = WorkingSet._build_master()
  File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 637, in _build_master
    return cls._build_from_requirements(__requires__)
  File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 650, in _build_from_requirements
    dists = ws.resolve(reqs, Environment())
  File "/usr/local/lib/python3.4/dist-packages/pkg_resources/__init__.py", line 829, in resolve
    raise DistributionNotFound(req, requirers)
pkg_resources.DistributionNotFound: The 'pip==1.5.4' distribution was not found and is required by the application

I hope I did not make it worser then before.

[henningwerner]

Any ideas?

[JoshData]

I would move to a clean box. I don't know how to get out of that problem.

What was the first version of Mail-in-a-Box you installed, if you know?

[henningwerner]

How can I move smoothless to a new box machine?

I've started with version v0.17b.

[JoshData]

https://mailinabox.email/maintenance.html#moving-boxes

[henningwerner]

So now I have migrated like in the description.

But got following errors during migration:
Restore destination directory /home/user-data already exists.

Solved with: duplicity restore --force

Now I got following error:
Error '[Errno 17] File exists' processing ssl/ssl_certificate.pem

syslogs says following:
box dovecot: imap-login: Fatal: Can't load private ssl_key: Key is for a different cert than ssl_cert

How could I solve this?
There is no RoundCube login possible now.

And back to the main problem the TLS Certificates:
Something unexpected went wrong: Error creating new cert :: Too many certificates already issued for exact set of domains:

Thanks!

[yodax]

It is likely that the restore wasn't successful. A new server key was generated. Therefore it doesn't match the certificate and the server can't sign request.

The last error seems to be a rate limit problem.

Where there no errors during the restore? Did you shutdown postfix and dovecot before the restore if you did it manually?

[henningwerner]

I done following steps:

1.) Install fresh installation of MIAB
2.) Rsync Backup files
3.) Restore them

Only the errors I listed above.
But you're right, I didn't stopped those services because im following the description from @JoshData

@yodax what should I do now?

[yodax]

I always restore the backup before doing the MIAB install, the advantage is that all the files are there, however you have to install duplicity your self.

Could you try to stop the services by running:

service php5-fpm stop
service postfix stop
service dovecot stop

After that restore the backup. Then restart the services.

Do you still have the original server? If so you can just rsync the /home/user-data directory.

[henningwerner]

Thanks @yodax

I now solved it by rsync /home/user-data

But now the main problem stays:
Something unexpected went wrong: Error creating new cert :: Too many certificates already issued for exact set of domains:

How can this be possible, are there any bug in MIAB?

[JoshData]

Sorry the server-switch process didn't go as well as I had hoped.

Let's Encrypt has rate-limiting that prevents too many requests for the same certificate over and over again. You may be out of luck with Let's Encrypt for a week - your alternative would be to use a different TLS certificate provider.

[henningwerner]

@JoshData no problem, but I think you should modify the description from your url.

The certificate expires in 5 days, I hope LE works before.
If I check everyday does the "counter" increments or is the rate limit a fix date?

[henningwerner]

‘‘‘* Stopping Postfix Mail Transport Agent postfix
...done.

• Starting Postfix Mail Transport Agent postfix
  ...done.
  dovecot stop/waiting
  dovecot start/running, process 4598
• Reloading nginx configuration nginx
  ...done.
  A TLS certificate was successfully installed for...‘‘‘

Seems to work fine now, thanks!

[tbhi]

Getting what appears to be the same thing:

Traceback (most recent call last):
  File "management/ssl_certificates.py", line 807, in <module>
    provision_certificates_cmdline()
  File "management/ssl_certificates.py", line 450, in provision_certificates_cmdline
    status = provision_certificates(env, agree_to_tos_url=agree_to_tos_url, logger=my_logger, force_domains=force_domains, show_extended_problems=show_extended_problems)
  File "management/ssl_certificates.py", line 337, in provision_certificates
    logger=my_logger)
  File "/usr/local/lib/python3.4/dist-packages/free_tls_certificates/client.py", line 76, in issue_certificate
    (cert_pem, chain) = request_certificate_issuance(client, challenges, csr, logger)
  File "/usr/local/lib/python3.4/dist-packages/free_tls_certificates/client.py", line 210, in request_certificate_issuance
    cert_pem = cert_to_pem(cert_response.body)
  File "/usr/local/lib/python3.4/dist-packages/free_tls_certificates/client.py", line 217, in cert_to_pem
    return OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, cert)
TypeError: must be X509, not ComparableX509

pip freeze output:

# pip3 freeze | egrep "cryptography|pyOpenSSL|free-tls"
cryptography==1.7.1
free-tls-certificates==0.1.6
pyOpenSSL==16.2.0

I've made the following change to correct the issue

# diff -u /usr/local/lib/python3.4/dist-packages/free_tls_certificates/client.py.orig /usr/local/lib/python3.4/dist-packages/free_tls_certificates/client.py
--- /usr/local/lib/python3.4/dist-packages/free_tls_certificates/client.py.orig	2017-01-11 11:28:48.332864370 +0000
+++ /usr/local/lib/python3.4/dist-packages/free_tls_certificates/client.py	2017-01-11 11:29:18.124909327 +0000
@@ -214,7 +214,7 @@
 
 
 def cert_to_pem(cert):
-    return OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, cert)
+    return OpenSSL.crypto.dump_certificate(OpenSSL.crypto.FILETYPE_PEM, cert.wrapped)
 
 
 def save_files(certificate_file, cert_pem, certificate_chain_file, chain, private_key_file, private_key_pem, logger):

[RomainFallet]

Same issue here and @stan3 workaround works for me. However, I still had the "Too many certificates" error when trying to renew all at once with :
sudo /root/mailinabox/management/ssl_certificates.py

I had to manually provision each domain with the command :
sudo /root/mailinabox/management/ssl_certificates.py mydomain.com

[mcblum]

Any chance this is still an issue? I tried to renew our certs via LE and I get a Something Went Wrong. After that message, none of the UI works and every screen (Setup, Users, etc...) displays that message. At that point I have to restart the server.

I even spun up a new VM and migrated all of our data. The results are exactly the same.

If I run the python script via the command line I get:

root@box:~/mailinabox# python ./management/ssl_certificates.py
  File "./management/ssl_certificates.py", line 500
    % request["url"], end='', flush=True)
                         ^
SyntaxError: invalid syntax

[JoshData]

(Let's go over to #1362...)