GSoC 2017: Support for "Let's Encrypt" ACME protocol

[angelhof]

Ejabberd support for "Let's Encrypt" ACME protocol

This is the final pull request for this: https://summerofcode.withgoogle.com/projects/#5214033996152832 GSoC 2017 project.

Project Outline

This project implements support for the "Let's Encrypt" ACME protocol. It is a protocol that allows for certificates to be acquired in an easy, automated way, without any human intervention. So the main functionality of the implementation is acquiring and managing the aforementioned certificates. Below I will describe in detail all the functionality that this project implements.

Detailed Description

There are two typical scenarios that the client has to support:

• Certificate Acquisition/Renewal
• Certificate Revocation

Despite being described thoroughly in the latest ACME draft: https://tools.ietf.org/html/draft-ietf-acme-acme-07 I will try to give a brief overview of those two scenarios.

In order to acquire a certificate for a domain, a client has to:

1. Create a new account at the Certificate Authority (CA) using a private key of their choice.
2. Solve a challenge, that the CA has issued, in order to verify the control over the domain.
3. Create a Certificate Signing Request (CSR) in order to request a certificate from the CA.
4. Acquire the certificate from the CA.

In order to revoke a certificate, the client has to just send a revocation request.

Knowing the above I decided to implement 4 commands that the user can use in order to acquire, manage and inspect certificates for their domains.

• get_certificate
• renew_certificate
• list_certificates
• revoke_certificate

Get Certificate

This is the most essential command that the client uses. It acquires a certificate for all hosts specified in the configuration file or for a specific set of hosts that the user requests. This command basically follows all the steps of the first scenario above.

Renew Certificate

The renew_certificate command renews all managed certificates that are 30 days (or closer) to expiration. This command should be scheduled for execution every day or so, in order to make sure that all certificates are up to date.

List Certificates

This command pretty prints all managed certificates in plain or verbose mode. Specifically, in plain mode, it prints their domain, SANS, expiration date and path. In verbose it prints the domain, SANs, expiration date and certificate/key.

Revoke Certificate

This command revokes a managed certificate for a specified domain. This command should be used when there exists serious concern that a certificate/key has been compromised.

Structural Description

The implementation is split in 3 modules that are clearly separated from each other.

• acme_challenge.erl
• ejabberd_acme_comm.erl
• ejabberd_acme.erl

acme_challenge

This module deals with solving the challenges that the CA issues. The only challenge type that is currently supported is "http-01". If more challenge types are to be supported they should be included in this module.

ejabberd_acme_comm

This module contains functions that implement all necessary http requests to the ACME CA. Its purpose is to facilitate the ACME client implementation by separating the handling/validating/parsing of all the requests from the application logic.

ejabberd_acme

This is the main module of the project. It contains implementations of all the aforementioned commands and the certificate management. It only contains application logic and calls functions from ejabberd_acme_comm in order to communicate with the CA.

Example Usage

An example scenario would be to first execute

ejabberdctl get_certificate all

And then have a script be scheduled to run the following command every day or so

ejabberdctl renew_certificate

Known Issues / Things left to do

The implementation offers basic support of the ACME protocol. However it is not complete and has some known issues. I will try to include all of the issues and future tasks here.

Issues

The main issue is that I haven't tested the client on the real "Let's Encrypt" ACME CA because I don't have control of any domain. That is why I have only tested the implementation with a local CA by redirecting all domains to localhost. Because of that I am not sure whether everything works correctly on the real CA.

Future Tasks

• Write documentation in processone/doc.ejabberd.im
• Support more key types
• Support more challenge types (dns-01, tls-sni-01)

Any feedback on this project is greatly appreciated.

[zinid]

@angelhof so intermediate certificates should be hard coded or what? Can't we get them via ACME protocol? I see in the I-D that the ACME server may return several certificates (I guess including intermediate ones).

[angelhof]

@zinid There is no need to hardcode them, we could get the intermediate certifcate everytime we need it from https://letsencrypt.org/certs/lets-encrypt-x3-cross-signed.pem.txt .

[zinid]

@angelhof but the code becomes Let's Encrypt only, so this is not ACME anymore.

[zinid]

Is it an issue of the LE ACME server implementation?

[zinid]

See this: https://tools.ietf.org/html/draft-ietf-acme-acme-07#section-7.4.2

[angelhof]

Ok it can be done. I will implement it. So the certificates that we save should contain the whole certificate-chain right?

[zinid]

Yes, just store them inside the same file (the order doesn't matter).
Please git pull before doing this, also it would be great to fix this by Monday as we're releasing 17.11 this day.

[angelhof]

@zinid There is a problem, when I save the end certificate, the issuer certificate, and the private key for the end certificate in a pem file and try to add it, ejabberd_pkix:build_chain_and_check returns the following error.

[error] <0.376.0>@ejabberd_pkix:build_chain_and_check:368 Failed to build certificate chain for .../ejabberd/certs/acme/test.free.pem: no matching private key found for certificate in the chain

[zinid]

Are you sure there is a private key?

[zinid]

Also, I don't have such problem, I just have the warning about unknown CA (due to missing intermediate cert).

[angelhof]

This problem appeared now that I am trying to include the issuer certificate in the pem file.
The pem file looks like this

-----BEGIN EC PRIVATE KEY-----
....
-----END EC PRIVATE KEY-----

-----BEGIN CERTIFICATE-----
.....
-----END CERTIFICATE-----

-----BEGIN CERTIFICATE-----
.....
-----END CERTIFICATE-----

[zinid]

Well I really cannot say what's going on without seeing these PEMs. Maybe private key is not for this certfile? What happens if you change the order?

[angelhof]

I changed the order and nothing happens. If you want to have a look I could make a new pull request with the code so that you can check it out.

[zinid]

Can't you just mail me this PEM file privately at ekhramtsov@process-one.net?

[zinid]

I got the PEM file, I'm checking.
Meanwhile, why don't I have it working with the staging url, i.e.:

acme:
    ca_url: "https://acme-staging.api.letsencrypt.org"

I have an error:

2017-11-17 08:28:33.600 [error] <0.732.0>@acme_challenge:ets_get_key_authorization:122 Unable to serve key authorization in: [<<"acme-challenge">>,<<"1ILvVy4wWEEPop462YLjaUdmTBZ...">>]

[zinid]

I found a problem with private key mismatch: that's because the certfile you receive is not signed by the intermediate certfile provided, because it's signed by "CN = h2ppy h2cker fake CA", but intermediate certfile is issued for "CN = happy hacker fake CA".

[zinid]

Can you please publish the patch somewhere? I will check with real domains.

[angelhof]

It is this branch https://github.com/angelhof/ejabberd/tree/acme_keep_issuer_cert

[zinid]

Sigh, I need now to mess with your branches?

[angelhof]

I can make a pr if you prefer

[zinid]

Is it so hard to publish git diff ...?

[zinid]

No matter, I got the idea from your branch's commits.

[zinid]

OK, your fixes make it work with real domains, thank you.

[zinid]

Another problem:

> ejabberd_acme:renew_certificates().
16:17:32.697 [warning] No acme configuration has been specified
{error,get_certificates}
(ejabberd@localhost)2> 16:17:32.697 [error] No CA url has been specified in configuration
16:17:32.699 [error] Unknown error:function_clause, [{ejabberd_acme,format_get_certificate,[{ok,<<"some.domain.net">>,no_expire}],[{file,"src/ejabberd_acme.erl"},{line,220}]},{ejabberd_acme,'-format_get_certificates_result/1-lc$^1/1-0-',1,[{file,"src/ejabberd_acme.erl"},{line,207}]},{ejabberd_acme,format_get_certificates_result,1,[{file,"src/ejabberd_acme.erl"},{line,207}]},{ejabberd_acme,renew_certificates,0,[{file,"src/ejabberd_acme.erl"},{line,378}]},{erl_eval,do_apply,6,[{file,"erl_eval.erl"},{line,674}]},{shell,exprs,7,[{file,"shell.erl"},{line,687}]},{shell,eval_exprs,7,[{file,"shell.erl"},{line,642}]},{shell,eval_loop,3,[{file,"shell.erl"},{line,627}]}]

[angelhof]

This is fixed by this:

diff --git a/src/ejabberd_acme.erl b/src/ejabberd_acme.erl
index cc64703..613fb84 100644
--- a/src/ejabberd_acme.erl
+++ b/src/ejabberd_acme.erl
@@ -112,9 +112,9 @@ get_commands_spec() ->
                        args_example = ["all | www.example.com;www.example1.net"],
                        args = [{domains, string}],
                        result = {certificates, string}},
-     #ejabberd_commands{name = renew_certificate, tags = [acme],
+     #ejabberd_commands{name = renew_certificates, tags = [acme],
                        desc = "Renews all certificates that are close to expiring",
-                       module = ?MODULE, function = renew_certificate,
+                       module = ?MODULE, function = renew_certificates,
                        args = [],
                        result = {certificates, string}},
      #ejabberd_commands{name = list_certificates, tags = [acme],
@@ -221,7 +221,7 @@ format_get_certificate({ok, Domain, saved}) ->
     io_lib:format("  Certificate for domain: \"~s\" acquired and saved", [Domain]);
 format_get_certificate({ok, Domain, not_found}) ->    
     io_lib:format("  Certificate for domain: \"~s\" not found, so it was not renewed", [Domain]);
-format_get_certificate({ok, Domain, exists}) ->    
+format_get_certificate({ok, Domain, no_expire}) ->    
     io_lib:format("  Certificate for domain: \"~s\" is not close to expiring", [Domain]);
 format_get_certificate({error, Domain, Reason}) ->
     io_lib:format("  Error for domain: \"~s\",  with reason: \'~s\'", [Domain, Reason]).

[zinid]

Indeed, thank you very much!

[lock]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs.