A guide to creating a LetsEncrypt client from scratch in < 150 lines of Ruby
Switch branches/tags
Nothing to show
Clone or download
Latest commit b426aa9 Apr 11, 2018

README.md

Building a Let's Encrypt client from scratch

A step-by-step guide to building a LE/ACME client in <150 lines of code

This is a (pretty detailed) how-to on building a simple ACME client from scratch, able to issue real certificates from Let's Encrypt. I've skipped things like error handling, object orientedness, tests - but not much tweaking would be needed for the client to be production-ready.

The code for the finished client is in client.rb.

About the guide

This guide assumes no particular knowledge of TLS/SSL, cryptography or ACME - a general understanding of programming, HTTP and REST APIs is probably needed. It would also be useful to have a vague idea of what Public-key cryptography is.

Hopefully this guide is useful to anyone looking to build a Let's Encrypt client, or anyone looking to understand more about how LE/ACME works. Following the guide, you should be able to create a fully fledged LE client and issue a valid certificate in less than an hour. The guide does assume you control a domain name.

Our specimen site is a static website powered by nginx, using DNSimple as the DNS provider (see Appendix 3: Our example site setup). The mechanics of how we pass LE's challenges are based on this sample setup - but treat these just as illustrative examples.

The guide and client code are all MIT licensed.

Technology

This example code is written in Ruby (I used 2.3), and is largely dependency free (apart from OpenSSL). We use HTTParty and Nitlink for more convenient API requests - but you could use vanilla Net::HTTP if you're a masochist 🙈. And we'll use additional gems to upload files and provision DNS records.

The choice of language is meant to be a background factor - the guide is (hopefully) illustrative & understandable even if you're not familiar with/interested in Ruby.

Credits

I heavily referenced Daniel Roesler's absolutely awesome acme-tiny and the ACME spec while writing this tutorial. I'd recommend checking out both as a supplement to this guide. Image credits at the bottom.

Table of Contents


1. Loading our client key-pair

The process of generating our certificate heavily depends on have a client key - or, more accurately key-pair (comprising our public key and private key).

We'll share our public key with Let's Encrypt when we register, and sign all our requests with our private key - Let's Encrypt can use our public key to ensure our requests are genuinely from us (that they've been signed by our private key). We'll never share our private key with Let's Encrypt. We won't share it with any 3rd parties; although our web-server (nginx in our example app) will need access to it in order to encrypt the data it sends to clients.

It's fine to use existing SSH keys, if you've already got them generated and they're long enough:

openssl rsa -in ~/.ssh/id_rsa -text -noout | head -n 1

If you see Private-Key: (2048 bit) or Private-Key: (4096 bit) you're good to go (if you're interested, there's more info about key size in Appendix 5). Otherwise, we'll need to generate them - Github has great instructions on how. Let's begin by loading our key-pair into Ruby:

require 'openssl'

client_key_path = File.expand_path('~/.ssh/id_rsa')
client_key = OpenSSL::PKey::RSA.new IO.read(client_key_path)

If our key is encrypted with a passphrase, we'll need to provide that as a 2nd argument:

client_key = OpenSSL::PKey::RSA.new(IO.read(client_key_path), 'letmein')

2. Constructing a Let's Encrypt API request

The first, and probably hardest step, is constructing requests in the very particular format that Let's Encrypt demands. It's important to remember though, that in principle, the Let's Encrypt API is the same as any other API.

For example, using the Github API I can programatically create an issue, by making a POST request to the target repo's /issues endpoint with a JSON payload that includes the issue title and body:

POST https://api.github.com/repos/alexpeattie/letsencrypt-fromscratch/issues

{
  "title": "Bad examples",
  "body": "The code examples in the guide are hard to understand!"
}

The key difference with the Let's Encrypt API is we can't just send our JSON payload in a nice human-readable format as above, because we'll be signing it with our client private key to prove our identity. This is what a request to the Let's Encrypt API looks like:

POST https://acme-v01.api.letsencrypt.org/acme/new-cert

{
  "payload": "eyJyZXNvdXJjZSI6Im5ldy1jZXJ0IiwiY3NyIjoiTUlJRVhEQ0NBa1FDQVFBd0Z6RVZNQk1HQTFVRUF3d01abWxzWlhNdWNHVm5MbU52TUlJQ0lqQU5CZ2txaGtpRzl3MEJBUUVGQUFPQ0FnOEFNSUlDQ2dLQ0FnRUE2ZG9JNWdlc1VWZVV2czJXN1h3LV9JcDg2eFl3ZnV0MDVNWE1aYWpWa3lMS1lhNHpjdGs3Y2hIN1ZuQWsxVF9uTXNaM0hYTlQ3X0J0R1hkYnlJR0FqRXhpR3F4cm5LejJqSS1JTVRNU1RKSklmRVhDUVJqUkx2U0c2S3VYbXk2aGhkS3BLMkpRam10OTh0QmxUY0NxbFFKNGRZWV9oMVFCTmYwZmUwN3p4T24zUXlaeU9Da05GMkdGQmZoSWZqTGRuVXJCbDBSejlTSUhLZkZTWW13SldKMTBBLWJiNVdRM2FkUWlNWF83amhYWHVBdUdDZnRBZ2h1UGdPWjlTalJXYVBpalNkOUxERWk1Y2pCalFsN1o4a0ZKTnV0VndSQlNFTDFIQVVNWE9ndkxKLW5mVjV4Tm15VHdmYTRsdXV4WEtsVnpJZFlmZDRUZWV1NHhwUTAxb29vQ0dLRUVCZ3VMQzdQLUtjemg4MUxXaTZtcExIRVZwOTNzWi1QZDZvNlROMFlabVZjaUwtNlJpTGRXY2hUeEtkbjNvTS1UYmRBTUVxb3VmTU5JYkh6LUVHREFxUkhGOUxCTU43bFlPcWJ0dWFmcjduN1EtVmQxN19KTGIxcnpONVFmclZvd2o4cUJpUHlRUndXbDhqN2hiLVpCR1NpMlJNb0V3LWNURG1KYjIweWUwQXZrWHhqVmxqbTN1aGpWVWRHTEtTQ0dfM1I4V0VuWEI3akRTV3Zpd0NEdDFKLWtPSW5EOEVUcjFvVDJKWWJ5N0FsaS12R25jdjJRdlhSb010RG9MN3F0MmkzSHNNZzhORjFDSHVhRUQ3RXdiTEMwRTRpWnZfcUw2WW45endqMVZ2bUZtbjA3T1ItanVOYkFnUXAtb01XR1lORDFKMnRpSW5QV0RtVUNBd0VBQWFBQU1BMEdDU3FHU0liM0RRRUJDd1VBQTRJQ0FRREdPdjUxc1hlUWNSLVhYMmUtbDZfSEt1WjNfVTdKbTJmNWtMMWJvbkpwOUM0UExacVNZMzNDZE5FbE1BcEVRczFzLTVhWEJCemRYWWE1X05hTFB2cm5fRm5mb2d1cnJHOXV6cU1vT0QtMjMtUnd5QkNLZFpNQ3gyVmd0YWNFU3RiZ2RLamNMRnRNRVE4YnR1NHIxMXVKQWlrblRIQnk4V3ZmaHREVS1Da0FkT2FYZV8zMktKSVV4Z05LSzhiYnRVUGlFc21jd3VqUGVzUkprWUh1QWVKc2JFQkY5ekVZNjlCazZiZVZKUUpxRjR4VjhYYmJheGZSX1N6TG5NWnJZNFhoNDNXbGRPN1UzZm9BZHYtLWk3eTlDbDUxaTJRV1RZMHFGcGVmd19nUU93SFFWMW9BRWJ0OWwyYkgyNGEtZ2NKUE9RNEhTdTBEV0ZHaFdSVkVuMUJsQ01XMkxGQnp2elpzMGdIaFhnQ1psVnNGcE1nYndJMThBLTA4UjZvS2FRWC1fM2tDb0FIaXcxQ1pdanaVQ1ZVOVRZNXNUMVlnZXBJVzBkT0VHYXY3YUJMXzNCbk9HVzVlMlZ2LXN5aGVSZS1ORzhXTEZiOHRyc2hMYTRPOVVjS3h3Nzl0MjFGaEhUYXhIblJLcDhFR3p3M2ZoZElMUW42YVlkb0k4Wm9faGJJaUE0cEhoMXlCbGpLU2Q3Zk1xTzkzX3JxV2Y4NzRfd2Q4N3RhcDFmb1pyZ1dYMVU5Wm9ZUnhFZ0FQOVN1cUdrcTJVUl9ucU9CQl9XaVBPM2ZGcFc3cTB6UEp1QUtBNWZIdDdFRG1HUldkTWNGXzM0SDdNenZPQk4tckI2S3VZTUtzWXpkS1ZEMDhwUnhUVVhKc3Nrb2t2MVF3aGNmNklzdEFtMDJ6bjhfWHBRIn0",
  "header": {
    "alg": "RS256",
    "jwk": {
      "e": "AQAB",
      "kty": "RSA",
      "n": "xVZG_h6B314tV_UNG-KUA_wldRuRjXvdcLwwtzOSBBjA1aGa-wabVUjazf2DrPWHlhiFlfom0sV0JgR2Ak5Ydlr4OOTqWCQ6m-ABnl71DvUs-u8eQwcLPsp-ccmRW3vYGuXoSP7-TEM9MSfAI-jeJ9vXeyDUGQDTD1FcBcZh886tR6LwyHBUbE0aD7I5I6pKr5kn24utnXcQ0LNoTOwjycexwzb-kGYHKfHdK5Chx1XLUkZIw7SYqePTchcBRsn6WOYLZ-orT4G58CRNbqpWa6qeRDijCOguUZfaJPuZLJl8ULIhtim0k1Y2e-X8tCNn-qacraicW6mPdlRcBUXAzQ"
    }
  },
  "protected": "eyJhbGciOiJSUzI1NiIsImp3ayI6eyJlIjoiQVFBQiIsImt0eTI6IlJTQSIsIm4iOiJ4VlpHX2g2QjMxNHRWX1VORy1LVUFfd2xkUnVSalh2ZGNMd3d0ek9TQkJqQTFhR2Etd2FiVlVqYXpmMkRyUFdIbGhpRmxmb20wc1YwSmdSMkFrNVlkbHI0T09UcVdDUTZtLTRMbmw3MUR2VXMtdThlUXdjTFBzcC1jY21SVzN2WUd1WG9TUDctVEVNOU1TZkFJLWplSjl2WGV5RFVHUURURDFGY0JjWmg4ODZ0UjZMd3lIQlViRTBhRDdJNUk2cEtyNWtuMjR1dG5YY1EwTE5vVE93anljZAv3emIta0dZSEtmSGRLNUNoeDFYTFVrWkl3N1NZcWVQVGNoY0JSc242V09ZTFotb3JUNEc1OENTTmJxcFdhNnFlUkRpakPNZ3VVWmZhSlB1WkxKbDhVTElodGltMGsxWTJlLVg4dENObi1xYWNyYWljVzZtUGRsUmNCVVhBelEifSwibm9uY2UiOiJidGY3SFpROHlvVERGNVphWjdaSnVGR05tOWR2cWhyNmdWVHR0NHZYbmFvIn0",
  "signature": "Mo1ZVEkT_QjsH4Yy98tTm3JEpsccnriVn5L18yjN2O1ea57V3apkDkkMb_3wleJ0YJskSuNrvtftJOC_-OqeT1_qbq4AjugEqMPle5I7VUAzshnh1DL7YiAgds5Fm06VtCuWUns5owF2MtVmjKMJHdHc9a_9-jilQsFWrTHEZgTt_ebBHazFpiEVcqoNCxhho-XxWZaHlvDOncJXUnqG0SWIa0OeM5Gm80jlPRlQoE5Wp6RqQvn1Fsb3NpzMUEQwD-s9JCvB4U2tQdpGLM5ynfbFwlgyS1AgKiQ4FLEftc55Yo9yOo0bXEugM7aDZS7-_TjqFD_N7r0IJHPp8fXrCQ"
}

This is what's called a JWS (JSON Web Signature), specifically a "JWS Using Flattened JWS JSON Serialization" from RFC 7515. Scary stuff eh 👻? Don't worry, we'll break down the anatomy of this strange looking request in the sections below.


a. Base64 all the things

One problem we'll run into is that when we sign our payload with our key, we might not get ASCII out, even if we're only putting ASCII in. We can see this for ourselves:

puts client_key.sign OpenSSL::Digest::SHA256.new, 'Hello world'
 #=> ��ۉ��7�xM��\�AU=�KGQ��ao�:Q-H�WW�a_Ԇ����+a≈��|X]�s}V�oya���'68L6����P�����f��yKV���≈�I@���a��[�����C���VXM+�≈��oQ�@�B�"]Uzr�N�R]]{9;�N:��G�ӗaM�S��H�ŵq���Bq�9��  ��So�Q���tk�;����z��d�<=�� +B≈_t�≈�����~����<˯ޤ≈�%Ê�k��

To avoid dealing with non-ASCII characters we'll need to Base64 encode most of our data. The good news is Ruby comes with Base64 handling as part of the standard library:

Base64.urlsafe_encode64('test')
 #=> "dGVzdA=="

There is a small tweak we'll need to make to keep Let's Encrypt happy - removing the padding characters (=) from our encoded data:

Base64.urlsafe_encode64('test').delete('=')
 #=> "dGVzdA"

(Or in Ruby 2.3)

Base64.urlsafe_encode64('test', padding: false)

Let's wrap that in a helper method - we'll be using it a lot as we build our request:

def base64_le(data)
  Base64.urlsafe_encode64(data).delete('=')
end

b. Payload

The payload is the simplest part of our request. It's just JSON that we'll Base64 encode using our method above:

base64_le '{"resource":"new-reg"}'
 #=> "eyJyZXNvdXJjZSI6ICJuZXctcmVnIn0"

This a totally valid payload that we can send to Let's Encrypt. Obviously it'll be more convienient not to have to construct JSON strings by hand - so let's load in the JSON library (again part of the Ruby standard lib):

require 'json'

base64_le JSON.dump(resource: 'new-reg')
 #=> "eyJyZXNvdXJjZSI6ICJuZXctcmVnIn0"

For further convenience, we can make our Base64 helper method smarter. If the data we pass in is an array or hash, it should JSONify the data before encoding it:

def base64_le(data)
  txt_data = data.respond_to?(:entries) ? JSON.dump(data) : data
  Base64.urlsafe_encode64(txt_data).delete('=')
end

That's all we need for our payload 😄! As well as providing information about the request we want to make, it will form one half of the data we'll be signing.


c. Header

We'll need to give Let's Encrypt two things for it to validate the authenticity of the request: our public key, and the cryptographic hashing algorithm we're using to generate the signature. This info will go in our header.

The static parts of our header are as follows:

header = {
  alg: 'RS256',
  jwk: {
    kty: 'RSA',
  }
}

alg corresponds with the hashing algorithm we want to use - in this case SHA-256 (or more technically RSA PKCS#1 v1.5 signature with SHA-256, but we don't really have to worry about that here). kty means key type - our keys are RSA keys. jwk stands for JSON web key - a standard for sharing keys via JSON.

The parts of the key we're interested in are the public key exponent (e) and the modulus (n). Helpfully our client_key has corresponding methods (client_key.e and client_key.n) - the only additionally steps we need to take are converting them to binary strings with to_s(2) (documented here), then (you guessed it), Base64 encoding them. Let's also create a header convenience method:

def header
  {
    alg: 'RS256',
    jwk: {
      e: base64_le(client_key.e.to_s(2)),
      kty: 'RSA',
      n: base64_le(client_key.n.to_s(2)),
    }
  }
end

d. Protected header and the nonce

We have our plaintext header - which contains the required components of our public key. We'll also need a protected header - basically a Base64 encoded version of our header which will form the other half of the data we'll be signing (alongside our payload).

The protected header contains one additional element which our unprotected header doesn't - a cryptographic nonce. The linked article goes into lots of details, but a nonce is basically a one-time use code which we must attach to our request. It means if an attacker somehow sniffs out a request we made, and makes a carbon-copy duplicate request, the attackers attempt will fail (because the nonce has already been used).

Let's Encrypt provides us a nonce in the headers of every response it gives us - so getting a nonce is just a case of requesting any Let's Encrypt API endpoint, and grabbing it from the Replay-Nonce header.

Ruby comes with the Net::HTTP library built in for making HTTP requests, but it's a bit cumbersome. To make our life easier, we'll use HTTParty - although this is by no means a necessity.

gem install httparty
require 'httparty'

(Note: you can also grab the Gemfile provided in this repository, and bundle install to save yourself some typing.)

We'll send HTTParty's debug output to $stdout so we can see easily see the requests/responses happening:

HTTParty::Basement.default_options.update(debug_output: $stdout)

As mentioned above, any Let's Encrypt API endpoint will do - let's use the /directory endpoint (effectively the root of the API). Because we only need the headers, we can just make a HEAD request:

nonce = HTTParty.head('https://acme-v01.api.letsencrypt.org/directory')['Replay-Nonce']

We can now create our protected header:

protected = base64_le(header.merge(nonce: nonce))

e. Signature

The last step to construct our request is proving its authenticity with a signature, generated using our client private key. First, let's consolidate everything we have so far:

require 'openssl'
require 'base64'
require 'json'
require 'httparty'

def base64_le(data)
  txt_data = data.respond_to?(:entries) ? JSON.dump(data) : data
  Base64.urlsafe_encode64(txt_data).delete('=')
end

client_key_path = File.expand_path('~/.ssh/id_rsa')
client_key = OpenSSL::PKey::RSA.new IO.read(client_key_path)

payload = { some: 'data'}

header = {
  alg: 'RS256',
  jwk: {
    e: base64_le(client_key.e.to_s(2)),
    kty: 'RSA',
    n: base64_le(client_key.n.to_s(2)),
  }
}

nonce = HTTParty.head('https://acme-v01.api.letsencrypt.org/directory')['Replay-Nonce']

request = {
  payload: base64_le(payload),
  header: header,
  protected: base64_le(header.merge(nonce: nonce))
}

As mentioned above, we'll be using the SHA-256 hash function for our signing:

hash_algo = OpenSSL::Digest::SHA256.new

The specific data we'll need to sign is our protected header and our payload, joined with a period:

request[:signature] = client_key.sign(hash_algo, [ request[:protected], request[:payload] ].join('.'))

f. Making requests

Now we've built the request data just as Let's Encrypt wants, we have everything we need to start making requests:

HTTParty.post(some_api_endpoint, body: JSON.dump(request))

Let's put everything into a reusable method that can take an arbitrary URL and payload. (We'll move client_key, hash_algo, header and nonce into methods at the same time):

HTTParty::Basement.default_options.update(debug_output: $stdout)

def client_key
  @client_key ||= begin
    client_key_path = File.expand_path('~/.ssh/id_rsa')
    OpenSSL::PKey::RSA.new IO.read(client_key_path)
  end
end

def header
  @header ||= {
    alg: 'RS256',
    jwk: {
      e: base64_le(client_key.e.to_s(2)),
      kty: 'RSA',
      n: base64_le(client_key.n.to_s(2))
    }
  }
end

def hash_algo
  OpenSSL::Digest::SHA256.new
end

def nonce
  HTTParty.head('https://acme-v01.api.letsencrypt.org/directory')['Replay-Nonce']
end

def signed_request(url, payload)
  request = {
    payload: base64_le(payload),
    header: header,
    protected: base64_le(header.merge(nonce: nonce))
  }
  request[:signature] = client_key.sign(hash_algo, [ request[:protected], request[:payload] ].join('.'))

  HTTParty.post(url, body: JSON.dump(request))
end

g. Fetching the endpoints

The /directory endpoint that we use to fetch our nonce serves another purpose: it lists all the other endpoints which act as the starting points for all the core actions (registering a user, authorizing a domain, issuing a certificate etc.):

{
  "key-change": "https://acme-v01.api.letsencrypt.org/acme/key-change",
  "meta": {
    "terms-of-service": "https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf"
  },
  "new-authz": "https://acme-v01.api.letsencrypt.org/acme/new-authz",
  "new-cert": "https://acme-v01.api.letsencrypt.org/acme/new-cert",
  "new-reg": "https://acme-v01.api.letsencrypt.org/acme/new-reg",
  "revoke-cert": "https://acme-v01.api.letsencrypt.org/acme/revoke-cert"
}

(Note: unlike the API's endpoints, the directory is viewable without any kind of signing, you can just visit it in your browser).

Here the keys in the JSON object indicate the resource type, and the values are the URI we'll need to make a signed request to. Even though Cool URIs don't change, using the directory means we don't have to hard-code the endpoints - and so our client is more resilient to any changes Let's Encrypt might make (credit to @kelunik for suggesting this).

To avoid making repeated requests to the directory, let's make an endpoints method:

def endpoints
  @endpoints ||= HTTParty.get('https://acme-v01.api.letsencrypt.org/directory').to_h
end

Since we're referencing the directory endpoint in both our endpoints and nonce methods, we can dry up our code by moving it into a constant. This will also make it easier to, for example, switch to LE's staging server.

DIRECTORY_URI = 'https://acme-v01.api.letsencrypt.org/directory'.freeze

def nonce
  HTTParty.head(DIRECTORY_URI)['Replay-Nonce']
end

def endpoints
  @endpoints ||= HTTParty.get(DIRECTORY_URI).to_h
end

The neat thing is that this DIRECTORY_URI is the only URI we need to hardcode, every other endpoint we can either pull from the directory, or from the Location or Link headers of the API's responses. Location is easy to work with (it's just a single URI) - but Link headers need to be parsed. I've written a gem (Nitlink) for just that - so let's install and load it:

gem install nitlink
require 'nitlink/response'

3. Registering with Let's Encrypt

OK, we've laid the foundations - let's make our first actual request to the Let's Encrypt API! The first step is to register our client public key with Let's Encrypt.

Since we're sending the public key with every request (in the header property of our JSON), we don't need to include much in the actual registration payload. At a minimum, we'll just need to specify the resource type: new-reg in this case.

new_registration = signed_request(endpoints['new-reg'], {
  resource: 'new-reg',
})

We can optionally provide contact details (highly recommended), this will allow us to recover our key in case we lose it. We'll need to include the protocols for the contact details we provide - e.g. mailto: for email addresses, tel: for phone numbers (it turns out Let's Encrypt doesn't currently support this, see here).

new_registration = signed_request(endpoints['new-reg'], {
  resource: 'new-reg',
  contact: ['mailto:me@alexpeattie.com']
})

Responses

Sending the request should give us back a successful response:

-> "HTTP/1.1 201 Created"
-> "Content-Type: application/json"
-> "Location: https://acme-v01.api.letsencrypt.org/acme/reg/12345"
-> "Link: <https://acme-v01.api.letsencrypt.org/acme/new-authz>;rel=\"next\", <https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf>;rel=\"terms-of-service\""
...
{
  "id": 12345,
  "key": {
    "kty": "RSA",
    "n": "wlpAF2eAhpzJDGCco-c9hhd31NGAyhkFeivqfmt7ZQiphRiuSwF_0_3lOnCRpdpRIeVheIPVK6FofcFVmRjzdyDeZmN5ssk5oi2v1y8hSB7SM2QCoqlZ3L8uEGKzzwQfzSIQGIR56X5GrTKaCjBrzqrSM0VzRg5-gp8ZDqsyceSUaf7SgScxexfgbcaRXtJ1aVLYT5FfsDgV768gRcBxaKQapFQ47M7JN8OTOq6QIla6acp24eNo6PMtH8Mf0hJwpcWOs2A_0VcNzV7XBl8shYEeERyqbNXIZsF7njF8WInk7-v0EiYPV2w0xjBuFnbX7cw8YqveG81yirYGScR5ASeER5dxtWNyXFXkK9KpI13Vvf-0ivzrgeJTUsKz7EAjL2vof2QleKZHjP6f63rvaIMK5FaGojhHSzzMdeP3FaG1mP7N5vY3J0oZzhny_Jd9vNysCiklsUNUr8ZT-ocTKHbiO6ZEZdj8Wtjmpr5kvfPUtosNodaMUNFv-7UFRWNf49qJKo21UzpeeM7Us0hKPNVd9VU0qD0jsya7w1EjimiBqwo6vD_KoH-R2bwWlaQ9Ucy6ahfNPogI3zqMTpUfMXGA0uMj6anp-daOSwuEus2ogY0x12OUn3XivB9VzbCNadAT9JqKRrhRHE-7tfN6TFt7CtLjGCe1ShMn3wsMFBU",
    "e": "AQAB"
  },
  "contact": ["mailto:me@alexpeattie.com"],
  "initialIp": "101.222.66.199",
  "createdAt": "2015-12-12T12:07:23.755314388Z"
}

The successful response basically just echoes back to us our registration details. We can see the exponent + modulus (e and n) values of our public key included at the top, as well as the unique id of our new account.

Note that LE verifies the domains of emails we provide (by checking their DNS A record), so make sure it's a real domain, otherwise you'll get an 400 (Bad Request) response:

{
  "type": "urn:acme:error:malformed",
  "detail": "Error creating new registration :: Validation of contact mailto:alex@artichokesandarmadillos.com failed: Server failure at resolver",
  "status": 400
}

If we try and register the same key again we'll get a 409 (Conflict) response:

-> "HTTP/1.1 409 Conflict"
-> "Content-Type: application/problem+json"
-> "Location: https://acme-v01.api.letsencrypt.org/acme/reg/12345"
...
-> "Connection: close"
{
  "type": "urn:acme:error:malformed",
  "detail": "Registration key is already in use",
  "status": 409
}

Don't worry, there are no side effects to attempting to re-register the same client key multiple times ☺️.

Accepting the ToS

Although we've successfully registered, Let's Encrypt won't let us do anything useful (like issue a certificate), until we accept their Subscriber Agreement.

To indicate our acceptance, we just need to make a request to the URI of our newly created user (returned in the response's Location header, in this case https://acme-v01.api.letsencrypt.org/acme/reg/12345) with the payload's agreement key set to the URI of the terms we're accepting. How do we know the URI of the terms? Eagle-eyed readers might have spotted above that it's returned as one of the response's Link headers:

Link: ... <https://letsencrypt.org/documents/LE-SA-v1.1.1-August-1-2016.pdf>;rel="terms-of-service"

Update August 2017: The URL of the current terms are now also available through the directory, and thus via our endpoints method, specifically endpoints['meta']['terms-of-service'].


We should also check that we got a 201 status (not a Conflict or malformed registration). Our final code for accepting the terms programatically looks like this:

if new_registration.status == 201
  signed_request(new_registration.headers['Location'], {
    resource: 'reg',
    agreement: new_registration.links.by_rel('terms-of-service').target
  })
end

(The .links method depends on the Nitlink gem, we'll get a NoMethodError if it's not installed). Notice that the resource type has changed, since we're not creating a new user, but modifying an existing one. Also, a real client should probably prompt the user to actually read the agreement - rather than just auto-accepting it 😇!


4. Passing the challenge

The next step is to inform Let's Encrypt which domain or subdomain we to provision a certificate for. In this guide I'm using the example le.alexpeattie.com. This is the first part of a multistep verification process to prove we're the legitimate owner of the domain:

  • a). We ask LE for the challenge
  • b). LE gives us a challenge to prove we control the domain
  • c) or d). We complete the HTTP- or DNS-based challenge, and notify LE that we're ready
  • e). LE checks the challenge has been completed to it's satisfaction
  • f). We verify that LE is happy the challenge has been passed 🏆

Challenges are how we prove a sufficient level of control over the identifier (domain name) in question. We can do this either by serving a specific response when LE hits a specific URL (which generally means uploading a file to our web-server), provisioning a DNS record, or by leveraging the Server Name Indication extension of TLS to serve a special self-signed certificate.

We'll cover the first two kinds of challenge: http-01 and dns-01 but not the third (tls-sni-01).


a. Asking Let's Encrypt for the challenges

Asking LE for our new challenge is just a case of making another request to the LE API - this time to create a new-authz (authz is short for authorization).

As you can probably guess, this means making a request to the new-authz endpoint with our resource option set to (you guessed it) new-authz. Beyond that just need tell LE what identifier (domain name) we want to authorize:

auth = signed_request(endpoints['new-authz'], {
  resource: 'new-authz',
  identifier: {
    type: 'dns',
    value: 'le.alexpeattie.com'
  }
})

The ACME spec is designed to be flexible enough to authorize more than just domain names in the future - which is why we have to explicitly state we're authorizing a domain name with type: 'dns'. We could authorize the root domain with value: 'alexpeattie.com'. We can also provide a Punycode encoded IDN, see Appendix 6.


b. Let's Encrypt gives us our challenges

Let's Encrypt should send up back a nice meaty response like the below 🍖 -

{
  "identifier": {
    "type": "dns",
    "value": "le.alexpeattie.com"
  },
  "status": "pending",
  "expires": "2016-01-15T19:28:33.644298086Z",
  "challenges": [{
    "type": "tls-sni-01",
    "status": "pending",
    "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/-gPc-DOOMPAqlaNV2_NCbwieC7cDgmsDxS4d0Ounp8A/5157173",
    "token": "rsFpjtnLgfXS8hMrAAcSsXJ98q7YNlA2Iyky-EWmoDY"
  }, {
    "type": "http-01",
    "status": "pending",
    "uri": "https://acme-v01.api.letsencrypt.org/acme/challenge/-gPc-DOOMPAqlaNV2_NCbwieC7cDgmsDxS4d0Ounp8A/5157174",
    "token": "w2iwBwQq2ByOTEBm6oWtq5nNydu3Oe0tU_H24X-8J10"
  }, {
    "type": "dns-01",
    "status": "pending",
    "uri": "https: //acme-v01.api.letsencrypt.org/acme/challenge/-gPc-DOOMPAqlaNV2_NCbwieC7cDgmsDxS4d0Ounp8A/5157175",
    "token": "U-85Krl7E2bPhqhdrjTuBoeIc7IVJ7Z4wyUhhn0uij0"
  }],
  "combinations": [
    [0],
    [2],
    [1]
  ]
}

Let's break this down. First our "identifier" is echoed back to us, along with its "status" - right now it's "pending" which means we haven't proven to LE that we control the domain; we're aiming to change it to "valid". Our challenge also has an expiry date - 1 week from now at the time of writing.

http_challenge, dns_challenge = ['http-01', 'dns-01'].map do |challenge_type|
  auth['challenges'].find { |challenge| challenge['type'] == challenge_type }
end

The "uri" of the challenge will allow us to notify LE that we're ready to take the challenge, on to check if we've passed. The "token" is a unique, unguessable, random value sent to us by LE that we'll need to incorporate into our challenge response to prove we control the domain.

"combinations" is a another feature that's designed for the future. Right now we only have to pass 1 challenge to convince LE we control the domain. In the future we might see something like this:

"challenges": [{
  "type": "email-01",
  "..."
}, {
  "type": "http-01",
}, {
  "type": "tls-sni-01",
}, {
  "type": "dns-01",
}],
"combinations": [
  [0, 1],
  [2],
  [3]
]

Which would mean we'd have to either pass both challenges 0 & 1 (the "email-01" and "http-01" challenges), or challenge 2 or challenge 3 ("tls-sni-01" or "dns-01").


c. Option 1: Completing the http-01 challenge

Our first option is the http-01 challenge. To pass this we need to ensure that when LE makes a request to

http://<< Domain >>/.well-known/acme-challenge/<< Challenge token >>

They receive a specific response (more on that below). Our domain is le.alexpeattie.com, .well-known/acme-challenge/ is a fixed path defined by ACME, and our challenge token is w2iwBwQq2ByOTEBm6oWtq5nNydu3Oe0tU_H24X-8J10, so the endpoint we'll need to serve the response from is:

http://le.alexpeattie.com/.well-known/acme-challenge/w2iwBwQq2ByOTEBm6oWtq5nNydu3Oe0tU_H24X-8J10

The key authorization

First we'll create our key authorization: the special response LE expects to be served. It's quite simple - it's the challenge token and a 'thumbprint' of our public key joined with a period.

We're using the JSON Web Key standard to share details of our public key already (in the jwk field of our header). To generate the thumbprint we need to generate a digest of that JSON using SHA256, and Base64 encode it (see RFC 7638 for more).

Our final code for the thumbprint method looks like this:

def thumbprint
  jwk = JSON.dump(header[:jwk])
  thumbprint = base64_le(hash_algo.digest jwk)
end

And for our final challenge response:

http_challenge_response = [http_challenge['token'], thumbprint].join('.')
Uploading the challenge response

To prove to LE that we control a domain, http://example.com/.well-known/acme-challenge/<< Challenge token >> needs respond with << Challenge token >>.<< JWK thumbprint >>. Because this is just a toy client, we'll create the file locally, then upload it (using SCP) to our remote nginx server - a more usual approach would be to run the LE client on the server (so we can just write the necessary files directly to disk).

We'll use the net-scp gem for easier SCP uploads:

gem install net-scp

Since we're serving static files with nginx from /usr/share/nginx/html, so we'll first want to create the .well-known/acme-challenge directory:

ssh root@162.243.201.152 'mkdir -p /usr/share/nginx/html/.well-known/acme-challenge'

The code for uploading the challenge is quite straightforward:

require 'net/scp'

def upload(local_path, remote_path)
  server_ip = '162.243.201.152' # see Appendix 3
  Net::SCP.upload!(server_ip, 'root', local_path, remote_path) 
end

# ..
destination_dir = '/usr/share/nginx/html/.well-known/acme-challenge/'

IO.write('challenge.tmp', http_challenge_response)
upload('challenge.tmp', destination_dir + http_challenge['token']) and File.delete('challenge.tmp')

Our simple nginx setup (see Appendix 3) serves static files (if they exist) for any endpoint, so this should be all we need to ensure that a request to http://le.alexpeattie.com/.well-known/acme-challenge/w2iwBwQq2ByOTEBm6oWtq5nNydu3Oe0tU_H24X-8J10 returns our key authorization as its response (we can verify this in a browser).


d. Option 2: Completing the dns-01 challenge

The dns-01 challenge was introduced at the beginning of 2016, allowing us to authorize our domain(s) by provisioning DNS records. The key differences between the http-01 challenge and the dns-01 challenge are:

  • We'll add a DNS TXT record rather than uploading a file
  • Rather than using "raw" key authorization as the record's contents, we'll use its (Base64 encoded) SHA-256 digest (see below)

There are lots of ways to add the required DNS record - most DNS services provide a web interface (instructions for common providers here) - we'll be programatically adding a record using the DNSimple API & associated gem.

The key ingredients of a DNS record are its type, name and value/contents. The type of the record is TXT, which is designed for adding arbitrary text data to a DNS zone. The name of the record takes the format _acme-challenge.subdomain.example.com. The root domain name is appended to a record's name automatically, so we just need to provide the name as _acme-challenge.subdomain or just _acme-challenge if we're authorization the root domain.

record_name = '_acme-challenge.le'

To construct the contents of our record, we'll start by creating our "raw" challenge response in the same manner as in the http-01 challenge:

raw_challenge_response = [dns_challenge['token'], thumbprint].join('.')

Additionally, for the dns-01 we'll need to digest the challenge response, and run it through our base64_le method:

dns_challenge_response = base64_le(hash_algo.digest raw_challenge_response)
Adding the record

We'll use the dnsimple-ruby gem to add our TXT record:

gem install dnsimple -v 2.2

We'll also need to get our API token from the DNSimple web interface. Then using the gem to add the TXT record, with the correct record name & content. We'll set a relatively low TTL (time to live) of 60 seconds, because we don't want our resolvers to cache the record for long - in case we need to redo the challenge, for example.

require 'dnsimple'

dnsimple = Dnsimple::Client.new(username: ENV['DNSIMPLE_USERNAME'], api_token: ENV['DNSIMPLE_TOKEN'])
challenge_record = dnsimple.domains.create_record('alexpeattie.com', {
  record_type: 'TXT',
  name: record_name,
  content: dns_challenge_response,
  ttl: 60
})

Lastly, we'll use Ruby's Resolv library (part of the std lib) to wait until the challenge record's been added:

loop do
  resolved_record = Resolv::DNS.open { |r| r.getresources(record_name + '.alexpeattie.com', Resolv::DNS::Resource::IN::TXT) }[0]
  break if resolved_record && resolved_record.data == challenge_response

  sleep 5
end

e. Telling LE we've completed the challenge

To tell LE we've completed the challenge, we need to make a request to the challenge URI we got earlier (https://acme-v01.api.letsencrypt.org/acme/challenge/-gPc-DOOMPAqlaNV2_NCbwieC7cDgmsDxS4d0Ounp8A/5157174 or /5157175).

Our request needs to include the field keyAuthorization with the key authorization we've just generated:

signed_request(http_challenge['uri'], { # or dns_challenge['uri']
  resource: 'challenge',
  keyAuthorization: http_challenge_response # or dns_challenge_response,
})

f. Wait for LE to acknowledge the challenge has been passed

Finally it's just a case of polling the challenge URI we've been given and wait for its status to become "valid". If it's still "pending" we'll sleep for 2 seconds then try again. Any other status means something's gone wrong 😭.

loop do
  challenge_result = HTTParty.get(challenge['uri'])  # or dns_challenge['uri']

  case challenge_result['status']
    when 'valid' then break
    when 'pending' then sleep 2
    else raise "Challenge attempt #{ challenge_result['status'] }: #{ challenge_result['error']['details'] }"
  end
end

If we chose the DNS challenge, we should also clean up after ourselves by deleting the record (so our challenge attempt doesn't interfere with future challenge attempts, which will also require TXT records using the _acme-challenge.le name):

dnsimple.domains.delete_record('alexpeattie.com', challenge_record.id)

5. Issuing the certificate 🎉

We've proven to Let's Encrypt we control the domain, which means we can now get our certificate. We'll need to generate a Certificate signing request (CSR). The CSR includes the public part of the key-pair tied to the certificate - secure traffic will be encrypted with the corresponding private part of the key-pair.

It's best to create a new key-pair for our CSR. We can generate it on the command line (as for the client key-pair), or with Ruby:

domain_key = OpenSSL::PKey::RSA.new(4096)
IO.write('domain.key', domain_key.to_pem)

*You might alternatively want to use a 2048 bit key (see Appendix 5 for more).

Ruby's OpenSSL module makes the generation of the CSR very straightforward:

csr = OpenSSL::X509::Request.new
csr.subject = OpenSSL::X509::Name.new([['CN', 'le.alexpeattie.com']])
csr.public_key = domain_key.public_key
csr.sign domain_key, hash_algo

We need to set the subject of the CSR - in this case the common name (domain name) we want to secure. Then we sign our certificate with our (private) domain_key.

LE needs us to send CSR in binary (.der) format - Base64 encoded of course. We'll be making a request for a new-cert:

certificate_response = signed_request(endpoints['new-cert'], {
  resource: 'new-cert',
  csr: base64_le(csr.to_der),
})

Let's Encrypt should respond with our brand new, DV certificate 🎉 🎉. It's not quite ready to use though.

Formatting tweaks

Certificates should be typically enclosed by a -----BEGIN CERTIFICATE----- header and -----END CERTIFICATE----- (RFC here) with each line wrapped at 64 characters. We could either do that manually (e.g. see tiny-acme's implementation) or let OpenSSL::X509::Certificate take care of it:

certificate = OpenSSL::X509::Certificate.new(certificate_response.body)

Adding intermediates

We also need to complete our trust chain, which means grabbing the LetsEncrypt cross-signed intermediate certificate (see https://letsencrypt.org/certificates/). Some browsers will resolve an incomplete trust chain, but it's something we want to avoid. There's much more info on why we need to complete this step and the difference between the different intermediates LE offers in Appendix 2: The trust chain & intermediate certificates.

Occasionally server software might want us to provide our intermediate certificates separately, but generally we'll bundle them together in a single file. Helpfully, LE provides a link to the latest intermediate certificate in the certificate response's Link header (it has the relation type "up"):

Link: </acme/issuer-cert>;rel="up"
intermediate = OpenSSL::X509::Certificate.new HTTParty.get(certificate_response.links.by_rel('up').target).body
IO.write('chained.pem', [certificate.to_pem, intermediate.to_pem].join("\n"))

That's it - we're done with our client and have our certificate (valid for the next 90 days) that will be accepted by all major browsers ! Completed authorizations are valid for 300 days, so we can our renew certificate without needing to take a challenge during that period.

This is the end of the main part of the guide, if you're interesting in the logistics of installing the certificate, keep reading...




Appendix 1: Installing and testing the certificate

Installation (with nginx)

Now we have our certificate, it's just a case of uploading it along with our private key and tweaking our nginx configuration to enable TLS. As with our HTTP challenge response, we can upload the necessary files with SCP using our upload helper method:

upload('chained.pem', '/etc/nginx/le-alexpeattie.pem')
upload('domain.key', '/etc/nginx/le-alexpeattie.key')

Then we'll need to point our nginx.conf to our certificate and key:

server {
  listen 443 ssl deferred;
  server_name le.alexpeattie.com;

  ssl_certificate /etc/nginx/le-alexpeattie.pem;
  ssl_certificate_key /etc/nginx/le-alexpeattie.key;
}

That's theoretically all we need, but we can improve on nginx's defaults for better security and performance. We'll use the settings recommended by https://cipherli.st/ (click "Yes, give me a ciphersuite that works with legacy / old software." if you need to support older browsers) and a couple of extra headers recommended by securityheaders.io. We'll use Google's DNS server (8.8.8.8) as our resolver (recommended for OSCP stapling on nginx):

ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "EECDH+AESGCM:EDH+AESGCM:AES256+EECDH:AES256+EDH";
ssl_ecdh_curve secp384r1;
ssl_session_cache shared:SSL:10m;
ssl_session_tickets off;
ssl_stapling on;
ssl_stapling_verify on;
resolver 8.8.8.8;
resolver_timeout 5s;
# add_header Strict-Transport-Security "max-age=63072000; preload" always;
add_header X-Frame-Options DENY always;
add_header X-Content-Type-Options nosniff always;
add_header X-XSS-Protection "1; mode=block" always;
add_header Content-Security-Policy "default-src https: data: 'unsafe-inline' 'unsafe-eval'" always;

(Note: that the Content-Security-Policy header will prevent assets being loaded over HTTP - this is recommended, but could break some sites. Read more about CSPs here)

We should keep the line enabling the Strict-Transport-Security header commented out until we're happy our HTTPS setup is working (as visitor's won't be able to access our non-HTTPS site once it's activated).

We can harden our configuration by dropping support for TLS < v1.2 - although that does have implications for supporting older browsers. If we happy to target just older browsers, we should also allow only cipher suites with a minimum 256-bit key length for AES (the symmetric cipher):

ssl_protocols TLSv1.2;
ssl_prefer_server_ciphers on;
ssl_ciphers "AES256+EECDH:AES256+EDH";

Unless we're using an ECDSA certificate (see Appendix 5) we should also generate a stronger DH parameter - nginx uses a 1024-bit prime which has been shown to be potentially vulnerable to state level adversaries (https://weakdh.org). Ideally our DH parameter shouldn't be smaller than our key size (i.e. 4096-bit or 2048-bit). We can generate a DH parameter like so:

ssh root@162.243.201.152

cd /etc/nginx
openssl dhparam -out dhparam.pem 4096

Bear in mind, the above is slooow (it took about 30 minutes for me) - so an alternative is to take a pre-generated prime from here:

curl -o dhparam.pem https://2ton.com.au/dhparam/4096/`shuf -i 0-127 -n 1`
openssl dhparam -in dhparam.pem -noout -text | head -n 1
#=>    PKCS#3 DH Parameters: (4096 bit)

Either way we'll need to tell nginx to use our stronger DH parameter:

ssl_dhparam /etc/nginx/dhparam.pem;

Lastly, we can redirect all HTTP traffic to our HTTPS endpoint:

server {
  listen 80;
  server_name le.alexpeattie.com;
  return 301 https://$host$request_uri;
}

Our final nginx.conf looks like this:

events {
  worker_connections  1024;
}

http {
  sendfile on;
  server_tokens off;
  root /usr/share/nginx/html;

  ssl_protocols TLSv1.2;
  ssl_prefer_server_ciphers on;
  ssl_ciphers "AES256+EECDH:AES256+EDH";
  ssl_ecdh_curve secp384r1;
  ssl_session_cache shared:SSL:10m;
  ssl_session_tickets off;
  ssl_stapling on;
  ssl_stapling_verify on;
  resolver 8.8.8.8;
  resolver_timeout 5s;
  add_header Strict-Transport-Security "max-age=63072000; preload";
  add_header X-Frame-Options DENY;
  add_header X-Content-Type-Options nosniff;

  server {
    listen 443 ssl deferred;
    server_name le.alexpeattie.com;

    ssl_certificate /etc/nginx/le-alexpeattie.pem;
    ssl_certificate_key /etc/nginx/le-alexpeattie.key;
    ssl_dhparam /etc/nginx/dhparam.pem;
    ssl_trusted_certificate /etc/nginx/le-alexpeattie.pem;
  }

  server {
    listen 80;
    server_name le.alexpeattie.com;
    return 301 https://$host$request_uri;
  }
}

Testing

Lastly let's run some tests to ensure our certificates are correctly and securely installed. There are a few tools out there, Qualys SSL Server Test is the most widely used. Using our new certificate with the strict cipher list, with either an ECDSA certificate or a standard certificate with a 4096-bit DH param we'll net top marks with a perfect A+ score:

A+ perfect score

Using cipherli.st's recommended ciphers, we'll score fractionally lower, with 90 points Cipher Strength:

A+ almost perfect score

We also do well on securityheader.io test:

A grade on securityheaders.io

As the report points out, we can harden our set up even further by implementing HTTP Public Key Pinning which could protect us if, for instance, Let's Encrypt itself was successfully attacked. However, this is currently considered quite advanced: as Peter Eckersley warns "HPKP pinning carries an inherent risk of bricking your site". But he does give some detailed best practices for the brave souls who do want to implement it.

Some other useful testing tools:


Appendix 2: The trust chain & intermediate certificates

The trusted status of a certificate (what gives us the green padlock) stems from a relatively small set of trusted Certificate Authorities (CAs) with corresponding "Root certificates". These are stored in the "trust stores" of browsers or operating systems. We can see Mac OS's trusted roots by going to Keychain Access -> System Roots for example:

If our certificate has been issued by a trusted CA (in our trust store) that certificate is trusted. If the CA isn't in our trust store, we can check if certificate of that CA was issued by a trusted root CA. A certificate issued by a CA, issued by another CA which was issued by a trusted CA is trusted, and so on. The trust chain can involve as many untrusted or "intermediate" CAs as we want, as long as it ultimately goes back to a trusted (root CA).

If it's still unclear, imagine Alice & Bob are having a birthday party. Guests who are invited by Alice or Bob can in turn invite other guests - those guests can invite other guests and so on. At the party, only guests who can prove their invitation leads back to Alice and Bob are trusted:

  • Carol ← invited by Bob = trusted
  • Doug ← invited by Steve ← invited by Alice = trusted
  • Fred ← invited by Gerard ← invited by Eve = untrusted ⛔️

At the party, a guest would have to provide information so we could verify the chain of invites led back to Alice or Bob. In the same way, a certificate should provide information about the chain of certificates (called the trust chain) which lead back to a trusted root CA.

In the future Let's Encrypt hopes to have its own trusted root CA: ISRG Root X1. Right now ISRG Root X1 isn't trusted by any browsers or operating systems - so IdenTrust is acting as their root CA instead. Let's Encrypt's intermediate CA is issued directly by Identrust's root CA (which is trusted by all major browsers/OSes) so our trust chain is only three links long:

  • Our certificate ← issued by Let's Encrypt CA ← 'issued' by Identrust CA

'Issued' is a bit of oversimplification here - in fact, Identrust just cross-signed LE's CA certificate, but it achieves the same end-result: trust in all major browsers/OSes.

So our complete trust chain should include our certificate, the certificate of Let's Encrypt's intermediate CA (Let’s Encrypt Authority X3), and optionally the Identrust CA's trusted root certificate. In reality there's no point making the client download the root certificate - it needs to already be in the trust store anywhere. As RFC 2246 says:

Because certificate validation requires that root keys be distributed independently, the self-signed certificate which specifies the root certificate authority may optionally be omitted from the chain, under the assumption that the remote end must already possess it in order to validate it in any case.

So basically we just need to concatenate our certificate with Let's Encrypt CA's certificate and we have a complete chain of trust* 👍.

FF 44 Chrome 48 IE 11 Safari 7.1 iOS 8 (Safari) Windows Phone 8.1 Android 6

*Some servers (like Apache) might want us to provide the our certificate and the rest of the trust chain separately. In this case the rest of the chain would just be the LE intermediate certificate.

Missing certificate chain

If we only provide our certificate without LE's intermediate certificate, we have a broken chain of trust. Most browsers can actually recover from this. LE certificates leverage Authority Information Access which embeds information about the trust chain even if we (system admins) forget to provide it.

We shouldn't rely on this though, most mobile browsers don't support AIA - nor does Firefox (who have explicitly said they won't be adding it).

Here's the result you'll get without providing the intermediate certificate:

FF 44 Chrome 48 IE 11 Safari 7.1 iOS 8 (Safari) Windows Phone 8.1 Android 6
⛔️ ⛔️ ⛔️ ⛔️

LE root certificate

Let's Encrypt has it's own root certificate authority that's separate from Identrust, called ISRG Root X1. When this root CA is widely trusted, expect it to take Identrust's place in the trust chain. We can already use an intermediate certificate issued by ISRG Root X1 - the problem is that, at the time of writing, ISRG Root X1 isn't trusted anywhere (to my knowledge).

If we try using the LE root-signed intermediate now, most browsers that support AIA will fallback to the valid trust chain, except desktop Safari.

FF 44 Chrome 48 IE 11 Safari 7.1 iOS 8 (Safari) Windows Phone 8.1 Android 6
⛔️ ⛔️ ⛔️ ⛔️ ⛔️

Appendix 3: Our example site setup

Below are the instructions to recreate the site setup used as the exemplar in this guide. You'll need:

  • A domain name you control
  • A DNSimple account (from $5/month, 30 day trial)
  • A DigitalOcean droplet (from $5/month)

1. Point our domain's nameservers to DNSimple

Digital Ocean has good instructions that cover common registrars. We'll want to point the nameservers to ns1.dnsimple.com, ns2.dnsimple.com, ns3.dnsimple.com and ns4.dnsimple.com. You'll need to copy over any existing records from your previous DNS provider.

2. Create our nginx server

First we'll need to create our droplet. We'll use a $5/month Ubuntu droplet:

Creating droplet

We'll also want to add our local machine's SSH key(s). We want to paste the public part of our key (e.g. cat ~/.ssh/id_rsa.pub):

Adding SSH key

Once our machine has been provisioned, take a note of the public IP, in this case 162.243.201.152:

Droplet's public IP

Using the IP, we'll SSH into our new box and install nginx:

ssh root@162.243.201.152

add-apt-repository ppa:nginx/stable
apt-get update
apt-get install nginx

A configuration like the below will be sufficient for passing the challenges - we'll update it when we actually install our certificate. This needs to go in /etc/nginx/nginx.conf:

events {
  worker_connections  1024;
}

http {
  sendfile on;
  server_tokens off;
  root /usr/share/nginx/html;

  server {
    listen 80;
    server_name le.alexpeattie.com;
  }
}

Lastly we'll restart nginx:

sudo service nginx restart

4. Point our subdomain to DigitalOcean

Log in to DNSimple, go to Domains and hit DNS in the sidebar:

DNSimple sidebar

The click + Manage records. We want to add an A record:

Add A record

We'll need to enter the name (our subdomain le) and set Address to our droplet's Public IP:

Configure A

We should be ready to go, and the domain (e.g. <le.alexpeattie.com>) should serve the default nginx welcome page. We might have to wait a while for our DNS changes to propagate.

nginx welcome

Once we've been issued our certificate, we can install it following the steps in Appendix 1.


Appendix 4: Multiple subdomains

Let's Encrypt can issue a single certificates which cover multiple, using the SubjectAltName extension. At the time of writing, Let's Encrypt supports a maximum of 100 SANs per certificate (full LE rate limits are detailed here).

LE has quite conservative per-domain rate limits right now (20 distinct certificates per domain per week) - so using SANs is crucial if you have lots of subdomains to secure*.

A common use-case is having a single certificate cover the naked domain and www. prefix. We have to authorize both domains; LE doesn't take it for granted that if we control the root domain we also control the www. subdomain or vice-versa.

domains = %w(example.com www.example.com)

domains.each do |domain|
  auth = signed_request(endpoints['new-authz'], {
    resource: 'new-authz',
    identifier: {
      type: 'dns',
      value: domain
    }
  })

  #.. rest of challenge passing code
end

Once we've authorized all the subdomains we want to include in the certificate, we can modify our CSR to use the SAN extension (warning: not the prettiest or most readable code you'll ever see):

alt_names = domains.map { |domain| "DNS:#{domain}" }.join(', ')

extension = OpenSSL::X509::ExtensionFactory.new.create_extension('subjectAltName', alt_names, false)
csr.add_attribute OpenSSL::X509::Attribute.new(
  'extReq',
  OpenSSL::ASN1::Set.new(
    [OpenSSL::ASN1::Sequence.new([extension])]
  )
)

That's all you need to get certificates to cover multiple host names, you can find the full code of the example in multiple_subdomains.rb.

*If you're running a site that, say, assigns thousands of subdomains to end users, you may be out of luck since "you can [only] issue certificates containing up to 2,000 unique subdomains per week" (source). The only current work-around is to get your domain added to Public Suffix list - which LE treats as a special case. Additionally, wildcard certificates will launch in January 2018.


Appendix 5: Key size

Broadly-speaking key size means how hard a key is to crack. Longer keys offer more security, but their bigger size leads to a somewhat slower TLS handshake.

SSL handshake speed at different key sizes

We don't have a very broad choice when it comes to choosing key size. 2048 bits has effectively been an enforced minimum since the beginning of 2014; 4096 bits is the upper bound. 4096 bits is favored by some, but is far from the standard right now. It's anticipated that 2048-bit keys will be considered secure until about 2030.

2048 is the default key size for cerbot. But you will need a 4096 bit key to score perfectly on the Key SSL Labs' test, and there are lively discussions advocating the LE default be raised to 4096 or 3072. CertSimple did an awesome, detailed rundown of the benefits of different key sizes, and basically concluded "it depends".

We will need a key size of 4096 bits to get a perfect SSL Labs score. Not all cloud providers support key sizes above 2048 bits though, AWS CloudFront being a notable example. If you want or need to use a 2048-bit key, you can specify the key length like so:

domain_key = OpenSSL::PKey::RSA.new(2048)

ECDSA keys

If you really care about picking a good key, you might not want to use RSA at all. ECDSA (Elliptic Curve Digital Signature Algorithm) which gives a much better size vs. security trade-off. A 384 bit ECDSA is considered equivalent to a 7680 bit RSA key, and will also give a perfect SSL Labs score. More importantly, a number recently discovered SSL vulnerabilities (DROWN, Logjam, FREAK) target RSA-specific vulnerabilities which are not present in ECDSA certificates.

We'll have to a bit more work to create an ECDSA CSR (see this blog post I wrote for a more detailed explanation):

# monkey patch to fix https://redmine.ruby-lang.org/issues/5600
OpenSSL::PKey::EC.send(:alias_method, :private?, :private_key?)

domain_key = OpenSSL::PKey::EC.new('secp384r1').generate_key
IO.write('domain.key', domain_key.to_pem)

csr = OpenSSL::X509::Request.new
csr.subject = OpenSSL::X509::Name.new(['CN', 'le.alexpeattie.com'])
csr.public_key = OpenSSL::PKey::EC.new(domain_key)
csr.sign domain_key, OpenSSL::Digest::SHA256.new

ECDSA is pretty well supported: Windows Vista and up, OS X 10.9, Android 3 and iOS 7*

*Source: CertSimple: What web developers should know about SSL but probably don't


Appendix 6: IDN support

Since October 2016 Let's Encrypt has supported Internationalized Domain Names (IDNs). When providing an IDN as the identifier's value in our new-authz request, and when setting the subject of the CSR, we need to use the Punycode representation of the IDN. For example, müller.de would become xn--mller-kva.de.

You can do the conversion with an online service like Punycoder or with a gem like SimpleIDN:

require 'simpleidn'

domain = SimpleIDN.to_unicode('müller.de')
=> 'xn--mller-kva.de'

Appendix 7: Using EC client keys

As well as support ECDSA-based certificates (see above), since 2016 LetsEncrypt has supported ECDSA for client (A.K.A account) keys. We'll have to make a few non-trivial modifications to our client to get EC client keys working though.

First, we'll need an EC keypair:

openssl ecparam -genkey -name secp256k1 -noout -out ec-private.pem
openssl ec -in ec-private.pem -pubout -out ec-public.pem

(Alternatively you can grab an Boulder's test EC key here

Then, we'll need to change our client_key method to load our EC private key.

OpenSSL::PKey::EC.new IO.read(client_key_path)

Simple enough so far, unfortunately, things begin to get a bit complicated. For starters, we previously only had to worry about a single signing algorithm: RSA + SHA256 (you might remember we refer to it as 'RS256' in our header method). We'll always use the SHA256 digest algorithm, regardless of the length of our RSA key.

With EC keys though, we'll use a different hashing algorithm depending on the curve used/key length (different curves = different key lengths):

Algorithm Curve name Key length (bits) Hashing algorithm
ES256 P-256 256 SHA-256
ES384 P-384 384 SHA-384
ES512 P-521 521 SHA-512

Eagle-eyed readers might spot something odd about the ES512: we have a key 521 bits long, but the associated digest size is 512 bits. It's not a typo, and it does make things a bit more awkward; we can't assume that key size = digest size. We can get the key's bit length/curve using client_key.group.degree, so let's write a method to get the associated digest size:

def digest_size
  { 256: 256, 384: 384, 521: 512 }[client_key.group.degree]
end

With this in place we can modify our hash_algo method to dynamically fetch the correct Digest class to match up with our EC key:

def hash_algo
  OpenSSL::Digest.const_get("SHA#{digest_size}").new
end

Next, we need to modify our header. We'll ditching be ditching our "e" and "n" keys (they're specific to RSA keys), and we'll need to add a "crv" key to indicate the EC key's curve name. As we can see from the table above, for the curves we're concerned with, it's just the key's bit length prefixed with "P-". Lastly, alg is now "ES" + digest_size, and kty (key type) is "EC":

@header ||= {
  alg: "ES#{ digest_size }",
  jwk: {
    crv: "P-#{ client_key.group.degree }",
    kty: 'EC'
  }
}

Before we go any further, let's add a helper method which splits a string into pieces of a certain length (this will come in handy later):

def split_into_pieces(str, opts = {})
  str.chars.each_slice(opts[:piece_size]).map(&:join)
end

# example:
split_into_pieces("abcdef", piece_size: 2)
# => ['ab', 'cd', 'ef']

Next, we have to add the public part of the key into the header. Running client_key.public_key.inspect we see something like:

"#<OpenSSL::PKey::EC::Point:0x007fad4209d728 #...

The public part of an EC key is called a "public key curve point", and it's literally a point in 2-dimensional space. We need to provide the x and y coordinates of this point, again this is a little bit tricky. First, let's convert our public key to a hexidecimal string:

pub_key_hex = client_key.public_key.to_bn.to_s(16)
# => "04170BD2669BB4EA2DDFAD293F9B3F47703F671139F8C1FDE643ECC3B46DB519AA4BCAD1FB47566BC9C0730D5F6EE9C5FDA8D2DCF419F90C0BA6CFB669D80B80F9"

Andreas M. Antonopoulos, gives a good explanation of what we're looking at:

As we saw previously, the public key is a point on the elliptic curve consisting of a pair of coordinates (x,y). It is usually presented with the prefix 04 followed by two 256-bit numbers, one for the x coordinate of the point, the other for the y coordinate. The prefix 04 is used to distinguish uncompressed public keys from compressed public keys that begin with a 02 or a 03.

We don't really care about the 04 prefix - once we've got rid of that, we'll need to split our long hexidecimal sequence in half, to extract the x and y values.

First, let's use our split_into_pieces to break it up into the individual octets:

pub_key_octets = split_into_pieces(pub_key_hex, piece_size: 2)

Next we'll drop our first octet (04), and split our sequence in half:

pub_key_octets.shift # drop the first octet (which just indicates key is uncompressed)
x_octets, y_octets = pub_key_octets.each_slice(pub_key_octets.size / 2).to_a

Lastly, we'll convert our hex values to binary data using the pack method (see To Hex and Back (With Ruby) for a detailed explanation):

x = x_octets.map(&:hex).pack('c*')
y = y_octets.map(&:hex).pack('c*')

We can shorten our code a bit by converting to binary first, and reusing our split_into_pieces method:

coords_binary_data = pub_key_octets.map(&:hex).pack('c*')
x, y = split_into_pieces(coords_binary_data, piece_size: coords_binary_data.size / 2)

Lastly, we'll need to Base64 encode our x and y values before sending them over the wire:

jwk: {
  crv: "P-#{ client_key.group.degree }",
  x: base64_le(x),
  kty: 'EC',
  y: base64_le(y)
}

To recap, our header method now looks like this:

def header
  @header ||= begin
    pub_key_hex = client_key.public_key.to_bn.to_s(16)
    pub_key_octets = split_into_pieces(pub_key_hex, piece_size: 2)

    pub_key_octets.shift # drop the first octet (which just indicates key is uncompressed)
    coords_binary_data = pub_key_octets.map(&:hex).pack('c*')
    x, y = split_into_pieces(coords_binary_data, piece_size: coords_binary_data.length / 2)

    {
      alg: "ES#{ digest_size }",
      jwk: {
        crv: "P-#{ client_key.group.degree }",
        x: base64_le(x),
        kty: 'EC',
        y: base64_le(y)
      }
    }
  end
end

Worn out yet 😅? There's one last step: we have to update how our signature is generated. When we sign a value using RSA, the signature is a single value σ, which is really just one long integer (see Digital signature on Wikipedia). But DSA (which we use with EC keys) returns a pair of integers, typically denoted r and s (see Wikipedia's DSA article), so we'll need to make a few modifications to allow for this. First, OpenSSL::PKey::EC equivalent method to #sign is #dsa_sign_asn1:

signature = client_key.dsa_sign_asn1 hash_algo.digest([request[:protected], request[:payload]].join('.'))

Then we need to extract the value of (r, s) as binary strings. The signature is ASN.1 encoded, so we'll first decode it and convert it to an array (of two elements, i.e. r and s):

decoded_signature = OpenSSL::ASN1.decode(signature).to_a

Then we'll map the values of r and s as binary strings:

r, s = decoded_signature.map { |v| v.value.to_s(2) }

Finally, we set the "signature" field in our JSON request to r and s concatenated together, and Base64 encoded:

request[:signature]  = base64_le(r + s)

All the changes we needed to make are collected below (also see ec_client.rb):

def client_key
  @client_key ||= begin
    client_key_path = File.expand_path('~/Desktop/ec-private.pem')
    OpenSSL::PKey::EC.new IO.read(client_key_path)
  end
end

def split_into_pieces(str, opts = {})
  str.chars.each_slice(opts[:piece_size]).map(&:join)
end

def header
  @header ||= begin
    combined_coordinates = client_key.public_key.to_bn.to_s(16)
    coord_octets = split_into_pieces(combined_coordinates, piece_size: 2)

    coord_octets.shift # drop the first octet (which just indicates key is uncompressed)
    coords_bin = coord_octets.map(&:hex).pack('c*')
    x, y = split_into_pieces(coords_bin, piece_size: coords_bin.length / 2)

    {
      alg: "ES#{ client_key.group.degree }",
      jwk: {
        crv: "P-#{ client_key.group.degree }",
        x: base64_le(x),
        kty: 'EC',
        y: base64_le(y)
      }
    }
  end
end

def hash_algo
  bit_size = client_key.group.degree
  bit_size = 512 if bit_size == 521

  OpenSSL::Digest.const_get("SHA#{bit_size}").new
end

def signed_request(url, payload)
  request = {
    payload: base64_le(payload),
    header: header,
    protected: base64_le(header.merge(nonce: nonce))
  }
  signature = client_key.dsa_sign_asn1 hash_algo.digest([request[:protected], request[:payload]].join('.'))
  decoded_signature = OpenSSL::ASN1.decode(signature).to_a

  r, s = decoded_signature.map { |v| v.value.to_s(2) }

  request[:signature]  = base64_le(r + s)
  HTTParty.post(url, body: JSON.dump(request))
end

Appendix 8: Certificate expiry and revocation

A fun factoid: Let's Encrypt certificates are technically only valid of 89 days and 23 hours, not for a whole 90 days. This is because LE backdates certificates by 1 hour to ensure the certificates can be validated immediately by clients whose clocks might be slightly out. Therefore a certificate issued on August 1st 12:34 will expire October 30th 11:34.

The validity period for Lets Encrypt certificates are relatively short. Per the CA/Browser Forum Baseline Requirements, Section 6.3.2:

Subscriber Certificates issued after 1 March 2018 MUST have a Validity Period no greater than 825 days. Subscriber Certificates issued after 1 July 2016 but prior to 1 March 2018 MUST have a Validity Period no greater than 39 months.

Accordingly, most commercial providers offer certificates with 1, 2 or 3 year validity periods (see GlobalSign's article on Maximum Certificate Validity). LE states the primary reasons for the shorter lifetime are:

  • Shorter lifetimes decrease the compromise window in situations like Heartbleed
  • Offering free certificates with a shorter lifetime provides encouragement for operators to automate issuance.
  • Let's Encrypt's total capacity is bound by its OCSP signing capacity, and LE is required to sign OCSP responses for each certificate until it expires. Shorter expiry period means less overhead for certificates that were issued and then discarded, which in turn means higher total issuance capacity.

(Source: Pros and cons of 90-day certificate lifetimes)

Let's Encrypt will send email reminders to the address(es) provided in the contacts field of your new-reg payload, at the following times:

  • 20 days before the date of expiry
  • 10 days before the date of expiry
  • 1 day before the expiry.

Additionally, various tools exist to monitor your certificates and alert you about upcoming expiries, including hosted services like LetsMonitor and Keychest or standalone applications like certinel. Dan Cvrcek posted a fairly extensive list on the LE forums.

Requesting revocation of a certificate for example.com

If the private keys of our certificates get compromised, we need to disable certificates before they expire. In these cases we can explicitly revoke certificates; as the diagram above shows, to do this we make a signed request to LE which includes the certificate to be revoked. LE then propagates the revocation to certificate revocation lists and OCSP responders, which in turns should ensure browsers won't accept requests signed by the revoked certificate (especially if OCSP stapling is enabled, see Appendix 1).

There are a number of different ways to perform a revocation, depending on which keys you have access to.

Scenario 1: You have access to the private key for the certificate

Revocation requests are different from other ACME request in that they can be signed either with an account key pair or the key pair in the certificate. If we still have access to this key, we can simply load it in as our client key:

client_key_path = File.expand_path('~/Desktop/domain.key')
OpenSSL::PKey::RSA.new IO.read(client_key_path)

For our payload, we'll need the certificate in question. Since we have our private key locally, we'll assume the certificate is locally available too (though see Scenario 2 for alternative approaches):

cert_path = File.expand_path('~/Desktop/chained.pem')
certificate = OpenSSL::X509::Certificate.new IO.read(cert_path)

To revoke our certificate, we'll need to send a Base64 encoded version of the certificate in DER format:

new_registration = signed_request(endpoints['new-reg'], {
  resource: 'revoke-cert',
  certificate: base64_le(certificate.to_der)
})

Scenario 2: You don't have access to the private key for the certificate, but you still have access to the client key for the account which issued the certificate

If the authorizations are still valid for the certificate's domain (i.e. the certificate is less that 30 days old, as of April 2017), you can revoke the certificate as above, but using your existing account key:

client_key_path = File.expand_path('~/.ssh/id_rsa')

# ...

new_registration = signed_request(endpoints['new-reg'], {
  resource: 'revoke-cert',
  certificate: base64_le(certificate.to_der)
})

Note that you still need to provide the certificate in DER format, even if you're not providing the certificate's corresponding private key. You can dynamically fetch the certificate like so:

uri, certificate = URI.parse("https://example.com"), nil
http = Net::HTTP.new(uri.host, uri.port)
http.use_ssl = true
http.verify_mode = OpenSSL::SSL::VERIFY_PEER
http.start { |h| certificate = h.peer_cert }

Note that if it's been 30 days since you issued the certificate, the account key won't help you, and you're in an equivalent position to Scenario 3.

Scenario 3: You don't have access to the client key for the account which issued the certificate, or the private key for the domain, but you still control the certificate's domain(s)

Per LetsEncrypt's article on revocation:

If someone issued a certificate after compromising your host or your DNS, you’ll want to revoke that certificate once you regain control. In order to revoke the certificate, Let’s Encrypt will need to ensure that you control the domain names in that certificate (otherwise people could revoke each other’s certificates without permission)! To validate this control, Let’s Encrypt uses the same methods it uses to validate control for issuance: you can put a value in a DNS TXT record, put a file on an HTTP server, or offer a special TLS certificate.

In other words, you'll need to create a new account, pass the challenges for the domain(s) of the compromised certificate (see Section 4), then revoke the certificate as in Scenario 2, but using the account key for your newly created and authorized account.

Further reading

TLS/SSL in general

Let's Encrypt


Image credits


Author

Alex Peattie / alexpeattie.com / @alexpeattie


Changelog

Version 1.2 - Aug 7 2017

  • Add Appendix 7 explaining how to use EC client keys
  • Add Appendix 8 about certificate expiry and revocation
  • Add note about terms of service URL now being available via the directory
  • Update Appendix 4 with up-to-date rate limits, note about forthcoming wildcard certs

Version 1.1 - Nov 19 2016

  • Use the directory and response headers, rather than hardcoding URIs (closes #1)
  • Add Appendix 6 about newly supported Internationalized Domain Names
  • Change reference to official Let's Encrypt client → certbot
  • Specify a TTL for DNS challenge record
  • Add note about certificate and authorization validity periods
  • Consistently prefer single quotes in all Ruby code
  • Remove example domains for the various certificate types
  • Added a couple more tools to the Testing section
  • Add Changelog & Author section
  • Harden example nginx config with additional security headers

Version 1.0 - Mar 29 2016

  • Initial release

🔝 Back to top