Rewrite ACME issuer for v2, and make validation process asynchronous

[munnerz]

What this PR does / why we need it:

Adds support for the ACME v2 protocol in favour of v1

Which issue this PR fixes: fixes #273

Special notes for your reviewer:

This PR is not ready yet. We need to validate and ensure boulder runs with the v2 protocol enabled during our e2e tests. In future we can switch to something like pebble, a lightweight alternative to boulder.

I've updated the PR to use Pebble instead of Boulder during e2e tests

I have also copied across the upcoming golang.org/x/crypto/acme/v2 package from the gerrit codereview so we can get ahead of the curve on testing this: https://go-review.googlesource.com/c/crypto/+/86635

I have also not tested whether it is possible to obtain wildcard certificates with this yet
I have managed to obtain a wildcard certificate with this PR!

Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number:
            fa:f5:cc:92:f2:02:15:58:37:05:02:9f:bf:53:9f:9c:d4:0c
    Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN=Fake LE Intermediate X1
        Validity
            Not Before: Mar 23 15:56:39 2018 GMT
            Not After : Jun 21 15:56:39 2018 GMT
        Subject: CN=*.james-gcp.redacted.net
        Subject Public Key Info:

Release note:

Remove support for ACMEv1 in favour of ACMEv2

[munnerz]

Still TODO: As part of making the authorization process asynchronous we need to make the DNS and HTTP01 challenge solver idempotent (so the names of resources it creates must be looked up instead of stored in memory)

[munnerz]

/retest

[munnerz]

Currently failing due to the aforementioned TODO:

Still TODO: As part of making the authorization process asynchronous we need to make the DNS and HTTP01 challenge solver idempotent (so the names of resources it creates must be looked up instead of stored in memory)

Also waiting for this to merge in Pebble: letsencrypt/pebble#94

[cpu]

Also waiting for this to merge in Pebble: letsencrypt/pebble#94

@munnerz That's on my plate for today 👍

[markns]

Hey @munnerz, I know you've marked this PR as not ready yet, but I was super-keen to try it out, so went ahead and compiled it myself. It didn't work for me, because the https://acme-staging-v02.api.letsencrypt.org/acme/new-acct endpoint was returning HTTP 200, but with an empty body which caused the decoding at third_party/crypto/acme/acme.go:556 to fail.

I0218 16:53:19.071128   82052 acme.go:105] getting private key (letsencrypt-staging->tls.key) for acme issuer cert-manager/letsencrypt-staging-cluster-issuer-v2
I0218 16:53:19.909230   82052 setup.go:59] letsencrypt-staging-cluster-issuer-v2: Failed to verify ACME account: acme: invalid response: EOF
I0218 16:53:19.909303   82052 helpers.go:112] Setting lastTransitionTime for ClusterIssuer "letsencrypt-staging-cluster-issuer-v2" condition "Ready" to 2018-02-18 16:53:19.909295374 +0100 CET m=+15.560314114
I0218 16:53:19.909332   82052 sync.go:40] Error initializing issuer: acme: invalid response: EOF

Feel free to ignore if you're already aware of this, but I thought it was a bit weird so thought I'd let you know.

Here's my cluster issuer yaml:

kind: ClusterIssuer
metadata:
  name: letsencrypt-staging-cluster-issuer-v2
spec:
  acme:
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    email: cert@gridarrow.io
    privateKeySecretRef:
      name: letsencrypt-staging
    http01: {}

[munnerz]

Thanks for trying it out! I'm aware of the issues around the new account registration and have been making some changes to Pebble to accommodate them (Pebble did not support parts of the user registration flow in the past). I'll be updating this PR in the coming week so it's passing e2e tests. In the meantime, I think reverting the (manual) changes I've made to the acme v2 library should make it work properly with the staging endpoint

…

On Sun, 18 Feb 2018 at 16:08, Mark Nuttall-Smith ***@***.***> wrote: Hey @munnerz <https://github.com/munnerz>, I know you've marked this PR as not ready yet, but I was super-keen to try it out, so went ahead and compiled it myself. It didn't work for me, because the https://acme-staging-v02.api.letsencrypt.org/acme/new-acct endpoint was returning HTTP 200, but with an empty body which caused the decoding at third_party/crypto/acme/acme.go:556 to fail. I0218 16:53:19.071128 82052 acme.go:105] getting private key (letsencrypt-staging->tls.key) for acme issuer cert-manager/letsencrypt-staging-cluster-issuer-v2 I0218 16:53:19.909230 82052 setup.go:59] letsencrypt-staging-cluster-issuer-v2: Failed to verify ACME account: acme: invalid response: EOF I0218 16:53:19.909303 82052 helpers.go:112] Setting lastTransitionTime for ClusterIssuer "letsencrypt-staging-cluster-issuer-v2" condition "Ready" to 2018-02-18 16:53:19.909295374 +0100 CET m=+15.560314114 I0218 16:53:19.909332 82052 sync.go:40] Error initializing issuer: acme: invalid response: EOF Feel free to ignore if you're already aware of this, but I thought it was a bit weird so thought I'd let you know. Here's my cluster issuer yaml: kind: ClusterIssuer metadata: name: letsencrypt-staging-cluster-issuer-v2 spec: acme: server: https://acme-staging-v02.api.letsencrypt.org/directory email: ***@***.*** privateKeySecretRef: name: letsencrypt-staging http01: {} — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#309 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AAMbP-2OEqwUQR9nAzr8jNTh34uEewzoks5tWEsJgaJpZM4SAsaf> .

[munnerz]

Follow up (just remembered there's another significant part of this PR that will make it not work):

Still TODO: As part of making the authorization process asynchronous we need to make the DNS and HTTP01 challenge solver idempotent (so the names of resources it creates must be looked up instead of stored in memory).

Until this is done, when using HTTP01 you'll see hundreds of acmesolver pods created so please do not use this on a production cluster 😄

[jetstack-ci-bot]

@munnerz PR needs rebase

[munnerz]

/retest

[munnerz]

/retest

[munnerz]

/retest

[munnerz]

/retest

[munnerz]

Re running tests with cloudflare dns01 provider tests enabled

/test e2e

[jetstack-bot]

@munnerz: The following test failed, say /retest to rerun them all:

Test name	Commit	Details	Rerun command
cert-manager-e2e-v1-10	acd927d	link	/test e2e v1.10

Full PR test history. Your PR dashboard. Please help us cut down on flakes by linking to an open issue when you hit one in your PR.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

[munnerz]

/skip

[kragniz]

/lgtm

[jetstack-bot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kragniz

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [kragniz]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment