Use cert-manager with Let's Encrypt to enable TLS for gcsweb

[ixdy]

The first commit is basically following the cert-manager setup guide and installing ClusterIssuers for the staging and prod Let's Encrypt endpoints.

The second commit then uses this setup to obtain a certificate for gcsweb and install it in the new Ingress.

I haven't deployed any of this to the real cluster, but I've tested a version of it on my own deployment and cluster: http://gcsweb.ixdyland.net/gcs/kubernetes-jenkins/

Fixes #38
(and paves the way to using this for spartakus and the k8s.io redirectors)

/assign @thockin @hh
cc @munnerz

[ixdy]

To bootstrap cert-manager components (per the cert-manager docs):

cd cert-manager/
kubectl create clusterrolebinding cluster-admin-binding --clusterrole=cluster-admin --user=$(gcloud config get-value core/account)
kubectl apply -f cert-manager.yaml --validate=false
kubectl apply -f letsencrypt-staging.yaml
kubectl apply -f letsencrypt-prod.yaml

Then create the Certificate and update the Ingress per normal:

cd gcsweb.k8s.io/
kubectl apply -f certificate.yaml
kubectl apply -f ingress.yaml

Subsequent migrations (spartakus, redirector, etc) only need the latter steps (creating a Certificate and updating the Ingress).

[munnerz]

Awesome to see, and this lgtm 😄

One thing to note - we've identified an issue with certificate rotation in the webhook component, which could leave the webhook solving with old certificates when renewal times comes along. More details here: cert-manager/cert-manager#1340

The certificate duration here is configured to be 1y for the serving cert, and 5y for the CA used. Just a heads up to upgrade to v0.7 once that is released some time in the next year, or otherwise {somehow remember} to restart your webhook pod in ~8mths time (we renew 4m before expiry).

If you want to keep an eye on the expiry state of your certificates, we also expose expiry time prometheus metrics. You might want to set up prometheus/alertmanager/stackdriver or the like, to automatically alert you if a certificate is nearing expiry 😄

[munnerz]

This all looks good to me (doing a visual review, I've not actually pulled and deployed these manifests) barring the one comment about ordering when it comes to applying these changes 😄

[munnerz]

nit: may want to pin latest to release-0.6?

[ixdy]

done

[munnerz]

nit: possibly add a note mentioning that --validate=false can go away with k8s 1.13+

[ixdy]

done

[munnerz]

Due to an issue with the way cert-manager obtains certificates from LE (we generate and store a private key before the corresponding certificate has been obtained from Let's Encrypt) (more info: cert-manager/cert-manager#1343) - if you add this field here when no Secret resource already exists, the ingress-gce ingress controller will fail to actually update the GCP API with the load balancer configuration.

Because of this, you must instead create the Certificate resource and obtain the certificate before updating the ingress resource to reference that secret.

We'll be building a workaround into v0.7 which is due in ~1.5wks (most likely, pre-generating a self-signed certificate which can be used in the interim whilst CM is obtaining a publicly trusted certificate).

What this means, is that in order to deploy this the first time, you'll need to apply these changes after creating all the other resources in this PR 😄

[ixdy]

I think I discovered this, but I appreciate the clarification. I added a note to our README as well.

[munnerz]

I've got a fix in progress that will be landing in v0.7 here: cert-manager/cert-manager#1392 😄

[hh]

During our meeting today approval for this PR was requested.
/cc @munnerz @thockin

[spiffxp]

/approve
/hold
I don't have enough context to full review, @munnerz can you take a look?

For followup, I feel like we should have a cert-manager/OWNERS file so this doesn't have to hit root OWNERS

[ixdy]

good point re: OWNERS, though I'm not sure who should be in it. :)

[munnerz]

Taking another look, this all looks 👍 to me (although sods law states ...)

I'll add my lgtm, but defer a /hold cancel to someone in a better position to manage the roll out of this change 😄

good point re: OWNERS, though I'm not sure who should be in it. :)

I'd be happy to put my name forward to be pinged for reviews etc. on this 😄

[munnerz]

/lgtm

[ixdy]

added OWNERS with me and @munnerz. any other volunteers? :)

[thockin]

I am OK with this, but defer actual review to you folks. I will approve but not remove he hold. At your leisure.

/approve

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: spiffxp, thockin

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [spiffxp]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[ixdy]

this needs one more /lgtm (for the OWNERS file).

[munnerz]

/lgtm

[ixdy]

/hold cancel