Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support specifying ACME HTTP01 ingress name as an annotation on Certificates #1666

Open
munnerz opened this issue May 10, 2019 · 4 comments

Comments

@munnerz
Copy link
Member

commented May 10, 2019

Since the changes to migrate to the new 'solvers' configuration format (#1450), it's cumbersome for users of ingress controllers such as GCE, which require specifying the name of an Ingress resource to edit in order to solve ACME HTTP01 challenges, to create new Certificate resources for newly created domains.

Previously, users could:

  • Create an Issuer resource with the HTTP01 provider enabled
  • Create a certificate and set certificate.spec.acme.config.http01.ingressName="name-of-ingress-to-edit" in order to obtain the certificate by adding the acme challenge path to the named ingress resource

Illustrated in YAML:

apiVersion: certmanager.k8s.io/v1alpha1
kind: Issuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    email: user@example.com
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: example-issuer-account-key
    http01: {}
---
apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
  name: example-com
spec:
  secretName: example-com-tls
  issuerRef:
    name: letsencrypt-staging
  dnsNames:
  - example.com
  - www.example.com
  acme:
    config:
    - http01:
        ingressName: ingress-to-edit-to-solve-challenges
      domains:
      - example.com
      - www.example.com

Because we are deprecating the certificate.spec.acme field altogether, to make certificate resources 'portable' between Issuer types, an ingress-gce user would need to add a new entry to their issuer.spec.acme.solvers configuration for each ingress they want to use to solve HTTP01 challenges:

To illustrate, with the new format the user will create an Issuer resource like so:

apiVersion: certmanager.k8s.io/v1alpha1
kind: Issuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    email: user@example.com
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: example-issuer-account-key
    solvers:
    - http01:
        ingress:
          name: ingress-to-edit-to-solve-challenges

and then a certificate resource:

apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
  name: example-com
spec:
  secretName: example-com-tls
  dnsNames:
  - example.com
  - www.example.com
  issuerRef:
    name: letsencrypt-staging
    kind: Issuer

This is a pretty annoying change in the UX for ingress-gce users, or any other ingress controller that maps a single "frontend IP" to each Ingress resource.

We need some way to make this experience better again, I see a few options:

  1. Add a field on the Certificate resource: this defeats the point of removing the certificate.spec.acme field altogether, so isn't really an option at all 😬

  2. Allow an annotation on the Certificate resource that specifies the name of an ingress resource to edit: these seems like a nice answer, and works well in future if other issuer types have a similar requirement. It'd be a simple string value, so we wouldn't be requiring putting complex JSON structures into string based fields:

apiVersion: certmanager.k8s.io/v1alpha1
kind: Certificate
metadata:
  name: example-com
  annotations:
    http01.acme.certmanager.k8s.io/ingress-to-edit: "ingress-to-edit-to-solve-challenges"
spec:
  secretName: example-com-tls
  dnsNames:
  - example.com
  - www.example.com
  issuerRef:
    name: letsencrypt-staging
    kind: Issuer

We'd probably also want to gate this as an option on the HTTP01 solver too:

apiVersion: certmanager.k8s.io/v1alpha1
kind: Issuer
metadata:
  name: letsencrypt-staging
spec:
  acme:
    email: user@example.com
    server: https://acme-staging-v02.api.letsencrypt.org/directory
    privateKeySecretRef:
      name: example-issuer-account-key
    solvers:
    - http01:
        ingress:
          allowManuallySpecifiedIngress: true

I'm keen to hear anyone else's thoughts on this matter. Whilst we continue to support both fields, ingress-gce users who are negatively affected by this can continue to use the old format. But we should definitely discuss and come up with a take on this ASAP 😄

/area acme
/area acme/http01
/area api
/kind feature
/priority important-soon
/milestone v0.9

@Quentin-M

This comment has been minimized.

Copy link

commented May 23, 2019

Hi @munnerz,

Just adding here that the issue does not only affect GCE users but any kubernetes administrators with large multi-tenants clusters that rely on separation of concerns. The administrators manage the ClusterIssuers (e.g. Route53 credentials, LE mail/private keys), while end-users create their own various ingress controllers for their different use cases & set ingress classes on them (e.g. private vs. public, nginx vs haproxy vs traefik, custom ports, enforced SAML auth vs no auth, etc).

We have no control over all the different ingress classes our end users set up, nor can we predict/match easily whether they will be suited to answer LE requests. Thus, the ability to specify an ingress class is critical.

@munnerz

This comment has been minimized.

Copy link
Member Author

commented May 23, 2019

Thanks for the feedback @Quentin-M - this is exactly why we have introduced it in this manner, so we can hear from users as to what they need 😄

I do wonder, does the proposed solution work for you? How would you most like to see it from an ops perspective? (e.g. from the perspective of someone managing ClusterIssuer resources)?

A few open questions:

  1. Do you have a requirement to be able to use a custom label key for this? e.g. allowing specifying what label the ingress class/name should be extracted from when solving?
  2. Do you need control over whether allowing Certificate's to override this should be allowed?
  3. Does an unstructured label/annotation work well for you?
  4. Would you prefer a label or an annotation? (especially relevant wrt. pt 2, as technically you could actually use the ingress-class label to configure this)
  5. Conceptually, do you see configuring which solver to use as an ops concern, or a developer concern? If you see it as a developer concern, would making this solver configuration a distinct resource type help? e.g. an ACMESolver resource that your developers could configure and could somehow be mapped to an Issuer/ClusterIssuer? If so, any more ideas on how this might be structured?

Long term, it is essential that we remove issuer specific fields from the Certificate resource, else we'll never be able to support out-of-tree Issuer types that require configuration beyond the absolute bare-minimum, especially given the future plans for the project (e.g. allowing cert-manager to not actually handle/manage private keys)

@Quentin-M

This comment has been minimized.

Copy link

commented May 24, 2019

Thank you so much for reaching back to me.

We mostly use Kubernetes as a self-service platform for infrastructure, we run multi-tenant clusters, with each tenant managing their manifests fully. Each cluster are provided by the Ops team and have a rather wide selection of public/private domains that our tenants can claim subdomains from, and then typically have a CA ClusterIssuer (to issue certificates to fully private subdomains), as well as a LE ClusterIssuer, with both HTTP01 and DNS01 enabled & configured with the right set of keys/credentials (only known by the Ops team).

As each tenant manage their manifests, within their own tenancy area (e.g. namespace, pools), via a templating system the Ops team provide, they create Ingress controllers, Ingress resources & Certificate resources as they need them, by themselves. Some of those controllers may be pretty custom, based on the specific needs of their applications - they may be exposed on public/private ELBs/ALBs/NLBs and all have different ingress classes to differentiate them & have them target specific services. In some cases, our tenants sometimes need to rely on HTTP01 to get their certificates, while most of the case they can use DNS01.

As far as I see the current situation, I may be missing a use-case or two.. The deprecation of the Ingress class annotation in the Ingress resource is rather fine, as that our tenants can still specify a ClusterIssuer - that only may have either DNS01 or HTTP01 enabled, and given that the cert-manager code currently will setup its own Ingress resource using the Ingress class specified in the targeted Ingress resource. Now, the fact that the Ingress class is being removed entirely from the Certificate object, there is no way now for our tenants to specify which Ingress class is suitable for LE servers to come verify ownership of the domain (e.g. the one that is used by the Ingress controller, that holds the LB to which the domain points to). We would have to let our tenants create new Issuers that target their Ingress class - which may have to contain DNS credentials / LE key (which our tenants can't actually know), or create ClusterIssuers on our side for each of the Ingress class they have (not scalable).

2/ Our use-case does not currently require any extra authorization.
3/ Completely understand your desire to generalize to make off-tree providers, in which case - an extra annotation on the Certificate object can definitely serve as "parameters" for the said provider as you mentioned - restoring the ability for tenants to specify their Ingress class.
5/ Figured I mostly answered this in my prose above - our tenants need to be able to pick DNS01/HTTP01/CA by themselves based on their use, but should not care about the implementation details or credentials required to make this happen.

I hope this makes sense? All in all, not complicated but hard to put in words. Otherwise happy to discuss further! Thank you for your consideration.

@cheukwing

This comment has been minimized.

Copy link
Member

commented Jun 7, 2019

/assign
/lifecycle active

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
You can’t perform that action at this time.