Join GitHub today
Certificate issuing doesn't work with rewrite-target #286
Is this a BUG REPORT or FEATURE REQUEST?:
I tried issuing a certificate for a host. I'm using the nginx ingress controller and its
apiVersion: extensions/v1beta1 kind: Ingress metadata: annotations: kubernetes.io/ingress.class: nginx kubernetes.io/tls-acme: "true" nginx.ingress.kubernetes.io/rewrite-target: / name: test spec: rules: - host: example.org http: paths: - backend: serviceName: test-service servicePort: 80 path: /test tls: - hosts: - example.org secretName: example-tls
I think the cause is that cert-manager adds an additional rule to the ingress for the validation:
- backend: serviceName: cm-main-tls-uwson servicePort: 8089 path: /.well-known/acme-challenge/XXX
It then tries to parse the challenge from the path (
Maybe it is possible to change the path that is added by cert-manager to just
What you expected to happen:
A certificate to be issued.
How to reproduce it (as minimally and precisely as possible):
See ingress definition above.
Hi @turbolent. By default for Ingress-triggered certificates
You can avoid this drama by getting
@munnerz is there or should there be an Ingress annotation that can trigger
In general this is a far more robust way to create certificates, though some abhorrent ingress controller implementations don't correct support multiple Ingress resources for the same domain (a mandatory ability since Ingresses are namespaced). Apparently the commercial
referenced this issue
Jan 28, 2018
Thanks for the explanation and the example @whereisaaron!
Does cert-manager stop adding it's own rule to an ingress for the
I worked around it for now, but I think this will still be useful documentation for somebody else who might run into this.
If you use the
This has the advantage of allowing per-ingress rewrite-targets, which is why this resolves your problem.
@whereisaaron not right now, but we also have the 'setting annotations on ingresses/services to disable auth etc' problem to solve, which is of a similar ilk. Not sure on what our best strategy should be right now, it'd be good to write up a few solutions somewhere so we can model it/mock up some manifests.
referenced this issue
Apr 18, 2018
Not only is the temporary workaround proposed by whereisaaron far from optimal, but the failing behaviour seems to be associated with some kind of memory leak.
One of our domains expired before the ingress was removed, and cert-manager has been trying to renew it for a few days. Here's the pod logs (the message repeats every 15 minutes):
Here's the pod memory usage, from before the domain expired up until today:
The steeper part of the graph is associated with having two domains failing at the same time.
Should I open a separate issue for this (apparent) memory leak?
Regarding the memory leak - I think this is due to the way HTTP challenges were cleaned up in the past. ACMEv2 (i.e. #309) has drastically changed how it works, and so I don't think that bug will exist still.
Are you able to confirm by trying the v0.3.0-alpha.1 release? You'll also need to update the acme URL in all of your Issuer resources after upgrading.
As for the memory leak, we just deployed a working v0.3.0-alpha.1 pod. We'll report back to you in a few hours after we've gathered some data about its memory consumption. Hopefully the leak is gone in the ACMEv2 issuer!