http-01 self check failed for domain #656
Comments
|
The problem solved by itself today. I don't know how. Thanks for cert-manager. It's really a great tool! |
|
I'm running into the same thing. I see in the logs say Super buggy |
|
Sorry for the delay. Glad this resolved itself.
FWIW, this is actually expected. Currently if the self check fails, we
update the status information with the reason (ie self check failed) and
try again later (to allow for propagation)
…On Tue, 19 Jun 2018 at 01:02, Arianit Uka ***@***.***> wrote:
I'm running into the same thing. I see in the logs say writing key...,
but if I look at the certificate, it says its still validating it.
Super buggy
—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
<#656 (comment)>,
or mute the thread
<https://github.com/notifications/unsubscribe-auth/AAMbP50YXrD8skbkWA7LyugiCup5gm9nks5t-D-qgaJpZM4UoJxb>
.
|
|
I am running into this same issue, How long does it take? |
|
It took me two-three days. But now when generate certificates for other subdomains, it takes less than a minute. |
|
it's strange behavior, i have 2 dns names (asd.team1.example.com and asd.another.com), and interesting points are:
And only for asd.another.com i can't obtain cert by cert-manager, but with certbot on this machine everything works fine :( |
|
I encountered the same issue. Any address I choose for my app works, except a single one whose validation is blocked by I'm trying to figure our from logs what could be a reason for this single subdomain to fail self check validation. |
|
Same error as the OP. Here is my |
|
Same error as OP.(2) |
|
updated now from v2.5 With the last the ingress get with the acme collange extended cert-manager still logs: |
|
I could solve it. The hairpin mode of the NLB in front of the cluster didnt work. |
|
I ran into the same problem.. but I tried it again after a while then it succeeded..! |
|
i just have same problem today. only 3 of those validated www.farmersflorals.com, farmersflorals.com, and identity.farmersflorals.com other not. all of those refer to same IP, i could access all of them, but only 3 validated, that's weird. i'm using helm chart version 0.4.1 |
|
Having the same issue. I have two clusters tuned absolutely identically in terms of nginx-ingress and cert-manager and the third one is lagging. all three domains self check failed. I have clusters for prod and staging - now it's QA turn. Nothing works. Logs don't say anything useful |
|
I'm on the GCP using GKE. Removing nginx-ingress and turn on it back helped. Ephemeral external IP seems preserved magically. |
|
same issue here - the pod of the challenge are up running with no logs, and cert manager is failing the self check. I manually deleted the secret for the TLS and it successfully generated the cert. I have tested a pod with the same service account name to create and update a secret and it succeeded so its not an RBAC solution. here's my log: |
|
I ran into the same issue today |
|
I have the same issue too |
|
me too |
|
@munnerz Could this issue be reopened? Seems to be happening to a lot of people, myself included. Kubectl reports "http-01 self check failed" while solver logs claim "Got successfull challenge request, writing key ..." and seem to be stuck in loop. Solver log: Cert-manager log (repeats this again and again): |
|
Hey @xaralis - this issue has been closed as this error message is expected whilst the self check is failing, and otherwise issues like this can become catch-alls for common misconfiguration by users. If you are experiencing issues, please try and put together a reproducible test case and open a new issue with instructions for how it can be reproduced, if you think you've encountered an actual bug in the self checking flow so we can (1) encode that test case into an actual automated test and (2) fix that test 😄 There's a real wide variety of issues that can cause this message to be printed - although your case, with the self check pod clearly receiving requests, does seem odd. That said, the timestamps between the two differ by 12-13 minutes, so it seems like you may be looking at different self check attempts here. We are trying to keep the repositories issue board clean of "support" related issues, and so would prefer if you could post on Slack in order to help debug the problem. Once we've identified that it is in fact a bug, and not simply a misconfiguration, opening an issue will then be the best route so we can track and triage an actual fix 😄 |
|
Mi problem persists only in prod server (https://acme-v02.api.letsencrypt.org/directory)... in stanging server it works good (https://acme-staging-v02.api.letsencrypt.org/directory)... How can I obtain more information ? |
|
@munnerz OK, I'll try the slack tomorrow if this doesn't fix itself. What is the reasonable amount of time to wait? |
|
Just in case it's helpful, I had a situation where the well-known path was set for both my main ingress and the one created by cert-manager. I think what happened is that the path set for my main ingress was the chosen one, and was automatically redirecting to SSL and failing because the certificate wasn't found. Removing the main ingress completely and recreating seemed to resolve the issue for me. |
|
@saward what do you mean the main ingress for the well-known path? is this a bug? did you manually set it up before cert manager? for me once the secret was deleted it was created immediately on the already running challenge pod |
|
It might be a bug. I'll explain a bit clearer, but I am not great with the terminology and concepts yet so I may not describe things well. I have my own ingress I've created with a few rules. cert-manager appears to create its own ingress for the domain with a rule matching a specific path, the 'well-known' path, used by let's encrypt to verify ownership. While cert-manager was trying and failing with the self check, I checked all ingresses (kubectl describe ing). I noticed that a 'well-known' path rule existed for both the ingress I'd created as well as the one created by cert manager, even though I had never added such a rule to my own ingress. I can only assume that cert manager created the rule under both ingresses, but why and under what conditions, I'm not sure. Edit: I just remembered, this may have been a result of me misconfiguring the certificate object, leading to the creation of an extraneous rule. |
|
For the record, my problem was: I've been following Rancher HA setup guide which suggests having public-facing nginx load balancer. That is OK, but the problem is: their sample nginx config redirects all the HTTP traffic to HTTPS. I was having HTTPS enabled using their default self-signed certificate. That was obviously stopping let's encrypt from reaching the challenge URL. So, if you bump into this, make sure your traffic either allows HTTP or has HTTPS with a trusted cert. |
|
@xaralis I had the same issue, and it wasn't the case. Nginx redirected http traffic to well-known location without a problem. Still, validation failed. I am using nginx-ingress-0.23.0 and cert-manager-v0.4.1 helm charts. |
|
@saward I faced the same problem. What misconfiguration you had? |
|
I know there is a lot of chatter on this topic and wanted to give what I was seeing as well as what fixed it. In my case, I have had ingress successfully setup with cert-manager for two domains I recently added another host/rule/backend I also saw the following in the ingress logs Additionally, (and what led me to this thread) was the output of
Upon trying different things, within seconds of running a command to delete the
|
|
Hello I've the same problem described in this issue, waited for 5 days but cert-manager loop on "http-01 self check failed for domain" don't know what can I do in order to figure out the problem, on the same machine if I shutdown the docker enviroment and try to use the certbot client everithing works fine, I'm using the http-01 challenge. |
|
I'd try |
|
@rbq can you explain how to do that? I am facing kinda same issue and getting these errors; |
|
@bertoost My Ingress was available from my workstation via HTTP, yet cert-manager complained that it couldn't verify that the HTTP challenge it added was visible to letsencrypt. So I finally figured out that it couldn't reach my WAN address from behind the NAT. To verify, I started a container with curl (something like But yours looks like a totally different problem to me—it seems to be stuck before it even gets to the self-check. |
|
Hm okay. It's on my hosted VPS (not my local machine) and the weird thing is, I can access the host and view the website. Therefor I have successfully requested more certificates earlier for other projects, the same way, the same setup etc.. So, I really don't understand why this one is not working |
|
@bertoost I think it would make sense to open a separate issue and post some configuration details. |
|
somehow it is working .. just want to continue working on it, and suddenly it has a valid certificate retrieved from LetsEncrypt.. weird, but okay |
|
Just got into this error
This is my config: Found this on my nginx ingress logs:
Just changed the host option on my ingress rule and the issue was fixed: After that I had to put it back in place. |
|
Also encountered After looking at all ingresses This "letsencrypt cm-acme-http-solver" ingress is a temporary one and apparently there to intercept and answer the call to Not working: Working (notice the very last line Not sure if this is just a lucky coincidence now, or if it is really needed - ymmv |
|
My case: I changed NS DNS records, but after TTL expired, the nameserver set in kubernetes node was still pointing to the old server, which obviously was returning 404 for the HTTP challange. You can verify this using As a quick workaround, I temporally changed the nameserver in the node that was running cert-manager in If there is a better solution, then I'd be happy to hear it. |
|
Removing the NGINX ingress, the cert manager and the deployment that had a failing certificate, and adding it all back afterwards, fixed it for me too. |
|
I had this problem, I was following a tutorial that suggested to install I installed everything using |
|
I encountered this problem and the issue ended up being due to the fact that I was the setting This caused the self check GET request to return a "connection timed out" error. Removing the IP restrictions allowed the certificate to be successfully granted. |
I was using also yaml files, installing with helm fixed the issue. |
|
@AlirezaHaghshenas & @jc-delvalle comments helped. For anyone who have wasted enough time using kuberctl appy with yaml and getting the this issue, here are the full set of commands |
|
@rbq I have the same problem as you it seems. I can't reach the endpoint from within the cluster, but I can reach it from my local workstation. From within the cluster vs from local workstation: How did you fix it? |
|
Turns out the internal services in the cluster were not able to reach things within the cluster through external IP because I had enabled PROXY protcol in my load balancer. When I disabled PROXY protocol, the certificates were issued almost immediately. Which meant I could turn PROXY protocol back on: My application requires the use of PROXY protocol in order to check the users IP addresses. Is there a way of fixing this without having to switch PROXY protcol on and off every 90 days to renew my certs? |
|
Huge thanks @bergkvist, same issue here, appreciate you sharing the solution; spent all day trying everything except that it seems 😅 |
|
Patch for this definitely needed, and it seems that there are people/organizations willing and knowing how to help: |
|
@stopsopa So there is actually another alternative, which makes the self-checks work even with PROXY protocol enabled. Notice that you have to explicitly write your hostname ("example.com") in order for the kubernetes iptables issue to be worked around. Not sure how this would work if you have multiple hostnames pointing to the same loadbalancer. Subdomains work fine though (like www.example.com, subdomain.example.com etc.) |
|
Thanks @bergkvist mate I spent two evenings trying to make it work (cert-manager together with ip address), pulling my hair off, and actually I've been doing this too but with other configuration. I have created new cluster and tried only do-loadbalancer-hostname and... Finally worked. 🎉 |
|
Hi all, I ran into the same issue. I've just published It uses CoreDNS rewriting to intercept traffic that would be heading toward the external load balancer. It then adds a PROXY line to requests originating from within the cluster. This allows cert-manager's self-check to pass. |
|
What a coincidence. Just today I published https://github.com/nabsul/k8s-letsencrypt with instructions on how to manually issue certificates in your Kubernetes cluster. The hope being that I'll only need to manually issue certs a few times until this issue is fixed. I wish I'd seen @compumike 's solution sooner!! |
|
@compumike Thanks so much!! 🥇 |
|
I was able to fix this, the chain of issues started as follow: I had the following in the annotation in my ingress controller nginx.ingress.kubernetes.io/use-regex: "true"nginx.ingress.kubernetes.io/rewrite-target: /this caused all URLs to be rewritten to / commenting these 2 lines made things work |
|
In my case: It worked. |
|
As @ahmed-adly-khalil said: Removing
Worked |
and others
Is this a BUG REPORT or FEATURE REQUEST?:
/kind bug
What happened:
I get the message: http-01 self check failed for domain ""
$ kubectl describe certificates website-cert
If I get all the events:
What you expected to happen:
The self check to succeed
How to reproduce it (as minimally and precisely as possible):
Here is my Ingress:
Here is my Issuer:
Here is my certificate:
Anything else we need to know?:
When I navigate to
I get:
Also, if I look at the logs of the cm-acme pod:
Environment:
kubectl version):I've been struggling for two days. It's probably something really stupid from my side :)
Any idea?
The text was updated successfully, but these errors were encountered: