[BUG] crc start fails on Waiting until the user's pull secret is written to the instance disk... step
#4110
Comments
|
Update: restarting the command was successful at passing the step |
|
lavocatt
changed the title
Waiting for kube-apiserver availability... step Waiting until the user's pull secret is written to the instance disk... step
|
Discussion with @praveenkumar and @cfergeau we found out that the issue was linked to the DNS resolution. This is the working configuration: Thanks for your help! |
|
Would stil be nice to fix the DNS resolution problem you were having :) |
I can reopen the issue if you want |
|
I'm getting this same error. I can't tell - is there a solution? I do have these set, which I thought would do it, but does not: This is with v2.34.1 |
DNS resolution is a network setup issue, ... What was the DNS server set to? Any specific would help to determine what causes a lookup failure.
More info about the DNS used is necessary to determine the problem. |
|
Well, that's interesting. I turned off my corporate VPN and it started fine. What would that indicate? Is there something I can do for this to work if I am on the VPN? I would like to remain on VPN if possible. Note, when I just got this to work, I did have I can try it without those settings just to see what happens. But at minimum, the above works for me when I am off the VPN. |
|
OK, this is interesting. It is hung again. This is what I did. I completely removed my So it seems like those network-mode/host-network-access had some positive effect when I am not on VPN. I am going to try this test again. I will completely remove $HOME/.crc again, only this time I will explicitly set those two network config options while remaining off the VPN. I'll see if that works. (and I would like to repeat - none of these problems occur with v2.32.0 - so something changed between that version of CRC and v2.34.1 that is causing this problem) |
|
It failed when I turned off those network settings, but now, even worse, after control-c'ing the hung crc process, I can't restart anything. Continue to get the following: I see in my crc output logs: I ran |
|
I'm back able to start (but this time I went back to 2.32.0). I'm staying on that version for now :) I am on Fedora 39 (RedHat CSB machine) - and I know one other person who sees the same problem as I do on 2.34.1 on the same operating system. I can likely help work on debugging this with someone on Monday - but for now I'm going to continue using with 2.32.0 and ignore upgrading. |
|
This is the log with the "hang" problem as it tries to store the pull secret. If you want to see the full series of calls I made to set up crc, I wrap it in this hack script. I was NOT on the VPN, and I did NOT have the network configs set. |
|
And here is the CRC logs when I can start it successfully: I was NOT on the VPN but I DID have the network configs set: (There is one other less important thing that is interesting. Trying to connect via ssh to the CRC cluster doesn't work like it does when I can start it on 2.32. This command: would normally just work and not require me to enter a password, but now it is asking me for a password. The -i identity should let me in though. So I don't know what is different. Maybe things changed in 2.34.1 wrt ssh access?) |
|
After investigating with Fedora-CSB laptops looks like the Workaround-1: Disable/mask the Workaround-2: Make sure |
I can confirm the above workaround works for me. |
|
@praveenkumar just so I'm clear - the workarounds you mentioned, are those going to be the final solution for those with this problem? Or is the plan for CRC to change in a future version so the workarounds will not be needed? |
|
The workaround does not apply to my setup as I routinely disable system-resolved and run by own DNS server through docker. see #4144 for details of my configuration. |
* [hack] bump crc to 2.34.1 / openshift 4.15 see crc-org/crc#4110 (comment) * add cleanup. now delete won't purge the downloaded bundles so it is faster to restart everything. but if you want to clean things up, the new cleanup command is what you want
|
By applying the workaround, I can execute OCP 4.15 on my Fedora 39, but as a side effect, I am unable to run a Minikube cluster 😄 minikube v1.32.0 on Fedora 39 If this happens, you have to unmask and then enable |
|
Just want to close the loop on this - all my problems seem to have gone away with the new CRC v2.36.0. I can run CRC without messing with systemd-resolved service and I am still able to also start minikube successfully (no longer get that error |
General information
crc setupbefore starting it (Yes/No)? yesCRC version
CRC status
crc status --log-level debug DEBU CRC version: 2.34.1+b470b5 DEBU OpenShift version: 4.15.3 DEBU Podman version: 4.4.4 DEBU Running 'crc status' CRC VM: Running OpenShift: Unreachable (v4.15.3) RAM Usage: 1.932GB of 10.96GB Disk Usage: 19.05GB of 32.68GB (Inside the CRC VM) Cache Usage: 26.27GB Cache Directory: /home/tlavocat/.crc/cacheCRC config
crc config view - consent-telemetry : yesHost Operating System
Steps to reproduce
Expected
I expect a normal startup.
Actual
Logs
Before gather the logs try following if that fix your issue
Please consider posting the output of
crc start --log-level debugon http://gist.github.com/ and post the link in the issue.See: https://gist.github.com/lavocatt/3f5d8fca0ac3ad52ddae982b906c3080
The text was updated successfully, but these errors were encountered: