[BUG] Times out the first time, every time

[bmaupin]

General information

• OS: Linux
• Hypervisor: KVM? (I didn't configure anything)
• Did you run crc setup before starting it (Yes/No)? Yes
• Running CRC on: Laptop

CRC version

CRC version: 2.10.2+1bc2e5a
OpenShift version: 4.11.7
Podman version: 4.2.0

CRC status

DEBU CRC version: 2.10.2+1bc2e5a                  
DEBU OpenShift version: 4.11.7                    
DEBU Podman version: 4.2.0                        
DEBU Running 'crc status'                         
CRC VM:          Running
OpenShift:       Running (v4.11.7)
RAM Usage:       6.944GB of 9.399GB
Disk Usage:      17.92GB of 32.74GB (Inside the CRC VM)
Cache Usage:     16.43GB
Cache Directory: /home/user/.crc/cache

CRC config

- consent-telemetry                     : yes

Host Operating System

$ cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.1 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.1 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy

Steps to reproduce

1. Run crc setup and then crc start when the machine is first booted

Expected

I would expect CRC to start.

Actual

CRC times out after 10 minutes:

INFO Operator operator-lifecycle-manager-packageserver is progressing 
INFO Operator operator-lifecycle-manager-packageserver is progressing 
ERRO Cluster is not ready: cluster operators are still not stable after 10m0.480525134s 
INFO Adding crc-admin and crc-developer contexts to kubeconfig... 
ERRO Cannot update kubeconfig: Head "https://oauth-openshift.apps-crc.testing": dial tcp 192.168.130.11:443: connect: connection refused

Then I run crc stop and crc start again and it works:

$ time crc start
INFO Checking if running as non-root              
INFO Checking if running inside WSL2              
INFO Checking if crc-admin-helper executable is cached 
INFO Checking for obsolete admin-helper executable 
INFO Checking if running on a supported CPU architecture 
INFO Checking minimum RAM requirements            
INFO Checking if crc executable symlink exists    
INFO Checking if Virtualization is enabled        
INFO Checking if KVM is enabled                   
INFO Checking if libvirt is installed             
INFO Checking if user is part of libvirt group    
INFO Checking if active user/process is currently part of the libvirt group 
INFO Checking if libvirt daemon is running        
INFO Checking if a supported libvirt version is installed 
INFO Checking if crc-driver-libvirt is installed  
INFO Checking crc daemon systemd socket units     
INFO Checking if AppArmor is configured           
INFO Checking if systemd-networkd is running      
INFO Checking if NetworkManager is installed      
INFO Checking if NetworkManager service is running 
INFO Checking if dnsmasq configurations file exist for NetworkManager 
INFO Checking if the systemd-resolved service is running 
INFO Checking if /etc/NetworkManager/dispatcher.d/99-crc.sh exists 
INFO Checking if libvirt 'crc' network is available 
INFO Checking if libvirt 'crc' network is active  
INFO Loading bundle: crc_libvirt_4.11.7_amd64...  
INFO Starting CRC VM for openshift 4.11.7...      
INFO CRC instance is running with IP 192.168.130.11 
INFO CRC VM is running                            
INFO Configuring shared directories               
INFO Check internal and public DNS query...       
INFO Check DNS query from host...                 
WARN Wildcard DNS resolution for apps-crc.testing does not appear to be working 
INFO Verifying validity of the kubelet certificates... 
INFO Starting kubelet service                     
INFO Waiting for kube-apiserver availability... [takes around 2min] 
INFO Waiting for user's pull secret part of instance disk... 
INFO Starting openshift instance... [waiting for the cluster to stabilize] 
INFO Operator operator-lifecycle-manager-packageserver is progressing 
INFO Operator operator-lifecycle-manager-packageserver is progressing 
INFO 2 operators are degraded: authentication, kube-controller-manager 
INFO 2 operators are degraded: authentication, kube-controller-manager 
INFO 2 operators are degraded: authentication, kube-controller-manager 
INFO 2 operators are degraded: authentication, kube-controller-manager 
INFO 2 operators are degraded: authentication, kube-controller-manager 
INFO 2 operators are degraded: authentication, kube-controller-manager 
INFO 2 operators are degraded: authentication, kube-controller-manager 
INFO 2 operators are degraded: authentication, kube-controller-manager 
INFO Operator authentication is degraded          
INFO Operator openshift-apiserver is not yet available 
INFO All operators are available. Ensuring stability... 
INFO Operators are stable (2/3)...                
INFO Operators are stable (3/3)...                
INFO Adding crc-admin and crc-developer contexts to kubeconfig... 
Started the OpenShift cluster.

The server is accessible via web console at:
  https://console-openshift-console.apps-crc.testing

Log in as administrator:
  Username: kubeadmin
  Password: xxxxxxxxxxxxxxx

Log in as user:
  Username: developer
  Password: developer

Use the 'oc' command line interface:
  $ eval $(crc oc-env)
  $ oc login -u developer https://api.crc.testing:6443

real	7m42.921s
user	0m0.731s
sys	0m0.147s

Logs

Before gather the logs try following if that fix your issue

$ crc delete -f
$ crc cleanup
$ crc setup
$ crc start --log-level debug

Please consider posting the output of crc start --log-level debug on http://gist.github.com/ and post the link in the issue.

I think I'd have to reboot my machine to try that since it seems to start up fine after the first time. But I was wondering, is there any way to increase the 10 minute timeout? I'm wondering if it would start the first time if I could increase it.

[praveenkumar]

@bmaupin We have default timeout set to 10mins and mostly during that time cluster reconcile happen successfully. Have you checked crc status after sometime even the reconciliation doesn't happen during set time?

[bmaupin]

I tried the steps above and it worked:

$ crc delete -f
$ crc cleanup
$ crc setup
$ crc start --log-level debug

Am I supposed to run crc setup every time before I run crc start? If so, I wonder why the functionality for crc setup isn't just done automatically when crc start is run. Seems like something people are likely to forget.

I wouldn't be surprised if my issues are caused by forgetting to run crc setup first.

Thanks!

[bmaupin]

I think I'm just forgetting to run crc setup first, so I'll go ahead and close this. But I might open a separate feature request because I forget to run crc setup a lot and it's annoying to not realize it until 10 minutes later when crc start times out.

[bmaupin]

I thought this was caused by me forgetting to run crc setup every time before running crc start, but that doesn't appear to be the case. I'll reopen it and try to generate more detailed logs as requested in the issue template.

[praveenkumar]

@bmaupin can we close this one since the reference issue #3548 is resolved?

[gbraad]

We are unable to reproduce this on multiple other distros.

[bmaupin]

I haven't experienced the issue with more recent versions of CRC.