--delete-on-faliure recreates cluster for kubeadm failures as well

[ComradeProgrammer]

feat: allow delete-on-faliure in setupKubeAdm
fix #16682

If we generate an error in setupKubeAdm deliberately, then:

Before:

tjm@tjm:~/workspace/minikube/out$ ./minikube start  --delete-on-failure
😄  minikube v1.30.1 on Ubuntu 22.04 (amd64)
✨  Automatically selected the docker driver
📌  Using Docker driver with root privileges
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
💾  Downloading Kubernetes v1.27.3 preload ...
    > preloaded-images-k8s-v18-v1...:  393.19 MiB / 393.19 MiB  100.00% 12.13 M
    > index.docker.io/kicbase/bui...:  430.77 MiB / 430.77 MiB  100.00% 8.91 Mi
❗  minikube was unable to download gcr.io/k8s-minikube/kicbase-builds:v0.0.39-1687538068-16731, but successfully downloaded docker.io/kicbase/build:v0.0.39-1687538068-16731 as a fallback image
🔥  Creating docker container (CPUs=2, Memory=2200MB) ...
🐳  Preparing Kubernetes v1.27.3 on Docker 24.0.2 ...

❌  Exiting due to K8S_INSTALL_FAILED: Failed to update cluster: This error is expected

╭───────────────────────────────────────────────────────────────────────────────────────────╮
│                                                                                           │
│    😿  If the above advice does not help, please let us know:                             │
│    👉  https://github.com/kubernetes/minikube/issues/new/choose                           │
│                                                                                           │
│    Please run `minikube logs --file=logs.txt` and attach logs.txt to the GitHub issue.    │
│                                                                                           │
╰───────────────────────────────────────────────────────────────────────────────────────────╯

When this error happens, minikube exits at once.

After:

tjm@tjm:~/workspace/minikube/out$ ./minikube start --delete-on-failure
😄  minikube v1.30.1 on Ubuntu 22.04 (amd64)
✨  Using the docker driver based on existing profile
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
🏃  Updating the running docker "minikube" container ...
🐳  Preparing Kubernetes v1.27.3 on Docker 24.0.2 .../ E0715 19:19:41.660942   29807 start.go:583] Failed to update cluster: This error is expected

❗  Node  failed to start, deleting and trying again.
🔥  Deleting "minikube" in docker ...
🔥  Deleting container "minikube" ...
🔥  Removing /home/tjm/.minikube/machines/minikube ...
💀  Removed all traces of the "minikube" cluster.
👍  Starting control plane node minikube in cluster minikube
🚜  Pulling base image ...
🔥  Creating docker container (CPUs=2, Memory=2200MB) ...- ^C

When this error happens, it trys to delete the cluster and retry

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ComradeProgrammer
Once this PR has been reviewed and has the lgtm label, please assign medyagh for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-ci-robot]

Hi @ComradeProgrammer. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[minikube-bot]

Can one of the admins verify this patch?

[medyagh]

/ok-to-test

[medyagh]

@ComradeProgrammer can u checkout the testerrorspam ?

seems like it might be adding a spam err in normal start

[ComradeProgrammer]

@ComradeProgrammer can u checkout the testerrorspam ?

seems like it might be adding a spam err in normal start

It's problem with minikube's version. My makefile is too old

I have rebased the PR so that the version number in the newest makefile will be used

I think it will be ok soon

[minikube-pr-bot]

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 16890) |
+----------------+----------+---------------------+
| minikube start | 52.3s    | 54.2s               |
| enable ingress | 28.1s    | 28.0s               |
+----------------+----------+---------------------+

Times for minikube start: 53.9s 51.8s 51.5s 52.1s 52.2s
Times for minikube (PR 16890) start: 52.7s 55.7s 52.8s 54.6s 55.1s

Times for minikube ingress: 27.8s 28.2s 27.8s 28.3s 28.3s
Times for minikube (PR 16890) ingress: 27.8s 28.8s 27.3s 27.8s 28.3s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 16890) |
+----------------+----------+---------------------+
| minikube start | 23.4s    | 23.8s               |
| enable ingress | 48.5s    | 48.8s               |
+----------------+----------+---------------------+

Times for minikube (PR 16890) start: 23.3s 25.1s 22.9s 25.0s 22.7s
Times for minikube start: 23.2s 23.7s 22.8s 22.6s 25.0s

Times for minikube ingress: 48.4s 48.4s 48.9s 47.9s 48.9s
Times for minikube (PR 16890) ingress: 48.4s 49.4s 49.4s 48.4s 48.4s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 16890) |
+----------------+----------+---------------------+
| minikube start | 21.3s    | 22.5s               |
| enable ingress | 40.9s    | 29.0s               |
+----------------+----------+---------------------+

Times for minikube start: 22.3s 20.6s 21.3s 20.8s 21.4s
Times for minikube (PR 16890) start: 21.1s 24.0s 24.1s 20.2s 22.9s

Times for minikube (PR 16890) ingress: 18.9s 32.4s 30.9s 31.4s 31.4s
Times for minikube ingress: 31.4s 31.4s 30.9s 31.4s 79.4s

[minikube-pr-bot]

These are the flake rates of all failed tests.

Environment	Failed Tests	Flake Rate (%)
KVM_Linux_containerd	TestErrorSpam/setup (gopogh)	0.58 (chart)
KVM_Linux	TestErrorSpam/setup (gopogh)	0.58 (chart)
KVM_Linux_crio	TestErrorSpam/setup (gopogh)	0.59 (chart)
Hyperkit_macOS	TestErrorSpam/setup (gopogh)	0.60 (chart)
Docker_Linux_crio	TestFunctional/parallel/ImageCommands/ImageTagAndLoadDaemon (gopogh)	2.94 (chart)
Hyperkit_macOS	TestNetworkPlugins/group/custom-flannel/Start (gopogh)	4.19 (chart)
Hyperkit_macOS	TestMinikubeProfile (gopogh)	7.78 (chart)
Docker_Linux_containerd_arm64	TestDockerEnvContainerd (gopogh)	10.00 (chart)
Docker_Linux_crio	TestPause/serial/SecondStartNoReconfiguration (gopogh)	14.12 (chart)
Docker_macOS	TestErrorSpam/setup (gopogh)	20.86 (chart)
Docker_Linux_containerd_arm64	TestErrorSpam/setup (gopogh)	21.05 (chart)
Docker_Linux_crio_arm64	TestErrorSpam/setup (gopogh)	21.05 (chart)
Docker_Linux_docker_arm64	TestErrorSpam/setup (gopogh)	21.05 (chart)
Docker_Linux	TestErrorSpam/setup (gopogh)	21.05 (chart)
Docker_Linux_containerd	TestErrorSpam/setup (gopogh)	21.18 (chart)
Docker_Linux_crio	TestErrorSpam/setup (gopogh)	21.18 (chart)

To see the flake rates of all tests by environment, click here.