Skip to content

iso: Disable grub timeout speeding up vm start by 5 seconds #20895

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 2 commits into from
Jun 6, 2025

Conversation

nirs
Copy link
Contributor

@nirs nirs commented Jun 5, 2025

This speeds up machine boot by 5 seconds. The timeout may be helpful for
debugging boot issues but we don't have a way to access the serial
console for debugging currently.

Testing shows about ~5 seconds speedup on arm64 and no difference on amd64.

os arch driver timeout start time
macOS arm64 vfkit[1] 5.0 24.01
macOS arm64 vfkit[1] 0.0 19.90
macOS arm64 qemu 5.0 29.46
macOS arm64 qemu 0.0 24.28
macOS arm64 krunkit[2] 5.0 25.14
macOS arm64 krunkit[2] 0.0 20.51
linux amd64 kvm2 5.0 43.02
linux amd64 kvm2 0.0 43.11

[1] Tested with #20833, booting using iso instead of direct kernel
boot. Direct kernel boot is little bit faster (e.g. 18.x).
[2] Tested with #20826

This speeds up machine boot by 5 seconds. The timeout may be helpful for
debugging boot issues but we don't have a way to access the serial
console for debugging currently.

Testing shows about ~5 seconds speedup.

| driver     | timeout | start time |
|------------|---------|------------|
| vfkit[1]   |     5.0 |      24.01 |
| vfkit[1]   |     0.0 |      19.90 |
| qemu       |     5.0 |      29.46 |
| qemu       |     0.0 |      24.28 |
| krunkit[2] |     5.0 |      25.14 |
| krunkit[2] |     0.0 |      20.51 |

[1] Tested with kubernetes#20833, booting using iso instead of direct kernel
    boot. Direct kernel boot is little bit faster (e.g. 18.x).
[2] Tested with kubernetes#20826
@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jun 5, 2025
@k8s-ci-robot
Copy link
Contributor

Hi @nirs. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. labels Jun 5, 2025
@nirs nirs force-pushed the iso-disable-timeout branch from eb5386e to 0eec04f Compare June 5, 2025 19:36
@nirs
Copy link
Contributor Author

nirs commented Jun 5, 2025

ok-to-build-iso

@medyagh
Copy link
Member

medyagh commented Jun 5, 2025

this would be huge, if it works and doesnt cause issues ! 5 seconds boost for VM drivers is a like a dream

@nirs
Copy link
Contributor Author

nirs commented Jun 5, 2025

@minikube-bot
Copy link
Collaborator

Hi @nirs, we have updated your PR with the reference to newly built ISO. Pull the changes locally if you want to test with them or update your PR further.

@nirs
Copy link
Contributor Author

nirs commented Jun 6, 2025

ok-to-test

@nirs
Copy link
Contributor Author

nirs commented Jun 6, 2025

/ok-to-test

@k8s-ci-robot
Copy link
Contributor

@nirs: Cannot trigger testing until a trusted user reviews the PR and leaves an /ok-to-test message.

In response to this:

/ok-to-test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@nirs
Copy link
Contributor Author

nirs commented Jun 6, 2025

@medyagh can you trigger test? I added ‘ok-to-test’ but it does not work, and I’m not trusted for using ‘/ok-to-test’.

@medyagh
Copy link
Member

medyagh commented Jun 6, 2025

ok-to-test

@medyagh medyagh added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Jun 6, 2025
@minikube-pr-bot
Copy link

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 20895) |
+----------------+----------+---------------------+
| minikube start | 52.0s    | 52.6s               |
| enable ingress | 15.1s    | 15.3s               |
+----------------+----------+---------------------+

Times for minikube start: 51.5s 52.7s 53.0s 51.8s 51.2s
Times for minikube (PR 20895) start: 52.2s 52.3s 55.1s 52.3s 51.3s

Times for minikube ingress: 14.5s 16.0s 15.1s 14.6s 15.1s
Times for minikube (PR 20895) ingress: 15.6s 15.1s 15.6s 15.5s 14.6s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 20895) |
+----------------+----------+---------------------+
| minikube start | 23.2s    | 25.4s               |
| enable ingress | 13.1s    | 12.4s               |
+----------------+----------+---------------------+

Times for minikube start: 21.2s 25.3s 23.9s 23.5s 21.9s
Times for minikube (PR 20895) start: 25.8s 26.2s 27.3s 24.4s 23.1s

Times for minikube (PR 20895) ingress: 12.8s 12.3s 13.8s 10.3s 12.8s
Times for minikube ingress: 13.8s 13.3s 13.8s 13.8s 10.8s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 20895) |
+----------------+----------+---------------------+
| minikube start | 22.7s    | 22.9s               |
| enable ingress | 36.0s    | 35.8s               |
+----------------+----------+---------------------+

Times for minikube ingress: 38.8s 40.3s 38.8s 38.8s 23.3s
Times for minikube (PR 20895) ingress: 38.8s 38.3s 39.3s 38.8s 23.8s

Times for minikube start: 22.6s 20.0s 25.6s 23.0s 22.2s
Times for minikube (PR 20895) start: 23.6s 21.0s 24.3s 23.7s 22.0s

@nirs
Copy link
Contributor Author

nirs commented Jun 6, 2025

I checked the errors:

  • 8 tests with single failure (vulkano, known issue)
  • KVM_Linux_crio has some failure that we see in most builds recently, cannot be related to the change timeout
  • Docker_* tests has some failures but they don't use the iso
  • Functional_Verified / functional_docker_ubuntu_arm64 (pull_request) - looks like CI issue and cannot be related the iso.

@nirs
Copy link
Contributor Author

nirs commented Jun 6, 2025

Full benchmark results

vfkit darwin/arm64

Using #20833

% base="minikube start --driver vfkit --container-runtime containerd --iso-url"
% before="$base $PWD/minikube-arm64.iso"
% after="$base $PWD/minikube-arm64-timeout-0.iso"
% hyperfine -r 10 -C "minikube delete" "$before" "$after"
Benchmark 1: minikube start --driver vfkit --container-runtime containerd --iso-url /Users/nir/src/minikube/minikube-arm64.iso
  Time (mean ± σ):     24.831 s ±  0.908 s    [User: 0.932 s, System: 0.600 s]
  Range (min … max):   23.482 s … 26.076 s    10 runs
 
Benchmark 2: minikube start --driver vfkit --container-runtime containerd --iso-url /Users/nir/src/minikube/minikube-arm64-timeout-0.iso
  Time (mean ± σ):     19.780 s ±  0.605 s    [User: 0.979 s, System: 0.591 s]
  Range (min … max):   18.807 s … 20.360 s    10 runs
 
Summary
  minikube start --driver vfkit --container-runtime containerd --iso-url /Users/nir/src/minikube/minikube-arm64-timeout-0.iso ran
    1.26 ± 0.06 times faster than minikube start --driver vfkit --container-runtime containerd --iso-url /Users/nir/src/minikube/minikube-arm64.iso

krunkit benchmarks darwin/arm64

Using #20826

% base="minikube start --driver krunkit --container-runtime containerd --iso-url"
% before="$base $PWD/minikube-arm64.iso"
% after="$base $PWD/minikube-arm64-timeout-0.iso"
% hyperfine -r 10 -C "minikube delete" "$before" "$after"
Benchmark 1: minikube start --driver krunkit --container-runtime containerd --iso-url /Users/nir/src/minikube/minikube-arm64.iso
  Time (mean ± σ):     26.519 s ±  1.120 s    [User: 1.067 s, System: 1.007 s]
  Range (min … max):   25.326 s … 28.687 s    10 runs
 
Benchmark 2: minikube start --driver krunkit --container-runtime containerd --iso-url /Users/nir/src/minikube/minikube-arm64-timeout-0.iso
  Time (mean ± σ):     21.126 s ±  0.658 s    [User: 1.095 s, System: 0.975 s]
  Range (min … max):   19.989 s … 22.168 s    10 runs
 
Summary
  minikube start --driver krunkit --container-runtime containerd --iso-url /Users/nir/src/minikube/minikube-arm64-timeout-0.iso ran
    1.26 ± 0.07 times faster than minikube start --driver krunkit --container-runtime containerd --iso-url /Users/nir/src/minikube/minikube-arm64.iso

qemu benchmark darwin/arm64

% base="minikube start --driver qemu --container-runtime containerd --iso-url"
% before="$base $PWD/minikube-arm64.iso"
% after="$base $PWD/minikube-arm64-timeout-0.iso"
% hyperfine -r 10 -C "minikube delete" "$before" "$after"
Benchmark 1: minikube start --driver qemu --container-runtime containerd --iso-url /Users/nir/src/minikube/minikube-arm64.iso
  Time (mean ± σ):     30.883 s ±  0.736 s    [User: 0.972 s, System: 0.874 s]
  Range (min … max):   28.990 s … 31.555 s    10 runs
 
Benchmark 2: minikube start --driver qemu --container-runtime containerd --iso-url /Users/nir/src/minikube/minikube-arm64-timeout-0.iso
  Time (mean ± σ):     25.787 s ±  0.819 s    [User: 1.001 s, System: 0.870 s]
  Range (min … max):   24.632 s … 27.021 s    10 runs
 
Summary
  minikube start --driver qemu --container-runtime containerd --iso-url /Users/nir/src/minikube/minikube-arm64-timeout-0.iso ran
    1.20 ± 0.05 times faster than minikube start --driver qemu --container-runtime containerd --iso-url /Users/nir/src/minikube/minikube-arm64.iso

kvm benchmark linux/amd64 - bare metal

$ base="minikube start --driver kvm2 --container-runtime containerd --network default --iso-url"
$ before="$base $HOME/.minikube/cache/iso/amd64/minikube-v1.36.0-1749066232-20832-amd64.iso"
$ after="$base $HOME/.minikube/cache/iso/amd64/minikube-v1.36.0-1749153077-20895-amd64.iso"
$ hyperfine -r 10 -C "minikube delete" "$before" "$after"
Benchmark 1: minikube start --driver kvm2 --container-runtime containerd --network default --iso-url /home/nsoffer/.minikube/cache/iso/amd64/minikube-v1.36.0-1749066232-20832-amd64.iso
  Time (mean ± σ):     41.763 s ±  2.042 s    [User: 1.945 s, System: 0.840 s]
  Range (min … max):   38.190 s … 46.214 s    20 runs
 
Benchmark 2: minikube start --driver kvm2 --container-runtime containerd --network default --iso-url /home/nsoffer/.minikube/cache/iso/amd64/minikube-v1.36.0-1749153077-20895-amd64.iso
  Time (mean ± σ):     42.377 s ±  2.449 s    [User: 2.022 s, System: 0.853 s]
  Range (min … max):   37.509 s … 46.511 s    20 runs
 
Summary
  minikube start --driver kvm2 --container-runtime containerd --network default --iso-url /home/nsoffer/.minikube/cache/iso/amd64/minikube-v1.36.0-1749066232-20832-amd64.iso ran
    1.01 ± 0.08 times faster than minikube start --driver kvm2 --container-runtime containerd --network default --iso-url /home/nsoffer/.minikube/cache/iso/amd64/minikube-v1.36.0-1749153077-20895-amd64.iso

kvm benchmark linux/amd64 - vm

$ base="minikube start --driver kvm2 --container-runtime containerd --network default --iso-url"
$ before="$base $HOME/.minikube/cache/iso/amd64/minikube-v1.36.0-1749066232-20832-amd64.iso"
$ after="$base $HOME/.minikube/cache/iso/amd64/minikube-v1.36.0-1749153077-20895-amd64.iso"
$ hyperfine -r 10 -C "minikube delete" "$before" "$after"
Benchmark 1: minikube start --driver kvm2 --container-runtime containerd --network default --iso-url /home/nsoffer/.minikube/cache/iso/amd64/minikube-v1.36.0-1749066232-20832-amd64.iso
  Time (mean ± σ):     44.420 s ±  1.587 s    [User: 1.839 s, System: 0.877 s]
  Range (min … max):   42.443 s … 47.012 s    10 runs
 
Benchmark 2: minikube start --driver kvm2 --container-runtime containerd --network default --iso-url /home/nsoffer/.minikube/cache/iso/amd64/minikube-v1.36.0-1749153077-20895-amd64.iso
  Time (mean ± σ):     45.004 s ±  1.412 s    [User: 1.805 s, System: 0.847 s]
  Range (min … max):   43.214 s … 47.836 s    10 runs
 
Summary
  minikube start --driver kvm2 --container-runtime containerd --network default --iso-url /home/nsoffer/.minikube/cache/iso/amd64/minikube-v1.36.0-1749066232-20832-amd64.iso ran
    1.01 ± 0.05 times faster than minikube start --driver kvm2 --container-runtime containerd --network default --iso-url /home/nsoffer/.minikube/cache/iso/amd64/minikube-v1.36.0-1749153077-20895-amd64.iso

@medyagh
Copy link
Member

medyagh commented Jun 6, 2025

this is incridible ! I cant believe we are deliering 5s faster minikube, in year 2025 ! that took us so much effort to cut "1s" and this simple PR shaves 5s, great job @nirs

@medyagh
Copy link
Member

medyagh commented Jun 6, 2025

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jun 6, 2025
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: afbjorklund, medyagh, nirs

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jun 6, 2025
@medyagh medyagh merged commit 57dd4b3 into kubernetes:master Jun 6, 2025
22 of 37 checks passed
@nirs nirs deleted the iso-disable-timeout branch July 2, 2025 16:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants