Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

rpm: Update virtualization packages #11256

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

andreabolognani
Copy link
Contributor

What this PR does

Update virtualization packages. Specifically:

  • QEMU (8.0.0 → 8.2.0)
  • libvirt (9.5.0 → 10.0.0)
  • SeaBIOS (1.16.1 → 1.16.3)
  • EDKII (20230524 → 20231122)
  • passt (20230818 → 20231204)
  • virtiofsd (1.7.2 → 1.1.10.1)
  • guestfs-tools (1.50.1 → 1.51.6)

Fixes #10208 by adding the usb-redir device on Arm.

Release note

This version of KubeVirt includes upgraded virtualization technology based on libvirt 10.0.0 and QEMU 8.2.0.
Each new release of libvirt and QEMU contains numerous improvements and bug fixes.

@kubevirt-bot kubevirt-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. do-not-merge/invalid-commit-message dco-signoff: yes Indicates the PR's author has DCO signed all their commits. size/XXL labels Feb 15, 2024
@xpivarc
Copy link
Member

xpivarc commented Feb 15, 2024

/test

@kubevirt-bot
Copy link
Contributor

@xpivarc: The /test command needs one or more targets.
The following commands are available to trigger required jobs:

  • /test pull-kubevirt-apidocs
  • /test pull-kubevirt-build
  • /test pull-kubevirt-build-arm64
  • /test pull-kubevirt-check-unassigned-tests
  • /test pull-kubevirt-client-python
  • /test pull-kubevirt-code-lint
  • /test pull-kubevirt-e2e-k8s-1.27-sig-compute
  • /test pull-kubevirt-e2e-k8s-1.27-sig-network
  • /test pull-kubevirt-e2e-k8s-1.27-sig-operator
  • /test pull-kubevirt-e2e-k8s-1.27-sig-performance
  • /test pull-kubevirt-e2e-k8s-1.27-sig-storage
  • /test pull-kubevirt-e2e-k8s-1.28-sig-compute
  • /test pull-kubevirt-e2e-k8s-1.28-sig-network
  • /test pull-kubevirt-e2e-k8s-1.28-sig-operator
  • /test pull-kubevirt-e2e-k8s-1.28-sig-storage
  • /test pull-kubevirt-e2e-k8s-1.29-ipv6-sig-network
  • /test pull-kubevirt-e2e-k8s-1.29-sig-compute
  • /test pull-kubevirt-e2e-k8s-1.29-sig-compute-migrations
  • /test pull-kubevirt-e2e-k8s-1.29-sig-network
  • /test pull-kubevirt-e2e-k8s-1.29-sig-operator
  • /test pull-kubevirt-e2e-k8s-1.29-sig-storage
  • /test pull-kubevirt-e2e-kind-1.27-vgpu
  • /test pull-kubevirt-e2e-kind-sriov
  • /test pull-kubevirt-e2e-windows2016
  • /test pull-kubevirt-fossa
  • /test pull-kubevirt-generate
  • /test pull-kubevirt-manifests
  • /test pull-kubevirt-prom-rules-verify
  • /test pull-kubevirt-unit-test
  • /test pull-kubevirt-verify-go-mod

The following commands are available to trigger optional jobs:

  • /test build-kubevirt-builder
  • /test pull-kubevirt-build-s390x
  • /test pull-kubevirt-check-tests-for-flakes
  • /test pull-kubevirt-conformance-arm64
  • /test pull-kubevirt-e2e-arm64
  • /test pull-kubevirt-e2e-k8s-1.29-sig-compute-realtime
  • /test pull-kubevirt-e2e-k8s-1.29-sig-compute-root
  • /test pull-kubevirt-e2e-k8s-1.29-sig-monitoring
  • /test pull-kubevirt-e2e-k8s-1.29-sig-network-multus-v4
  • /test pull-kubevirt-e2e-k8s-1.29-sig-storage-root
  • /test pull-kubevirt-e2e-k8s-1.29-single-node
  • /test pull-kubevirt-e2e-k8s-1.29-swap-enabled
  • /test pull-kubevirt-gosec
  • /test pull-kubevirt-goveralls
  • /test pull-kubevirt-metrics-lint
  • /test pull-kubevirt-unit-test-arm64
  • /test pull-kubevirt-verify-rpms

Use /test all to run the following jobs that were automatically triggered:

  • pull-kubevirt-apidocs
  • pull-kubevirt-build
  • pull-kubevirt-build-arm64
  • pull-kubevirt-check-tests-for-flakes
  • pull-kubevirt-check-unassigned-tests
  • pull-kubevirt-client-python
  • pull-kubevirt-code-lint
  • pull-kubevirt-conformance-arm64
  • pull-kubevirt-e2e-arm64
  • pull-kubevirt-e2e-k8s-1.27-sig-performance
  • pull-kubevirt-e2e-k8s-1.29-sig-compute
  • pull-kubevirt-e2e-k8s-1.29-sig-compute-migrations
  • pull-kubevirt-e2e-k8s-1.29-sig-network
  • pull-kubevirt-e2e-k8s-1.29-sig-operator
  • pull-kubevirt-e2e-k8s-1.29-sig-storage
  • pull-kubevirt-fossa
  • pull-kubevirt-generate
  • pull-kubevirt-goveralls
  • pull-kubevirt-manifests
  • pull-kubevirt-prom-rules-verify
  • pull-kubevirt-unit-test
  • pull-kubevirt-unit-test-arm64
  • pull-kubevirt-verify-go-mod
  • pull-kubevirt-verify-rpms

In response to this:

/test

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@xpivarc
Copy link
Member

xpivarc commented Feb 15, 2024

/test pull-kubevirt-verify-rpms

@andreabolognani
Copy link
Contributor Author

/retest-required

@xpivarc
Copy link
Member

xpivarc commented Feb 15, 2024

@andreabolognani Thanks
/approve
/lgtm

@kubevirt-bot kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Feb 15, 2024
@kubevirt-bot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: xpivarc

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubevirt-bot kubevirt-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 15, 2024
@kubevirt-commenter-bot
Copy link

Required labels detected, running phase 2 presubmits:
/test pull-kubevirt-e2e-windows2016
/test pull-kubevirt-e2e-kind-1.27-vgpu
/test pull-kubevirt-e2e-kind-sriov
/test pull-kubevirt-e2e-k8s-1.29-ipv6-sig-network
/test pull-kubevirt-e2e-k8s-1.27-sig-network
/test pull-kubevirt-e2e-k8s-1.27-sig-storage
/test pull-kubevirt-e2e-k8s-1.27-sig-compute
/test pull-kubevirt-e2e-k8s-1.27-sig-operator
/test pull-kubevirt-e2e-k8s-1.28-sig-network
/test pull-kubevirt-e2e-k8s-1.28-sig-storage
/test pull-kubevirt-e2e-k8s-1.28-sig-compute
/test pull-kubevirt-e2e-k8s-1.28-sig-operator

@xpivarc
Copy link
Member

xpivarc commented Feb 15, 2024

/hold

@kubevirt-bot kubevirt-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Feb 15, 2024
@andreabolognani
Copy link
Contributor Author

@xpivarc I'm seeing mostly timeouts among the failures. I'll start looking into it, but do you have any tips or gut feelings about what could be causing the issues? Just to speed things up if possible :)

@victortoso
Copy link
Member

Fixes #10208 by adding the usb-redir device on Arm.

Ah, cool.

Perhaps we can make this test run on arm? Based on this commit, all is needed would be adding [arm64] to it.

@andreabolognani
Copy link
Contributor Author

Perhaps we can make this test run on arm? Based on this commit, all is needed would be adding [arm64] to it.

Good idea, but I'll let @zhlhahaha take care of that after this PR has been merged. I wouldn't want to risk holding up the RPM update any longer than it's necessary.

@rmohr
Copy link
Member

rmohr commented Feb 16, 2024

@andreabolognani FYI, seeing the same migration tests failing in #10346 (comment) after bumping only qemu to install an additional tool in the containers.

@zhlhahaha
Copy link
Contributor

Perhaps we can make this test run on arm? Based on this commit, all is needed would be adding [arm64] to it.

Good idea, but I'll let @zhlhahaha take care of that after this PR has been merged. I wouldn't want to risk holding up the RPM update any longer than it's necessary.

I will give it a try, but currently we get some issue on the e2e tests on Arm64, I will fix it as soon as possible.

@zhlhahaha
Copy link
Contributor

this test

I have verified the USB Redirection e2e tests. All three tests have passed on Arm64 platform.

@andreabolognani
Copy link
Contributor Author

@rmohr I haven't been able to reproduce failure locally (at least for pull-kubevirt-e2e-k8s-1.27-sig-operator, which is the one I've tried so far).

In all cases we seem to be hitting timeouts though, so I'm not sure if things are actually stuck, gotten slower enough that what was previously a tight deadline could no longer be reached in time, or being interfered with by higher than average load on the test cluster? How does one tell those apart, and decide whether e.g. bumping the timeout would be an acceptable course of action?

I'm kinda stumped here, so any help would be highly appreciated.

@andreabolognani
Copy link
Contributor Author

Pick up libvirt-10.0.0-5.el9 with a hack so that we can at least make sure that there are no other issues waiting for us after that update has been unblocked.

@xpivarc
Copy link
Member

xpivarc commented Apr 10, 2024

Live migration failed error encountered during MigrateToURI3 libvirt api call: virError(Code=67, Domain=31, Message='unsupported configuration: Target CPU feature count 32 does not match source 103')",

@kubevirt-bot kubevirt-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 10, 2024
@andreabolognani
Copy link
Contributor Author

Live migration failed error encountered during MigrateToURI3 libvirt api call: virError(Code=67, Domain=31, Message='unsupported configuration: Target CPU feature count 32 does not match source 103')",

@xpivarc that's not surprising, it's the other issue that I was talking about (RHEL-30622) and that hasn't been resolved upstream yet :)

Can you please try to fish out the various XMLs used in that test? The one for the VM running on the original node, the one used for migration, anything else that might be relevant? That could be useful to track down the issue.

kubevirt-bot pushed a commit to kubevirt/kubevirtci that referenced this pull request Apr 10, 2024
* Bump alpine to 3.19

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

* Add networking service to alpine start

Otherwise, the network interfaces will not be brought up:
kubevirt/kubevirt#11256 (comment)

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>

---------

Signed-off-by: Alex Kalenyuk <akalenyu@redhat.com>
@jschintag jschintag mentioned this pull request Apr 18, 2024
8 tasks
@kubevirt-bot kubevirt-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 19, 2024
@andreabolognani
Copy link
Contributor Author

Updated to pick up the official 3.19 alpine-with-test-tooling-container-disk image from kubevirt/kubevirtci#1167. Sadly, we're still missing the fix for RHEL-30622.

The function that is used to create VMIs takes the amount of
memory as a parameter, but the contents of the corresponding
spec.domain.resources.requests entry were not affected by it
and always used the default value of 128Mi.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
The values can change over time, and when they do it shouldn't
be necessary to update the messages too.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
The current sed script works under the assumption that the
sshd_config file contains the line

  PasswordAuthentication no

That is the case for the current Alpine 3.16 image, though I
haven't been able to figure out where it comes from since it
doesn't appear to exits even in the old Alpine package. The
line is not there in the Alpine 3.19 image, which means that
the sed script does nothing and password authentication
remains disabled, leading to a bunch of failed test cases.

Anyway, cloud-init implements a native way to enable password
authentication, so we can just switch to that one and make
things both simpler and more robust.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
It comes with a newer kernel which implements a virtio-mem
driver that's compatible with the QEMU 8.2 device.

kubevirt/kubevirtci#1167

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
Specifically:

       QEMU        8.0.0 -> 8.2.0
    libvirt        9.5.0 -> 10.0.0
    SeaBIOS       1.16.1 -> 1.16.3
      EDKII     20230524 -> 20231122
      passt     20230818 -> 20231204
  virtiofsd        1.7.2 -> 1.1.10.1
  guestfs-tools   1.50.1 -> 1.51.6

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
@andreabolognani
Copy link
Contributor Author

Updated to pick up libvirt 10.0.0-7, which should contain fixes for all known issues. Unfortunately that build is not available from composes yet, so I'm fetching it straight from koji as a temporary workaround.

This is the build from

  https://kojihub.stream.centos.org/koji/buildinfo?buildID=60845

which is currently not available from actual CentOS Stream 9
composes. It contains fixes for

  https://issues.redhat.com/browse/RHEL-25995
  https://issues.redhat.com/browse/RHEL-30622

which we need to make some of the CI tests pass.

While we wait for it to available through the usual channels,
pick it up in a hacky way so that we can get ahead of things
by potentially uncovering other issues that might be lingering.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
To match the RPM update.

Signed-off-by: Andrea Bolognani <abologna@redhat.com>
@kubevirt-bot
Copy link
Contributor

kubevirt-bot commented Apr 23, 2024

@andreabolognani: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
pull-kubevirt-e2e-kind-1.27-vgpu 0efd263 link true /test pull-kubevirt-e2e-kind-1.27-vgpu
pull-kubevirt-e2e-windows2016 0efd263 link true /test pull-kubevirt-e2e-windows2016
pull-kubevirt-e2e-k8s-1.27-sig-operator 0efd263 link true /test pull-kubevirt-e2e-k8s-1.27-sig-operator
pull-kubevirt-e2e-k8s-1.28-sig-operator 0efd263 link true /test pull-kubevirt-e2e-k8s-1.28-sig-operator
pull-kubevirt-unit-test-arm64 edff17f link false /test pull-kubevirt-unit-test-arm64
pull-kubevirt-conformance-arm64 edff17f link false /test pull-kubevirt-conformance-arm64
pull-kubevirt-e2e-k8s-1.29-sig-compute edff17f link true /test pull-kubevirt-e2e-k8s-1.29-sig-compute

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@alicefr
Copy link
Member

alicefr commented Apr 24, 2024

@andreabolognani any updates on this? QEMU packages for the version 8.0.0-13 aren't available anymore from centos stream. Hence, make rpms fails

@andreabolognani
Copy link
Contributor Author

@alicefr see my last update. Things are progressing, albeit slowly. I know this is becoming increasingly problematic. There's not much I can do about it beyond what I'm already doing.

@alicefr
Copy link
Member

alicefr commented Apr 24, 2024

@alicefr see my last update. Things are progressing, albeit slowly. I know this is becoming increasingly problematic. There's not much I can do about it beyond what I'm already doing.

And using libvirt version 10.0.0-4 isn't an option?

@andreabolognani
Copy link
Contributor Author

10.0.0-4 contains two migration-related issues, so the CI jobs will understandably not pass. 10.0.0-7 addresses them.

@alicefr
Copy link
Member

alicefr commented Apr 24, 2024

10.0.0-4 contains two migration-related issues, so the CI jobs will understandably not pass. 10.0.0-7 addresses them.

Ok, sorry for the noise and thanks for the explanation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. release-note Denotes a PR that will be considered when it comes time to generate release notes. sig/buildsystem Denotes an issue or PR that relates to changes in the build system. sig/network size/XXL
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Arm64 -- qemu-kvm-device-usb-redirect package is missing on Arm64 platform