Use centos:stream9 as a base image, cleanup unused code #1983

maya-r · 2021-10-11T20:38:22Z

What this PR does / why we need it:
KubeVirt has done similar with centos stream8.
Fedora has some downsides: frequent releases, and shorter end of life.
CentOS stream will mean fewer base image changes, and it means we use a base image which has vulnerabilities reported to it, as requested on kubevirt-dev.

Special notes for your reviewer:
Blocked on #1982 which changes the builder image.
Biggest change was switching away from what might be a self-made Fedora RPM for golang-govmomi to downloading the releases and untarring with the entrypoint.sh file.
VDI test now uses a pre-generated tinyCore.vdi as the qemu-img vdi support is read-only.

Release note:

Use centos:stream9 as a base image for our containers.

brybacki · 2021-10-12T08:03:03Z

You say "Removes VDI importer code"
Does it mean we lose part of functionality? Is it used by some migration tools or end users?

maya-r · 2021-10-12T08:11:31Z

Looking again, in the original PR by @visheshtanksale (#1512) he added VDI support, so it might be used.
I wonder if I can request to change the RPM in centos.

maya-r · 2021-10-12T15:38:07Z

RFE for adding qemu-img convert vdi support on CentOS

visheshtanksale · 2021-10-12T22:08:25Z

Looking again, in the original PR by @visheshtanksale (#1512) he added VDI support, so it might be used.
I wonder if I can request to change the RPM in centos.

Yes some of our key functionality will be lost if we merge this without support for VDI.

maya-r · 2021-10-13T12:17:04Z

/hold
waiting for VDI support in centos stream packages

jsquyres · 2021-10-18T18:31:03Z

Per https://groups.google.com/g/kubevirt-dev/c/Vu89kytp2mQ, I'm interested in having both Kubevirt and CDI move away from Fedora. Is there a time estimate on when Centos 8 stream will gain VDI support so that this PR can progress?

jsquyres · 2021-10-20T18:26:35Z

Gentle ping. I see in #1982, you cited https://bugzilla.redhat.com/show_bug.cgi?id=2013331. It doesn't look like there has been any activity on that bug yet. Do you have any kind of estimate on if/when that will be acted on, which would unblock both #1982 and this PR?

maya-r · 2021-10-26T14:16:30Z

@jsquyres No idea about timelines. The bug report requesting VDI support seems to get a positive response but unfortunately all the replies are set to private.

jsquyres · 2021-10-26T15:05:12Z

@maya-r That's somewhat of a bummer. Is it a sensitive topic?

I notice that the issue is currently marked as RHEL9.

alicefr · 2021-12-06T08:10:00Z

cmd/cdi-importer/BUILD.bazel

            "@nbdkit-server-aarch64//file",
            "@nbdkit-basic-filters-aarch64//file",
+            #"@nbdkit-vddk-plugin-aarch64//file",


Don't we need the vddk plugin?

The plugin doesn't seem to be built for aarch64 - I'm not sure why that is the case.
(I didn't notice I'm the one introducing this change, I thought it's an existing one)

Worth noting that vmware aarch64 support is in tech preview and not really likely to be in heavy use

maya-r · 2021-12-07T06:04:08Z

/retest

maya-r · 2021-12-07T06:08:25Z

looks like the push builder job is a periodic that hasn't run yet

maya-r · 2021-12-08T09:55:41Z

Oops, I'm going to need kubevirt/project-infra#1796 to update the builder.

maya-r · 2021-12-15T11:45:49Z

Blocked as the builder push job is having connectivity issues. Daniel Hiller tried re-running it but it failed again.

jsquyres · 2022-03-03T21:36:45Z

Is there any progress on this PR, perchance (and #2087)? It would be really, really great to ditch Fedora and switch to Centos stream...

They expire faster than we can update checksums, this is unfortunate but perhaps they will soon publish images at a lower rate allowing us to keep up. Signed-off-by: Maya Rashish <mrashish@redhat.com>

Signed-off-by: Maya Rashish <mrashish@redhat.com>

Noticed due to: duplicated checksum but no problem in testsuite, lack of aarch64 equivalent. Signed-off-by: Maya Rashish <mrashish@redhat.com>

Signed-off-by: Maya Rashish <mrashish@redhat.com>

Needed after kubevirt#2174 Signed-off-by: Maya Rashish <mrashish@redhat.com>

The previous version we were using can't be fetched any more Signed-off-by: Maya Rashish <mrashish@redhat.com>

Now updating the dependencies can be done by running `make rpm-deps` and committing the change, like kubevirt. This creates a small complication that we need to run update-ca-trust to trust root CAs. Do this on the pod, using the entrypoint to do so. Use a single image with all the dependencies for the test tools, we don't benefit from making them minimal and it saved some trouble in the conversion. Signed-off-by: Maya Rashish <mrashish@redhat.com>

awels · 2022-04-12T16:57:01Z

Looks like all the image io tests are failing on something.

Run update-ca-trust and update-crypto-policies before running ovirt-imageio, to stop error messages. Signed-off-by: Maya Rashish <mrashish@redhat.com>

maya-r · 2022-04-14T09:09:47Z

/retest-required

kubevirt-bot · 2022-04-14T11:42:04Z

@maya-r: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
pull-containerized-data-importer-e2e-k8s-1.19-hpp	`4eca286`	link	true	`/test pull-containerized-data-importer-e2e-k8s-1.19-hpp`
pull-containerized-data-importer-e2e-k8s-1.20-hpp-destructive	`4eca286`	link	true	`/test pull-containerized-data-importer-e2e-k8s-1.20-hpp-destructive`
pull-containerized-data-importer-e2e-k8s-1.20-hpp-istio	`4eca286`	link	true	`/test pull-containerized-data-importer-e2e-k8s-1.20-hpp-istio`
pull-containerized-data-importer-e2e-k8s-1.20-upg	`4eca286`	link	true	`/test pull-containerized-data-importer-e2e-k8s-1.20-upg`
pull-containerized-data-importer-e2e-k8s-1.20-hpp	`4eca286`	link	true	`/test pull-containerized-data-importer-e2e-k8s-1.20-hpp`
pull-containerized-data-importer-e2e-k8s-1.20-nfs	`4eca286`	link	true	`/test pull-containerized-data-importer-e2e-k8s-1.20-nfs`
pull-containerized-data-importer-e2e-k8s-1.19-ceph	`4eca286`	link	true	`/test pull-containerized-data-importer-e2e-k8s-1.19-ceph`

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

mhenriks · 2022-04-14T19:24:59Z

/retest

maya-r · 2022-04-18T11:02:08Z

all green @mhenriks

awels

Looks good just a question about the arm base image not being set in a few places

awels · 2022-04-20T12:30:26Z

cmd/cdi-apiserver/BUILD.bazel

-        "@io_bazel_rules_go//go/platform:linux_arm64": "@fedora-aarch64//image",
-        "//conditions:default": "@fedora//image",
-    }),
+    base = "//:centos_base",


Don't we also need the arm version here?

centos_base is a different image if the platform is arm64.
the cross build works, I pushed the binaries and when inspecting one by hand it seems to be made up of arm64 binaries and executables.

awels · 2022-04-20T12:30:47Z

cmd/cdi-cloner/BUILD.bazel

-        "@io_bazel_rules_go//go/platform:linux_arm64": "@fedora-aarch64//image",
-        "//conditions:default": "@fedora//image",
-    }),
+    base = "//:centos_base",


Same we need arm version here.

awels · 2022-04-20T12:30:59Z

cmd/cdi-controller/BUILD.bazel

-        "@io_bazel_rules_go//go/platform:linux_arm64": "@fedora-aarch64//image",
-        "//conditions:default": "@fedora//image",
-    }),
+    base = "//:centos_base",


arm version?

awels · 2022-04-20T14:15:39Z

/approve

kubevirt-bot · 2022-04-20T14:15:48Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: awels

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [awels]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

awels · 2022-04-20T14:24:14Z

/lgtm

Forgotten in kubevirt#1983, needed to run `make rpm-deps` Signed-off-by: Maya Rashish <mrashish@redhat.com>

Forgotten in #1983, needed to run `make rpm-deps` Signed-off-by: Maya Rashish <mrashish@redhat.com>

kubevirt-bot added release-note Denotes a PR that will be considered when it comes time to generate release notes. dco-signoff: yes Indicates the PR's author has DCO signed all their commits. size/XXL labels Oct 11, 2021

kubevirt-bot requested review from aglitke and awels October 11, 2021 20:39

kubevirt-bot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Oct 13, 2021

kubevirt-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Oct 27, 2021

maya-r force-pushed the centos8-stream-part2 branch from 4eca286 to 5817274 Compare December 5, 2021 07:52

kubevirt-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Dec 5, 2021

maya-r changed the title ~~Use centos:stream8 as a base image, cleanup unused code~~ Use centos:stream9 as a base image, cleanup unused code Dec 5, 2021

alicefr reviewed Dec 6, 2021

View reviewed changes

kubevirt-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 7, 2022

maya-r force-pushed the centos8-stream-part2 branch from 5817274 to 94730f8 Compare January 12, 2022 15:24

kubevirt-bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 12, 2022

kubevirt-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Jan 21, 2022

awels mentioned this pull request Feb 14, 2022

Switch to CentOS stream9 builder #2087

Merged

maya-r added 11 commits April 12, 2022 17:44

Avoid specifying checksum for CentOS images.

bfc45fc

They expire faster than we can update checksums, this is unfortunate but perhaps they will soon publish images at a lower rate allowing us to keep up. Signed-off-by: Maya Rashish <mrashish@redhat.com>

Bump number of open file limit to avoid bazel crashes

ff26c2b

Signed-off-by: Maya Rashish <mrashish@redhat.com>

Update builder to include kubevirt#2087, builder based on centos stream9

49a3d61

Signed-off-by: Maya Rashish <mrashish@redhat.com>

Update checksums that seem wrong

7714ed2

Signed-off-by: Maya Rashish <mrashish@redhat.com>

Update ovirt links: old ones were removed

424b3d2

Signed-off-by: Maya Rashish <mrashish@redhat.com>

Remove unused RPMs

1bb4ac4

Noticed due to: duplicated checksum but no problem in testsuite, lack of aarch64 equivalent. Signed-off-by: Maya Rashish <mrashish@redhat.com>

Put nbdkit-vddk-plugin back for amd64.

709f599

Signed-off-by: Maya Rashish <mrashish@redhat.com>

Use quay.io instead of dockerhub.

e474b36

Signed-off-by: Maya Rashish <mrashish@redhat.com>

Install util-linux-core for /usr/sbin/blockdev

f06bfdf

Needed after kubevirt#2174 Signed-off-by: Maya Rashish <mrashish@redhat.com>

Update nbdkit/libnbd/nginx/ovirt versions to the latest

a7c787a

The previous version we were using can't be fetched any more Signed-off-by: Maya Rashish <mrashish@redhat.com>

Fixup imageio test container

a55e82a

Run update-ca-trust and update-crypto-policies before running ovirt-imageio, to stop error messages. Signed-off-by: Maya Rashish <mrashish@redhat.com>

maya-r force-pushed the centos8-stream-part2 branch from 674b452 to a55e82a Compare April 13, 2022 12:25

awels reviewed Apr 20, 2022

View reviewed changes

kubevirt-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Apr 20, 2022

kubevirt-bot assigned awels Apr 20, 2022

kubevirt-bot added the lgtm Indicates that a PR is ready to be merged. label Apr 20, 2022

kubevirt-bot merged commit 02e70a5 into kubevirt:main Apr 20, 2022

maya-r added a commit to maya-r/containerized-data-importer that referenced this pull request May 14, 2022

Add repo.yaml

075cf21

Forgotten in kubevirt#1983, needed to run `make rpm-deps` Signed-off-by: Maya Rashish <mrashish@redhat.com>

maya-r added a commit to maya-r/containerized-data-importer that referenced this pull request May 14, 2022

Add missing file repo.yaml

4b66986

Forgotten in kubevirt#1983, needed to run `make rpm-deps` Signed-off-by: Maya Rashish <mrashish@redhat.com>

maya-r mentioned this pull request May 14, 2022

Add missing file repo.yaml #2275

Merged

kubevirt-bot pushed a commit that referenced this pull request May 15, 2022

Add missing file repo.yaml (#2275)

0f2277a

Forgotten in #1983, needed to run `make rpm-deps` Signed-off-by: Maya Rashish <mrashish@redhat.com>

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Use centos:stream9 as a base image, cleanup unused code #1983

Use centos:stream9 as a base image, cleanup unused code #1983

maya-r commented Oct 11, 2021 •

edited

brybacki commented Oct 12, 2021

maya-r commented Oct 12, 2021

maya-r commented Oct 12, 2021

visheshtanksale commented Oct 12, 2021

maya-r commented Oct 13, 2021

jsquyres commented Oct 18, 2021

jsquyres commented Oct 20, 2021

maya-r commented Oct 26, 2021 •

edited

jsquyres commented Oct 26, 2021

alicefr Dec 6, 2021

maya-r Dec 6, 2021

maya-r Dec 7, 2021

maya-r commented Dec 7, 2021

maya-r commented Dec 7, 2021

maya-r commented Dec 8, 2021

maya-r commented Dec 15, 2021

jsquyres commented Mar 3, 2022 •

edited

awels commented Apr 12, 2022

maya-r commented Apr 14, 2022

kubevirt-bot commented Apr 14, 2022 •

edited

mhenriks commented Apr 14, 2022

maya-r commented Apr 18, 2022

awels left a comment

awels Apr 20, 2022

maya-r Apr 20, 2022

awels Apr 20, 2022

awels Apr 20, 2022

awels Apr 20, 2022

awels commented Apr 20, 2022

kubevirt-bot commented Apr 20, 2022

awels commented Apr 20, 2022

Use centos:stream9 as a base image, cleanup unused code #1983

Use centos:stream9 as a base image, cleanup unused code #1983

Conversation

maya-r commented Oct 11, 2021 • edited

brybacki commented Oct 12, 2021

maya-r commented Oct 12, 2021

maya-r commented Oct 12, 2021

visheshtanksale commented Oct 12, 2021

maya-r commented Oct 13, 2021

jsquyres commented Oct 18, 2021

jsquyres commented Oct 20, 2021

maya-r commented Oct 26, 2021 • edited

jsquyres commented Oct 26, 2021

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

maya-r commented Dec 7, 2021

maya-r commented Dec 7, 2021

maya-r commented Dec 8, 2021

maya-r commented Dec 15, 2021

jsquyres commented Mar 3, 2022 • edited

awels commented Apr 12, 2022

maya-r commented Apr 14, 2022

kubevirt-bot commented Apr 14, 2022 • edited

mhenriks commented Apr 14, 2022

maya-r commented Apr 18, 2022

awels left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

awels commented Apr 20, 2022

kubevirt-bot commented Apr 20, 2022

awels commented Apr 20, 2022

maya-r commented Oct 11, 2021 •

edited

maya-r commented Oct 26, 2021 •

edited

jsquyres commented Mar 3, 2022 •

edited

kubevirt-bot commented Apr 14, 2022 •

edited