feat: build rocky linux 9.1 images #637

supershal · 2023-01-24T01:36:34Z

What problem does this PR solve?:

E2E tests for rocky linux 9.1

Which issue(s) does this PR fix?:

https://d2iq.atlassian.net/browse/D2IQ-94873

Special notes for your reviewer:

We had disabled yum version lock for RHEL 8 because of dependency errors with already installed packets.
https://github.com/mesosphere/konvoy-image-builder/blob/main/ansible/roles/setup_versionlock/tasks/redhat.yaml#L2-L5
We had fixed this issue by not upgrading already installed packages.
We will need to revisit this logic for RHEL 8 and see if we can add back this version lock for RHEL 8.
I will test and file separate PR for this.

Does this PR introduce a user-facing change?:

supershal · 2023-01-24T01:39:02Z

ansible/roles/containerd/tasks/redhat.yaml

@@ -1,6 +1,6 @@
 ---
 - name: remove versionlock for containerd
-  command: yum versionlock delete containerd.io
+  command: yum versionlock {{ '--disablerepo "*" --enablerepo offline' if offline_mode_enabled else '' }} delete containerd.io


yum versionlock commands try to download repo metadata from internet. Adding logic for preventing yum from updating repo metadata when using yum versionlock in air-gapped installations for "RHEL" ansible distributions.

Wow so that means the rockylinux already comes wit Containerd installed? 🤔

Discussed offline, the behavior for RockyLinux is different from CentoOS/RHEL yum and it tries to reach other repos when running yum versionlock.
But this is a good generic change that will prevent this bug in future RHEL releases too.

github-actions · 2023-01-24T01:45:53Z

File	Coverage
All files	`14%`	❌
cmd/konvoy-image-wrapper/cmd/wrapper.go	`8%`	❌
cmd/konvoy-image-wrapper/image/common.go	`0%`	❌
cmd/konvoy-image-wrapper/image/image_not_embedded.go	`0%`	❌
pkg/ansible/runner.go	`0%`	❌
pkg/app/artifacts.go	`0%`	❌
pkg/app/build.go	`0%`	❌
pkg/app/build_azure.go	`2%`	❌
pkg/app/build_gcp.go	`0%`	❌
pkg/app/config.go	`50%`	❌
pkg/app/errors.go	`0%`	❌
pkg/app/provision.go	`0%`	❌
pkg/app/root.go	`0%`	❌
pkg/app/utils.go	`7%`	❌
pkg/app/validate.go	`0%`	❌
pkg/appansible/io.go	`0%`	❌
pkg/appansible/playbook.go	`0%`	❌
pkg/azure/azure.go	`0%`	❌
pkg/logging/logger.go	`0%`	❌
pkg/packer/manifest.go	`0%`	❌
pkg/packer/packer.go	`0%`	❌
pkg/stringutil/rand.go	`0%`	❌
pkg/version/info.go	`8%`	❌

Minimum allowed coverage is 75%

Generated by 🐒 cobertura-action against cc4575c

faiq · 2023-01-24T04:31:09Z

make/ci.mk

+e2e.build.rocky-9.1-offline: rocky91-offline infra.aws.destroy
+
+e2e.build.rocky-9.1: rocky91
+
 e2e.build.rhel-8.6-nvidia: rhel86-nvidia



should we not do these any more?

I am keeping it in sync with other related targets for other OS. We can remove all related target at once when we deprecate them in separate PR.

ansible/group_vars/all/defaults.yaml

supershal · 2023-01-24T23:03:19Z

images/azure/rocky-91.yaml

+packer:
+  distribution: "rockylinux-9"                # Offer
+  distribution_version: "rockylinux-9"  # SKU
+  image_publisher: "erockyenterprisesoftwarefoundationinc1653071250513"


This is ID for the official free rocky linux image template. https://portal.azure.com/#view/Microsoft_Azure_Marketplace/GalleryItemDetailsBladeNopdl/id/erockyenterprisesoftwarefoundationinc1653071250513.rockylinux-9

supershal · 2023-01-24T23:34:56Z

Created TC configurations for building Rockylinux release AMI (fips, non-fips) , Azure image (non-fips)

dkoshkin · 2023-01-24T23:45:24Z

ansible/roles/containerd/tasks/redhat.yaml

@@ -1,6 +1,6 @@
 ---
 - name: remove versionlock for containerd
-  command: yum versionlock delete containerd.io
+  command: yum versionlock {{ '--disablerepo "*" --enablerepo offline' if offline_mode_enabled else '' }} delete containerd.io


Wow so that means the rockylinux already comes wit Containerd installed? 🤔

ansible/roles/kubeadm/tasks/redhat.yaml

dkoshkin · 2023-01-24T23:46:57Z

images/azure/rocky-91.yaml

+packer:
+  distribution: "rockylinux-9"                # Offer
+  distribution_version: "rockylinux-9"  # SKU
+  image_publisher: "erockyenterprisesoftwarefoundationinc1653071250513"


dkoshkin · 2023-01-24T23:47:22Z

make/images.mk

+rocky91-offline:
+	$(MAKE) aws-rocky-9.1_offline
+
+.PHONY: rocky91-fips-offline


Did we build FIPS offline packages? I do't think RockyLinux supports the correct 140-2 FIPS version only https://rockylinux.org/news/certifications-fips-2022-06-11/ 140-3.

FIPS offline packages for rocky. Link
I have still not run any FIPS validation tests

supershal · 2023-01-25T00:10:53Z

images/azure/rocky-91.yaml

  image_publisher: "erockyenterprisesoftwarefoundationinc1653071250513"
  image_version: "latest"
  ssh_username: "azureuser"
+  plan_image_sku: "rockylinux-9" # SKU


Azure plan information needed to use marketplace image.
Packer throwing following error when plan information was not provided.
Creating a virtual machine from Marketplace image or a custom image sourced from a Marketplace image requires Plan information in the request . TC log link

This thread has lots of useful information https://forums.rockylinux.org/t/azure-rocky-image-on-marketplace/5230/46

Thank you for looking it up

faiq · 2023-01-26T21:47:18Z

pkg/packer/manifests/azure/packer.pkr.hcl

@@ -469,7 +469,7 @@ build {
    vars_inline = {
      ARCH     = "amd64"
      OS       = lower(var.distribution)
-      PROVIDER = "amazon"
+      PROVIDER = "azure"


wow good find.

supershal · 2023-01-26T21:51:04Z

RHEL 8.6 NVIDIA tests are failing because kernel header rpm version 4.18.0-372.41.1.el8_6.x86_64 is not available yet. To unblock further work on rockylinux, I plan to merge this PR now.

github-actions bot added the feature label Jan 24, 2023

supershal commented Jan 24, 2023

View reviewed changes

supershal requested review from faiq, dkoshkin, a team and dlipovetsky and removed request for faiq January 24, 2023 01:39

supershal force-pushed the shalin/rocklylinux-9.1 branch from fcdb3ae to 32bb89f Compare January 24, 2023 04:08

faiq reviewed Jan 24, 2023

View reviewed changes

ansible/group_vars/all/defaults.yaml Outdated Show resolved Hide resolved

supershal force-pushed the shalin/rocklylinux-9.1 branch from 79fb2dd to be7ff3f Compare January 24, 2023 22:59

supershal commented Jan 24, 2023

View reviewed changes

supershal force-pushed the shalin/rocklylinux-9.1 branch from be7ff3f to 3bc7626 Compare January 24, 2023 23:11

dkoshkin reviewed Jan 24, 2023

View reviewed changes

supershal commented Jan 25, 2023

View reviewed changes

shalin patel added 6 commits January 24, 2023 16:38

ci: makefile targets for rocky linux 9.1

d0c4d78

feat: build rocky linux 9.1 offline image

3f66c94

ci: build targets for rocky linux 9.1

d9ea87f

add rockylinux 9.1 nvidia and nvidia-offline in exclude list

e45b19e

create kib images for rocky 9.1 in azure

1a40cfa

add azure plan information to use image from marketplace

0a186aa

supershal force-pushed the shalin/rocklylinux-9.1 branch from 028a8bf to 0a186aa Compare January 25, 2023 00:38

add goss tests for rocky linux

df662e5

dkoshkin approved these changes Jan 25, 2023

View reviewed changes

supershal changed the title ~~feat: build rocky linux 9.1 images~~ feat: build rocky linux 9.1 images Jan 25, 2023

github-actions bot added feature and removed feature labels Jan 25, 2023

ci: create symlink from rhel goss test to rocky goss tests

cc4575c

supershal changed the title ~~feat: build rocky linux 9.1 images~~ feat: build rocky linux 9.1 images Jan 26, 2023

github-actions bot added feature and removed feature labels Jan 26, 2023

faiq reviewed Jan 26, 2023

View reviewed changes

faiq approved these changes Jan 26, 2023

View reviewed changes

supershal merged commit 579e620 into main Jan 26, 2023

supershal deleted the shalin/rocklylinux-9.1 branch January 26, 2023 23:15

mesosphere-ci mentioned this pull request Jan 26, 2023

chore(main): release 2.0.0 #616

Merged

mesosphere-ci mentioned this pull request Feb 8, 2023

chore(main): release 2.0.0 #660

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: build rocky linux 9.1 images #637

feat: build rocky linux 9.1 images #637

supershal commented Jan 24, 2023

supershal Jan 24, 2023 •

edited

Loading

dkoshkin Jan 24, 2023

dkoshkin Jan 25, 2023

github-actions bot commented Jan 24, 2023 •

edited

Loading

faiq Jan 24, 2023

supershal Jan 24, 2023

supershal Jan 24, 2023

dkoshkin Jan 24, 2023

supershal commented Jan 24, 2023

dkoshkin Jan 24, 2023

dkoshkin Jan 24, 2023

dkoshkin Jan 24, 2023

supershal Jan 25, 2023

supershal Jan 25, 2023 •

edited

Loading

dkoshkin Jan 25, 2023

supershal Jan 25, 2023

faiq Jan 26, 2023

supershal commented Jan 26, 2023

feat: build rocky linux 9.1 images #637

feat: build rocky linux 9.1 images #637

Conversation

supershal commented Jan 24, 2023

supershal Jan 24, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

github-actions bot commented Jan 24, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

supershal commented Jan 24, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

supershal Jan 25, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

supershal commented Jan 26, 2023

supershal Jan 24, 2023 •

edited

Loading

github-actions bot commented Jan 24, 2023 •

edited

Loading

supershal Jan 25, 2023 •

edited

Loading