🌱 Replace uses of kubectl and cmctl binaries in e2e #1440

mquhuy · 2023-11-10T09:51:19Z

Which issue(s) this PR fixes (optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close the issue(s) when PR gets merged):
Fixes #1373

I chose different implementations from what was suggested in #1373:

Instead of using krusty package, I copied the kustomize build workflow used by capi. This is only for simplicity, and to be aligned with CAPI. The krusty package seemed to be not too complicated, but imo the simplier, the better.
Changed (back) to krusty after discussion with @lentzi90.
The cert-manager workflow from CAPI takes into control many of CAPI internal data structures, therefore is not easy to replicate. For our usecase, I think simply download the cert-manager manifests, apply and ~~wait for all deployments to be ready~~ perform a dry-run of self-signed issuer&certificate creation on cert-manager webhook should be enough.

I'm fully open to all suggestion/negotiation.

mquhuy · 2023-11-10T10:42:07Z

/test-centos-e2e-integration-main
/test-ubuntu-integration-main

test/e2e/common.go

test/e2e/cert_manager.go

test/e2e/e2e_suite_test.go

hack/ci-e2e.sh

mquhuy · 2023-11-13T09:07:31Z

/test-centos-e2e-integration-main
/test-ubuntu-integration-main

mquhuy · 2023-11-13T10:01:57Z

/test-centos-e2e-integration-main
/test-ubuntu-integration-main

mquhuy · 2023-11-13T10:53:43Z

/test-centos-e2e-integration-main
/test-ubuntu-integration-main

mquhuy · 2023-11-13T13:32:09Z

/cc @kashifest @Rozzii @tuminoid @maxrantil @mboukhalfa

lentzi90

Great job! Just some minor comments

lentzi90 · 2023-11-13T13:51:12Z

test/go.mod

@@ -3,6 +3,7 @@ module github.com/metal3-io/baremetal-operator/test
 go 1.20

 require (
+	github.com/cert-manager/cert-manager v1.10.0


The version seems strange considering that we have v1.13 in the e2e config.

Yes. Basically using newer versions would trigger upgrades in many of the dependencies, which'd make everything a mess :-s 1.10 is the newest one I could find that didn't trigger too many breaks.

Ok I see. We can take this for now probably, but at some point we will need to bump it

lentzi90 · 2023-11-13T13:53:52Z

test/e2e/e2e_suite_test.go

+		manifest, err := buildKustomizeManifest(kustomization)
+		Expect(err).NotTo(HaveOccurred())
+		err = clusterProxy.Apply(ctx, manifest)
+		Expect(err).NotTo(HaveOccurred())


lentzi90 · 2023-11-13T13:55:32Z

test/e2e/e2e_suite_test.go

+			Eventually(func() bool {
+				err := checkCertManagerWebhook(ctx, clusterProxy)
+				return err == nil
+			}, e2eConfig.GetIntervals("default", "wait-available")...).Should(BeTrue())


I may be missing something, but is it not possible to just return the error directly and way for Succeed instead?

Suggested change

Eventually(func() bool {

err := checkCertManagerWebhook(ctx, clusterProxy)

return err == nil

}, e2eConfig.GetIntervals("default", "wait-available")...).Should(BeTrue())

Eventually(func() err {

return checkCertManagerWebhook(ctx, clusterProxy)

}, e2eConfig.GetIntervals("default", "wait-available")...).Should(Succeed())

Thanks a lot! That's the way to go 😅 I'm still pretty not familiar yet with Ginkgo. Initially I waited for BeNil(), thinking because the function returns an error. It always timed out 😅

lentzi90 · 2023-11-13T13:57:05Z

This is the test to trigger for e2e. The other tests are not relevant 😅
/metal3-bmo-e2e-test

tuminoid

LGTM

mquhuy · 2023-11-13T15:28:43Z

This is the test to trigger for e2e. The other tests are not relevant 😅 /metal3-bmo-e2e-test

Thanks. It seems that kubectl still needs to be there 😅

mquhuy · 2023-11-13T15:34:10Z

/metal3-bmo-e2e-test

lentzi90 · 2023-11-14T06:18:04Z

It seems that kubectl still needs to be there 😅

Any idea why?

mquhuy · 2023-11-14T06:27:56Z

It seems that kubectl still needs to be there 😅

Any idea why?

It's because CAPI test framework ClusterProxy.Apply() uses exec.KubectlApply, which uses kubectl command.

I can make a separate apply function to get rid of the kubectl, but I'm wondering if we should stick to the test framework? Or maybe we can make that contribution to CAPI instead?

lentzi90 · 2023-11-14T06:30:59Z

I see! It is fine like this I would say. No need to make it unnecessarily complicated 🙂

lentzi90 · 2023-11-14T06:40:36Z

test/e2e/config/ironic.yaml

@@ -27,6 +27,7 @@ variables:
  BOOT_MAC_ADDRESS: "00:60:2f:31:81:01"
  IMAGE_URL: "http://192.168.222.1/cirros-0.6.2-x86_64-disk.img"
  IMAGE_CHECKSUM: "c8fc807773e5354afe61636071771906"
+  CERT_MANAGER_VERSION: "v1.13.0"


This needs to be added in fixture.yaml also!
(We will have automated tests to check the fixture tests when #1437 is merged.)

lentzi90

/lgtm

lentzi90 · 2023-11-14T06:57:09Z

/metal3-bmo-e2e-test

Rozzii · 2023-11-14T07:11:36Z

/test-centos-e2e-integration-main
/test-ubuntu-integration-main
/lgtm

metal3-io-bot · 2023-11-14T07:11:41Z

@Rozzii: adding LGTM is restricted to approvers and reviewers in OWNERS files.

In response to this:

/test-centos-e2e-integration-main
/test-ubuntu-integration-main
/lgtm

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

mquhuy · 2023-11-14T07:41:08Z

/cc @honza

kashifest

/approve

metal3-io-bot · 2023-11-14T07:47:41Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kashifest, Rozzii

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [kashifest]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

metal3-io-bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Nov 10, 2023

mquhuy force-pushed the mquhuy/replace-os-Exec-e2e branch 4 times, most recently from c5f3a3a to fd6eed1 Compare November 10, 2023 10:35

lentzi90 reviewed Nov 10, 2023

View reviewed changes

test/e2e/common.go Outdated Show resolved Hide resolved

test/e2e/cert_manager.go Show resolved Hide resolved

test/e2e/cert_manager.go Show resolved Hide resolved

test/e2e/e2e_suite_test.go Outdated Show resolved Hide resolved

hack/ci-e2e.sh Show resolved Hide resolved

mquhuy force-pushed the mquhuy/replace-os-Exec-e2e branch from fd6eed1 to bc4ebaf Compare November 13, 2023 09:00

mquhuy requested a review from lentzi90 November 13, 2023 09:07

mquhuy force-pushed the mquhuy/replace-os-Exec-e2e branch from bc4ebaf to bf0a5e2 Compare November 13, 2023 10:01

mquhuy force-pushed the mquhuy/replace-os-Exec-e2e branch 2 times, most recently from 553ac04 to 82a585e Compare November 13, 2023 10:33

metal3-io-bot requested review from kashifest, maxrantil, mboukhalfa, Rozzii and tuminoid November 13, 2023 13:32

lentzi90 reviewed Nov 13, 2023

View reviewed changes

tuminoid reviewed Nov 13, 2023

View reviewed changes

mquhuy force-pushed the mquhuy/replace-os-Exec-e2e branch from 82a585e to a5d2b08 Compare November 13, 2023 15:34

mquhuy requested a review from lentzi90 November 14, 2023 05:46

mquhuy force-pushed the mquhuy/replace-os-Exec-e2e branch 2 times, most recently from 126a20a to 2aa51bc Compare November 14, 2023 05:56

lentzi90 reviewed Nov 14, 2023

View reviewed changes

Replace uses of kubectl and cmctl binaries in e2e

63a8382

mquhuy force-pushed the mquhuy/replace-os-Exec-e2e branch from 2aa51bc to 63a8382 Compare November 14, 2023 06:52

lentzi90 reviewed Nov 14, 2023

View reviewed changes

metal3-io-bot added the lgtm Indicates that a PR is ready to be merged. label Nov 14, 2023

Rozzii approved these changes Nov 14, 2023

View reviewed changes

metal3-io-bot requested a review from honza November 14, 2023 07:41

kashifest reviewed Nov 14, 2023

View reviewed changes

metal3-io-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 14, 2023

metal3-io-bot merged commit 16c7f1f into metal3-io:main Nov 14, 2023
14 of 15 checks passed

metal3-io-bot deleted the mquhuy/replace-os-Exec-e2e branch November 14, 2023 08:12

mquhuy mentioned this pull request Nov 14, 2023

🌱 Add github workflow for e2e fixture tests #1437

Merged

Sunnatillo changed the title ~~✨ Replace uses of kubectl and cmctl binaries in e2e~~ 🌱 Replace uses of kubectl and cmctl binaries in e2e Jan 8, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🌱 Replace uses of kubectl and cmctl binaries in e2e #1440

🌱 Replace uses of kubectl and cmctl binaries in e2e #1440

mquhuy commented Nov 10, 2023 •

edited

mquhuy commented Nov 10, 2023

mquhuy commented Nov 13, 2023

mquhuy commented Nov 13, 2023

mquhuy commented Nov 13, 2023

mquhuy commented Nov 13, 2023

lentzi90 left a comment

lentzi90 Nov 13, 2023

mquhuy Nov 13, 2023

lentzi90 Nov 14, 2023

lentzi90 Nov 13, 2023

lentzi90 Nov 13, 2023

mquhuy Nov 13, 2023 •

edited

lentzi90 commented Nov 13, 2023

tuminoid left a comment

mquhuy commented Nov 13, 2023

mquhuy commented Nov 13, 2023

lentzi90 commented Nov 14, 2023

mquhuy commented Nov 14, 2023 •

edited

lentzi90 commented Nov 14, 2023

lentzi90 Nov 14, 2023

mquhuy Nov 14, 2023

lentzi90 left a comment

lentzi90 commented Nov 14, 2023

Rozzii commented Nov 14, 2023

metal3-io-bot commented Nov 14, 2023

mquhuy commented Nov 14, 2023

kashifest left a comment

metal3-io-bot commented Nov 14, 2023

🌱 Replace uses of kubectl and cmctl binaries in e2e #1440

🌱 Replace uses of kubectl and cmctl binaries in e2e #1440

Conversation

mquhuy commented Nov 10, 2023 • edited

mquhuy commented Nov 10, 2023

mquhuy commented Nov 13, 2023

mquhuy commented Nov 13, 2023

mquhuy commented Nov 13, 2023

mquhuy commented Nov 13, 2023

lentzi90 left a comment

Choose a reason for hiding this comment

lentzi90 Nov 13, 2023

Choose a reason for hiding this comment

mquhuy Nov 13, 2023

Choose a reason for hiding this comment

lentzi90 Nov 14, 2023

Choose a reason for hiding this comment

lentzi90 Nov 13, 2023

Choose a reason for hiding this comment

lentzi90 Nov 13, 2023

Choose a reason for hiding this comment

mquhuy Nov 13, 2023 • edited

Choose a reason for hiding this comment

lentzi90 commented Nov 13, 2023

tuminoid left a comment

Choose a reason for hiding this comment

mquhuy commented Nov 13, 2023

mquhuy commented Nov 13, 2023

lentzi90 commented Nov 14, 2023

mquhuy commented Nov 14, 2023 • edited

lentzi90 commented Nov 14, 2023

lentzi90 Nov 14, 2023

Choose a reason for hiding this comment

mquhuy Nov 14, 2023

Choose a reason for hiding this comment

lentzi90 left a comment

Choose a reason for hiding this comment

lentzi90 commented Nov 14, 2023

Rozzii commented Nov 14, 2023

metal3-io-bot commented Nov 14, 2023

mquhuy commented Nov 14, 2023

kashifest left a comment

Choose a reason for hiding this comment

metal3-io-bot commented Nov 14, 2023

mquhuy commented Nov 10, 2023 •

edited

mquhuy Nov 13, 2023 •

edited

mquhuy commented Nov 14, 2023 •

edited