🌱 Add BMO e2e upgrade test #1451

mquhuy · 2023-11-20T08:01:26Z

Fixes #1430

mquhuy · 2023-11-20T09:58:41Z

/metal3-bmo-e2e-test

mquhuy · 2023-11-20T10:10:28Z

/metal3-bmo-e2e-test

test/e2e/config/ironic.yaml

test/e2e/config/fixture.yaml

test/e2e/upgrade_test.go

hack/ci-e2e.sh

mquhuy · 2023-11-23T11:10:18Z

/metal3-bmo-e2e-test

mquhuy · 2023-11-23T11:43:19Z

/override test-centos-e2e-integration-main
/override test-ubuntu-integration-main

metal3-io-bot · 2023-11-23T11:43:25Z

@mquhuy: mquhuy unauthorized: /override is restricted to Repo administrators, approvers in top level OWNERS file.

In response to this:

/override test-centos-e2e-integration-main
/override test-ubuntu-integration-main

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

mquhuy · 2023-11-23T12:53:17Z

/metal3-bmo-e2e-test

hack/ci-e2e.sh

lentzi90

Good progress!
I hope all my comments are not discouraging!

Makefile

hack/ci-e2e.sh

hack/e2e/ensure_kustomization_paths.sh

test/e2e/e2e_suite_test.go

test/e2e/upgrade_test.go

mquhuy · 2023-11-28T11:09:11Z

/metal3-bmo-e2e-test

lentzi90

This looks pretty good now!
Could you set up a pipeline for this upgrade job also? Then we could test it from this PR before we merge 🙂

test/e2e/config/fixture.yaml

hack/ci-e2e.sh

lentzi90 · 2024-01-15T09:01:09Z

test/e2e/config/ironic-upgrade.yaml

Not sure if it is worth it to have a separate file for this. Is it more than just the *DEPLOY_* variables that differ?
If it is just them I think we can just override them in ci-e2e.sh (e.g. export DEPLOY_BMO=false).

mquhuy · 2024-01-16T12:26:34Z

This looks pretty good now! Could you set up a pipeline for this upgrade job also? Then we could test it from this PR before we merge 🙂

Sure. I added the pipeline in these commits:
metal3-io/project-infra#617
https://gerrit.nordix.org/c/infra/cicd/+/20387/1/jjb/metal3/baremetal-operator/jjb-bmo-e2e.yaml

And I updated this PR with your suggested changes: the two jobs are now differentiated by GINKGO_FOCUS. If "upgrade" is in the focus list, it will be the only test, otherwise it will not be tested.

lentzi90 · 2024-01-17T06:44:39Z

hack/ci-e2e.sh

+  *)
+    echo "'upgrade' not found in env var GINKGO_FOCUS. Will NOT run upgrade tests."
+    export GINKGO_SKIP="upgrade ${GINKGO_SKIP:-}"
+    ;;


The less logic we have here the better. If someone wants to try to run upgrade together with the rest of the tests that is allowed. Maybe they have a configuration that works?

Suggested change

*)

echo "'upgrade' not found in env var GINKGO_FOCUS. Will NOT run upgrade tests."

export GINKGO_SKIP="upgrade ${GINKGO_SKIP:-}"

;;

lentzi90 · 2024-01-17T06:46:43Z

hack/ci-e2e.sh

@@ -142,6 +157,14 @@ echo "${IRONIC_PASSWORD}" > "${BMO_OVERLAY}/ironic-password"
 echo "${IRONIC_INSPECTOR_USERNAME}" > "${BMO_OVERLAY}/ironic-inspector-username"
 echo "${IRONIC_INSPECTOR_PASSWORD}" > "${BMO_OVERLAY}/ironic-inspector-password"

+if [[ "${GINKGO_FOCUS:-}" == "upgrade" ]]; then


Suggested change

if [[ "${GINKGO_FOCUS:-}" == "upgrade" ]]; then

if [[ "${GINKGO_FOCUS:-}" == *"upgrade"* ]]; then

I'm not sure it is really worth it to have the if statement here at all. It does not cost much to add the passwords regardless if we use them or not.

lentzi90 · 2024-01-17T07:48:09Z

hack/ci-e2e.sh

+    echo "'upgrade' found in env var GINKGO_FOCUS. Will ONLY run upgrade tests."
+    export GINKGO_FOCUS="upgrade"


We should not change the variable. If someone has a reason to set this to something more than just upgrade, we just accept it.

Suggested change

echo "'upgrade' found in env var GINKGO_FOCUS. Will ONLY run upgrade tests."

export GINKGO_FOCUS="upgrade"

echo "'upgrade' found in env var GINKGO_FOCUS. Will ONLY deploy 'upgrade from' versions of BMO and Ironic."

mquhuy · 2024-01-17T12:23:07Z

/metal3-bmo-e2e-test

mquhuy · 2024-01-17T12:35:47Z

/metal3-bmo-e2e-test

Signed-off-by: Huy Mai <huy.mai@est.tech>

mquhuy · 2024-01-18T06:05:58Z

/metal3-bmo-e2e-test

mquhuy · 2024-01-18T07:05:05Z

/metal3-bmo-e2e-test

mquhuy · 2024-01-18T07:22:44Z

/metal3-bmo-e2e-test

lentzi90 · 2024-01-18T07:37:19Z

/metal3-bmo-e2e-optional-test

mquhuy · 2024-01-18T07:44:24Z

/metal3-bmo-e2e-test

lentzi90

Awesome job @mquhuy !
Thanks for the patience through all those comments 😅
/lgtm

mquhuy · 2024-01-18T10:31:23Z

/cc @kashifest

kashifest

Few questions/comments in line

kashifest · 2024-01-18T10:39:31Z

config/overlays/e2e-release-0.4/ironic.env

@@ -0,0 +1,10 @@
+HTTP_PORT=6180


The name of the directory could be ironic-release-0.4 or e2e-ironic-release-0.4, otherwise having e2e- in one and fixture- in another seems a bit out of convention.

The name of these directories are to aligned with the equivalent from main: fixtures and e2e. I think your naming convention makes senses, but do you think we should change these original names (fixtures and e2e) as well? What about you @lentzi90 ?

I don't have a strong preference. It looks alright like this to me.
My reasoning when I named the folders e2e and fixture is that they are used for the e2e and fixture tests respectively. So I think it also makes sense then with e2e-release-04 and fixture-release-0.4

Ok. I dont have a strong opinion either. This is certainly not a blocker for the tests to go in.

/approve

kashifest · 2024-01-18T10:41:20Z

config/overlays/fixture-release-0.4/kustomization.yaml

@@ -0,0 +1,24 @@
+apiVersion: kustomize.config.k8s.io/v1beta1


Same comment, the name of the directory could be e2e-fixture-release-0.4 or we can keep it as it is and then add ironic- in the other directory name

My answer to this one would be the same as in the other kustomization, so let's continue the discussion there.

kashifest · 2024-01-18T10:51:21Z

hack/ci-e2e.sh

+case "${GINKGO_FOCUS:-}" in
+  *upgrade*)
+    export DEPLOY_IRONIC="false"
+    export DEPLOY_BMO="false"


I dont understand this, by default we dont deploy BMO even in upgrade test?

The upgrade test has its own logic to install all these components, and they follow the UPGRADE_DEPLOY_* configs instead. The reason for that is because we use a different BMO version for upgrade test than the rest of the suite.
In the fixtures setup, we use two separate clusters for this purpose, so we could run all tests in the same run, but in ci-e2e.sh, we have only 1 cluster, so if we run upgrade test, we don't install anything in the BeforeSuite and leave these installations to the test setup.

metal3-io-bot · 2024-01-18T15:06:22Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kashifest

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [kashifest]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

lentzi90 · 2024-01-19T06:27:18Z

/test-centos-e2e-integration-main
/test-ubuntu-integration-main

metal3-io-bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label Nov 20, 2023

mquhuy force-pushed the mquhuy/e2e-upgrade-test branch from 95c2149 to fa9dc42 Compare November 20, 2023 09:15

lentzi90 reviewed Nov 20, 2023

View reviewed changes

test/e2e/config/ironic.yaml Outdated Show resolved Hide resolved

test/e2e/config/fixture.yaml Outdated Show resolved Hide resolved

test/e2e/upgrade_test.go Show resolved Hide resolved

hack/ci-e2e.sh Outdated Show resolved Hide resolved

hack/ci-e2e.sh Outdated Show resolved Hide resolved

mquhuy force-pushed the mquhuy/e2e-upgrade-test branch 3 times, most recently from 545e205 to 216772a Compare November 22, 2023 13:01

metal3-io-bot added needs-rebase Indicates that a PR cannot be merged because it has merge conflicts with HEAD. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/L Denotes a PR that changes 100-499 lines, ignoring generated files. labels Nov 22, 2023

mquhuy force-pushed the mquhuy/e2e-upgrade-test branch 5 times, most recently from cd4830e to ad37ef2 Compare November 23, 2023 10:29

metal3-io-bot removed the needs-rebase Indicates that a PR cannot be merged because it has merge conflicts with HEAD. label Nov 23, 2023

mquhuy force-pushed the mquhuy/e2e-upgrade-test branch 2 times, most recently from d1215ff to 9336c5b Compare November 23, 2023 10:51

maxrantil reviewed Nov 23, 2023

View reviewed changes

hack/ci-e2e.sh Outdated Show resolved Hide resolved

hack/ci-e2e.sh Outdated Show resolved Hide resolved

lentzi90 reviewed Nov 23, 2023

View reviewed changes

mquhuy force-pushed the mquhuy/e2e-upgrade-test branch 3 times, most recently from b96c192 to e52d63d Compare November 28, 2023 11:04

mquhuy force-pushed the mquhuy/e2e-upgrade-test branch from e52d63d to 57d6593 Compare November 29, 2023 13:01

lentzi90 reviewed Jan 15, 2024

View reviewed changes

mquhuy force-pushed the mquhuy/e2e-upgrade-test branch 3 times, most recently from 437df13 to bb0d40e Compare January 16, 2024 09:23

lentzi90 reviewed Jan 17, 2024

View reviewed changes

mquhuy force-pushed the mquhuy/e2e-upgrade-test branch 2 times, most recently from f48d684 to 90e3930 Compare January 17, 2024 11:59

mquhuy force-pushed the mquhuy/e2e-upgrade-test branch from 90e3930 to bcd28f5 Compare January 18, 2024 05:58

Add bmo upgrade test

a252707

Signed-off-by: Huy Mai <huy.mai@est.tech>

mquhuy force-pushed the mquhuy/e2e-upgrade-test branch from bcd28f5 to a252707 Compare January 18, 2024 06:05

lentzi90 reviewed Jan 18, 2024

View reviewed changes

metal3-io-bot added the lgtm Indicates that a PR is ready to be merged. label Jan 18, 2024

metal3-io-bot requested a review from kashifest January 18, 2024 10:31

kashifest reviewed Jan 18, 2024

View reviewed changes

mquhuy requested a review from kashifest January 18, 2024 14:37

metal3-io-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jan 18, 2024

metal3-io-bot merged commit e6bb1d6 into metal3-io:main Jan 19, 2024
24 checks passed

metal3-io-bot deleted the mquhuy/e2e-upgrade-test branch January 19, 2024 07:40

mquhuy mentioned this pull request Feb 21, 2024

⚠️ Remove Inspector from all deployments scripts #1544

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

🌱 Add BMO e2e upgrade test #1451

🌱 Add BMO e2e upgrade test #1451

mquhuy commented Nov 20, 2023

mquhuy commented Nov 20, 2023

mquhuy commented Nov 20, 2023

mquhuy commented Nov 23, 2023

mquhuy commented Nov 23, 2023

metal3-io-bot commented Nov 23, 2023

mquhuy commented Nov 23, 2023

lentzi90 left a comment

mquhuy commented Nov 28, 2023

lentzi90 left a comment

lentzi90 Jan 15, 2024

mquhuy commented Jan 16, 2024

lentzi90 Jan 17, 2024

lentzi90 Jan 17, 2024

lentzi90 Jan 17, 2024

mquhuy commented Jan 17, 2024

mquhuy commented Jan 17, 2024

mquhuy commented Jan 18, 2024

mquhuy commented Jan 18, 2024

mquhuy commented Jan 18, 2024

lentzi90 commented Jan 18, 2024

mquhuy commented Jan 18, 2024

lentzi90 left a comment

mquhuy commented Jan 18, 2024

kashifest left a comment

kashifest Jan 18, 2024

mquhuy Jan 18, 2024

lentzi90 Jan 18, 2024

kashifest Jan 18, 2024

kashifest Jan 18, 2024

mquhuy Jan 18, 2024

kashifest Jan 18, 2024

mquhuy Jan 18, 2024

metal3-io-bot commented Jan 18, 2024

lentzi90 commented Jan 19, 2024

	if [[ "${GINKGO_FOCUS:-}" == "upgrade" ]]; then
	if [[ "${GINKGO_FOCUS:-}" == "upgrade" ]]; then

		echo "'upgrade' found in env var GINKGO_FOCUS. Will ONLY run upgrade tests."
		export GINKGO_FOCUS="upgrade"

🌱 Add BMO e2e upgrade test #1451

🌱 Add BMO e2e upgrade test #1451

Conversation

mquhuy commented Nov 20, 2023

mquhuy commented Nov 20, 2023

mquhuy commented Nov 20, 2023

mquhuy commented Nov 23, 2023

mquhuy commented Nov 23, 2023

metal3-io-bot commented Nov 23, 2023

mquhuy commented Nov 23, 2023

lentzi90 left a comment

Choose a reason for hiding this comment

mquhuy commented Nov 28, 2023

lentzi90 left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mquhuy commented Jan 16, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

mquhuy commented Jan 17, 2024

mquhuy commented Jan 17, 2024

mquhuy commented Jan 18, 2024

mquhuy commented Jan 18, 2024

mquhuy commented Jan 18, 2024

lentzi90 commented Jan 18, 2024

mquhuy commented Jan 18, 2024

lentzi90 left a comment

Choose a reason for hiding this comment

mquhuy commented Jan 18, 2024

kashifest left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

metal3-io-bot commented Jan 18, 2024

lentzi90 commented Jan 19, 2024