splitting e2e tests between bundle and operator by blublinsky · Pull Request #1548 · openshift/lightspeed-operator

blublinsky · 2026-04-22T15:28:34Z

Description

Summary
Consolidates Konflux integration Tekton Pipeline YAML into one file per test family (operator e2e, operator upgrade e2e, service integration) with pipeline parameters for OpenShift minor line (openshift-version-prefix, artifact-oci-tag-prefix where needed, test-name, namespace). Removes 13 per-version pipeline files and the stale e2e-tests.yaml scenario that pointed at a non-existent pipeline.

Extracts bundle install logic into hack/install/ (install-operator-bundle.sh, _lib.sh) and reuses it from the merged pipelines and rapidast-scan.yaml (shallow clone at ols-bundle snapshot revision, then script).

Adds konflux-integration-test-scenarios.yaml with 12 IntegrationTestScenario resources (v1beta2) targeting the merged pipelines and passing the correct parameters per minor (4.16–4.19).

Upgrade pipeline: installs baseline bundle, then snapshot bundle using PRE_BUNDLE_IMAGE + BUNDLE_IMAGE and a single install-operator-bundle.sh invocation (matches script contract and make test-upgrade).

config/default/deployment-patch.yaml: intentionally unchanged (keeps REPLACE*_ placeholders; no pinned digests in this PR).

Next steps (post-merge / follow-ups)
Konflux operations

Remove or rename legacy IntegrationTestScenario objects if names conflict with the new CRs (operator-e2e-tests-*, etc.).
Update any required integration test or release policy references that still point at ols-e2e-tests-416 or the old pipeline path.
Confirm test-name semantics

Operator e2e and service integration scenarios both use ols-e2e-tests-4.xx as the test-name param while metadata.name differs. Verify Konflux/reporting does not require globally unique test-name values; adjust if needed.
Git resolver revision

Scenarios pin revision: main. If you want PRs to exercise pipeline YAML on the PR branch, adopt whatever revision substitution pattern your tenant supports (e.g. dynamic revision), or document that pipeline changes apply after merge to main.
Optional: IDMS parity

rapidast-scan.yaml sets IMAGE_DIGEST_MIRROR_SET_URL; merged operator e2e install does not. If ephemeral clusters show pull errors for registry.redhat.io content, export the same URL in the operator e2e install step.
Security / hygiene

Consider dropping cat $KUBECONFIG (or redacting) from Tekton step logs in a follow-up PR.
Direct install path (not in this PR)

install-operator-direct.sh exists but is not wired into a parallel Konflux pipeline or scenario.
Later: add a pipeline/scenario (or parameterized branch), SKIP_IDMS / Hypershift behavior as needed, and document when to use direct vs bundle.
Timeouts

If integration runs hit default timeouts, add Konflux IntegrationTestScenario annotations for pipeline/tasks/finally timeouts per Konflux docs.
Developer ergonomics

Type of change

Related Tickets & Documents

Related Issue #
https://redhat.atlassian.net/browse/OLS-2857
Closes #

Checklist before requesting a review

I have performed a self-review of my code.
PR has passed all pre-merge test jobs.
If it is a core feature, I have added thorough tests.

Testing

Please provide detailed steps to perform tests related to this code change.
How were the fix/results from this change verified? Please provide relevant screenshots or results.

openshift-ci · 2026-04-22T15:28:50Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign joshuawilson for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

blublinsky · 2026-04-23T14:48:36Z

/retest

JoaoFula · 2026-04-30T06:21:39Z

+              git remote add origin https://github.com/openshift/lightspeed-operator.git
+              git fetch --depth 1 origin "${COMMIT}"
+              git checkout -q FETCH_HEAD
+              ./.tekton/integration-tests/scripts/run-konflux-operator-install.sh "$(params.install-mode)"


Any reason for us to install the operator in the service tests considering we already have that logic in place in the service tests?

ols-install puts the operator from this SNAPSHOT on the cluster (bundle/direct via hack/install). The service step only runs lightspeed-service e2e (test-e2e-cluster.sh / pytest), which reconfigures OLS per suite (adapt_ols_config) and does not replace that snapshot install contract.

Pytest calls install_ols() only when the operator is not already there (OLM: CSV; direct: lightspeed-operator-controller-manager deployment). Otherwise it skips install and still runs adapt_ols_config() each run.

JoaoFula · 2026-04-30T07:23:03Z

+		echo "error: ${REPO_ROOT}/config/default/deployment-patch.yaml not found" >&2
+		exit 1
+	}
+	if [[ -z "$operator_img" ]]; then


at script entry we're verifying that $IMG is not empty. Therefore this and the following ifs will never activate.

good catch, fixed

JoaoFula · 2026-04-30T07:46:54Z

+			echo "error: catalog dir not found: ${catalog_dir} (cwd: $(pwd))" >&2
+			exit 1
+		fi
+		oldest_ver="$(ls "${catalog_dir}"/bundle-v*.yaml | sed -n 's/.*bundle-v\(.*\)\.yaml/\1/p' | sort -V | head -n1)"


This will cause every upgrade test to fail since we're picking 1.0.8 and there's a known upgrade issue from 1.0.9 to 1.0.10. So if we pick 1.0.8 to any newer version, it will always fail. Thus why in my PR, I'm picking latest - 1. Assuming we're always testing it from now on, we never will run into problems and we will test all upgrades.

Agreed — we switched the upgrade base from the oldest catalog entry to latest − 1 (second-newest semver in lightspeed-catalog-), with a hard error if there are fewer than two bundles. That avoids long broken chains from very old bases (e.g. 1.0.8) while still testing a real catalog → snapshot upgrade. Pipeline descriptions are updated to match.

Made-with: Cursor

openshift-ci · 2026-04-30T14:31:36Z

@blublinsky: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/bundle-e2e-4-21	`11cfd36`	link	true	`/test bundle-e2e-4-21`

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

openshift-ci Bot requested a review from raptorsun April 22, 2026 15:28

openshift-ci Bot requested a review from xrajesh April 22, 2026 15:28

blublinsky force-pushed the tests-split-between-operator-bundle branch 2 times, most recently from 33ee02f to 9731cba Compare April 23, 2026 09:31

blublinsky force-pushed the tests-split-between-operator-bundle branch 15 times, most recently from 3897154 to 053c2e8 Compare April 26, 2026 17:03

blublinsky changed the title ~~spliting e2e tests between bundle and operator~~ splitting e2e tests between bundle and operator Apr 27, 2026

blublinsky force-pushed the tests-split-between-operator-bundle branch 3 times, most recently from 411e277 to 499dee7 Compare April 28, 2026 15:16

openshift-ci Bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 29, 2026

JoaoFula reviewed Apr 30, 2026

View reviewed changes

blublinsky force-pushed the tests-split-between-operator-bundle branch from 499dee7 to 5b143e7 Compare April 30, 2026 12:52

Splitting e2e tests between bundle and operator

11cfd36

Made-with: Cursor

blublinsky force-pushed the tests-split-between-operator-bundle branch from 5b143e7 to 11cfd36 Compare April 30, 2026 13:23

openshift-ci Bot removed the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Apr 30, 2026

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

splitting e2e tests between bundle and operator#1548

splitting e2e tests between bundle and operator#1548
blublinsky wants to merge 1 commit intoopenshift:mainfrom
blublinsky:tests-split-between-operator-bundle

blublinsky commented Apr 22, 2026

Uh oh!

openshift-ci Bot commented Apr 22, 2026

Uh oh!

blublinsky commented Apr 23, 2026

Uh oh!

JoaoFula Apr 30, 2026

Uh oh!

blublinsky Apr 30, 2026

Uh oh!

JoaoFula Apr 30, 2026

Uh oh!

blublinsky Apr 30, 2026

Uh oh!

JoaoFula Apr 30, 2026 •

edited

Loading

Uh oh!

blublinsky Apr 30, 2026

Uh oh!

openshift-ci Bot commented Apr 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

blublinsky commented Apr 22, 2026

Description

Type of change

Related Tickets & Documents

Checklist before requesting a review

Testing

Uh oh!

openshift-ci Bot commented Apr 22, 2026

Uh oh!

blublinsky commented Apr 23, 2026

Uh oh!

JoaoFula Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

blublinsky Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

JoaoFula Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

blublinsky Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

JoaoFula Apr 30, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

blublinsky Apr 30, 2026

Choose a reason for hiding this comment

Uh oh!

openshift-ci Bot commented Apr 30, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

JoaoFula Apr 30, 2026 •

edited

Loading