Skip to content

Conversation

@tmshort
Copy link
Contributor

@tmshort tmshort commented Nov 24, 2025

The current set of upgrade-e2e tests upgrade from the last release to the current PR/repo state. Standard is tested against standard, experimental is tested against experimental.

We should have upgrade e2e tests that upgrade from standard to experimental, including from the current PR/repo state, and also from the prior release.

This also renames the targets to make it clear what is being upgraded.

Description

Reviewer Checklist

  • API Go Documentation
  • Tests: Unit Tests (and E2E Tests, if appropriate)
  • Comprehensive Commit Messages
  • Links to related GitHub Issue(s)

@tmshort tmshort requested a review from a team as a code owner November 24, 2025 16:26
@openshift-ci openshift-ci bot requested review from dtfranz and pedjak November 24, 2025 16:26
@netlify
Copy link

netlify bot commented Nov 24, 2025

Deploy Preview for olmv1 ready!

Name Link
🔨 Latest commit fc683b1
🔍 Latest deploy log https://app.netlify.com/projects/olmv1/deploys/6925bfbbd919320008a409a7
😎 Deploy Preview https://deploy-preview-2360--olmv1.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@tmshort
Copy link
Contributor Author

tmshort commented Nov 24, 2025

/override upgrade-e2e
The upgrade-e2e has been renamed.

@openshift-ci
Copy link

openshift-ci bot commented Nov 24, 2025

@tmshort: Overrode contexts on behalf of tmshort: upgrade-e2e

In response to this:

/override upgrade-e2e
The upgrade-e2e has been renamed.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@tmshort
Copy link
Contributor Author

tmshort commented Nov 24, 2025

upgrade-ex2ex-e2e will continue to fail until we issue a release.

@tmshort
Copy link
Contributor Author

tmshort commented Nov 24, 2025

When this merges, the upgrade-e2e will no longer be required, but we will want to make the other upgrade tests possibly required.

@codecov
Copy link

codecov bot commented Nov 24, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 74.45%. Comparing base (dde685c) to head (fc683b1).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #2360   +/-   ##
=======================================
  Coverage   74.45%   74.45%           
=======================================
  Files          93       93           
  Lines        7300     7300           
=======================================
  Hits         5435     5435           
  Misses       1433     1433           
  Partials      432      432           
Flag Coverage Δ
e2e 44.51% <ø> (ø)
experimental-e2e 48.74% <ø> (ø)
unit 58.47% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Comment on lines 114 to 144
upgrade-st2ex-e2e:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6

- uses: actions/setup-go@v6
with:
go-version-file: go.mod

- name: Run the upgrade e2e test
run: ARTIFACT_PATH=/tmp/artifacts make test-upgrade-st2ex-e2e

- uses: actions/upload-artifact@v5
if: failure()
with:
name: upgrade-st2ex-e2e-artifacts
path: /tmp/artifacts/

st2ex-e2e:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v6

- uses: actions/setup-go@v6
with:
go-version-file: go.mod

- name: Run the upgrade e2e test
run: ARTIFACT_PATH=/tmp/artifacts make test-st2ex-e2e

- uses: actions/upload-artifact@v5
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given that we are adding two more test jobs in this PR (in total now 4), would it make sense to declare a GitHub matrix job to execute them in parallel?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are similar enough...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They are similar enough...

Right, hence I though we could pack them in matrix - they differ just by the makefile target it gets invoked.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's a bit more than that, but it's been updated to be a matrix

Signed-off-by: Todd Short <tshort@redhat.com>
@tmshort
Copy link
Contributor Author

tmshort commented Nov 25, 2025

Due to the renaming of some tests, the e2e-kind and upgrade-e2e are no longer found, but they do pass (as e2e/e2e and e2e/upgrade-st2st-e2e)

Copy link
Contributor

@pedjak pedjak left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Nov 25, 2025
@pedjak
Copy link
Contributor

pedjak commented Nov 26, 2025

[nit] Given that this PR is about test infra work, and no user-facing changes, does in qualify to be marked as ✨ ?

Copy link
Member

@rashmigottipati rashmigottipati left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@tmshort
Copy link
Contributor Author

tmshort commented Nov 26, 2025

/approve

@openshift-ci
Copy link

openshift-ci bot commented Nov 26, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: pedjak, rashmigottipati, tmshort

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Nov 26, 2025
@tmshort
Copy link
Contributor Author

tmshort commented Nov 26, 2025

/override e2e-kind

@openshift-ci
Copy link

openshift-ci bot commented Nov 26, 2025

@tmshort: Overrode contexts on behalf of tmshort: e2e-kind

In response to this:

/override e2e-kind

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@tmshort
Copy link
Contributor Author

tmshort commented Nov 26, 2025

/override upgrade-e2e

@openshift-ci
Copy link

openshift-ci bot commented Nov 26, 2025

@tmshort: Overrode contexts on behalf of tmshort: upgrade-e2e

In response to this:

/override upgrade-e2e

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@openshift-merge-bot openshift-merge-bot bot merged commit 045989d into operator-framework:main Nov 26, 2025
34 of 35 checks passed
@tmshort tmshort deleted the new-e2es branch November 26, 2025 14:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants