Skip to content

Conversation

camilamacedo86
Copy link
Contributor

@camilamacedo86 camilamacedo86 commented Jul 25, 2025

This PR adds the setup to initialise the test environment and access the cluster config.
It also includes the first test migrated from OpenShift/Origin (more will be added in follow-up PRs; I want to ensure that we have a good base before migrating all).

We added extlogs to improve test logs. It uses Ginkgo’s writer to show clear info and warnings,
which helps when tests fail. Since we will need to migrate the tests, it is an excellent opportunity to ensure that we have good logs to cover the scenarios.

The setup utilises the controller-runtime client, similar to our E2E tests.
This lets us run the tests locally with KinD (with OLMv1 installed) or in CI on OpenShift.
Only a few parts are OCP-specific; we have only 3/4 of the features specific to OCP, while the rest is the same as used in upstream. Therefore, most tests can run using Kind and locally, which facilitates the development and troubleshooting process. Furthermore, my vision for the future is to explore the possibility of reusing the tests created for our e2e tests within our organisation, rather than duplicating staff efforts

This makes writing and running tests for OCP much easier and faster.

c/c @tmshort @kuiwang02 @dtfranz @anik120 @grokspawn

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 25, 2025
@openshift-ci-robot openshift-ci-robot added the jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. label Jul 25, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 25, 2025

@camilamacedo86: This pull request references OPRUN-3983 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@camilamacedo86 camilamacedo86 force-pushed the ote-add-first-test branch 4 times, most recently from bd38fcc to 67e2139 Compare July 26, 2025 06:10
@camilamacedo86 camilamacedo86 changed the title WIP: OPRUN-3983: OTE add first test from openshift/origin olmv1.go OPRUN-3983: OTE add first test from openshift/origin olmv1.go Jul 26, 2025
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 26, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 26, 2025

@camilamacedo86: This pull request references OPRUN-3983 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

This PR adds the setup to initialise the test environment and access the cluster config.
It also includes the first test migrated from OpenShift/Origin (more will be added in follow-up PRs; I want to ensure that we have a good base before migrating all).

We added extlogs to improve test logs. It uses Ginkgo’s writer to show clear info and warnings,
which helps when tests fail. Since we will need to migrate the tests, it is an excellent opportunity to ensure that we have good logs to cover the scenarios.

The setup utilises the controller-runtime client, similar to our E2E tests.
This lets us run the tests locally with KinD (with OLMv1 installed) or in CI on OpenShift.
Only a few parts are OCP-specific, so most tests can run using Kind and locally, which facilitates the development and troubleshooting process. Furthermore, my vision for the feature is to see if we can reuse the tests done for our e2e tests within.

This makes writing and running tests for OCP much easier and faster.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 26, 2025

@camilamacedo86: This pull request references OPRUN-3983 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

This PR adds the setup to initialise the test environment and access the cluster config.
It also includes the first test migrated from OpenShift/Origin (more will be added in follow-up PRs; I want to ensure that we have a good base before migrating all).

We added extlogs to improve test logs. It uses Ginkgo’s writer to show clear info and warnings,
which helps when tests fail. Since we will need to migrate the tests, it is an excellent opportunity to ensure that we have good logs to cover the scenarios.

The setup utilises the controller-runtime client, similar to our E2E tests.
This lets us run the tests locally with KinD (with OLMv1 installed) or in CI on OpenShift.
**Only a few parts are OCP-specific; we have only 3/4 of the features specific to OCP, while the rest is the same as used in upstream.**Therefore, most tests can run using Kind and locally, which facilitates the development and troubleshooting process. Furthermore, my vision for the feature is to see if we can reuse the tests done for our e2e tests within.

This makes writing and running tests for OCP much easier and faster.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 26, 2025

@camilamacedo86: This pull request references OPRUN-3983 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

This PR adds the setup to initialise the test environment and access the cluster config.
It also includes the first test migrated from OpenShift/Origin (more will be added in follow-up PRs; I want to ensure that we have a good base before migrating all).

We added extlogs to improve test logs. It uses Ginkgo’s writer to show clear info and warnings,
which helps when tests fail. Since we will need to migrate the tests, it is an excellent opportunity to ensure that we have good logs to cover the scenarios.

The setup utilises the controller-runtime client, similar to our E2E tests.
This lets us run the tests locally with KinD (with OLMv1 installed) or in CI on OpenShift.
Only a few parts are OCP-specific; we have only 3/4 of the features specific to OCP, while the rest is the same as used in upstream. Therefore, most tests can run using Kind and locally, which facilitates the development and troubleshooting process. Furthermore, my vision for the feature is to see if we can reuse the tests done for our e2e tests within.

This makes writing and running tests for OCP much easier and faster.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 26, 2025

@camilamacedo86: This pull request references OPRUN-3983 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

This PR adds the setup to initialise the test environment and access the cluster config.
It also includes the first test migrated from OpenShift/Origin (more will be added in follow-up PRs; I want to ensure that we have a good base before migrating all).

We added extlogs to improve test logs. It uses Ginkgo’s writer to show clear info and warnings,
which helps when tests fail. Since we will need to migrate the tests, it is an excellent opportunity to ensure that we have good logs to cover the scenarios.

The setup utilises the controller-runtime client, similar to our E2E tests.
This lets us run the tests locally with KinD (with OLMv1 installed) or in CI on OpenShift.
Only a few parts are OCP-specific; we have only 3/4 of the features specific to OCP, while the rest is the same as used in upstream. Therefore, most tests can run using Kind and locally, which facilitates the development and troubleshooting process. Furthermore, my vision for the feature is to see if we can reuse the tests done for our e2e tests within.

This makes writing and running tests for OCP much easier and faster.

c/c @tmshort @kuiwang02 @dtfranz @anik120

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

}
]
```

Copy link
Contributor Author

@camilamacedo86 camilamacedo86 Jul 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kuiwang02 It’s totally fine to add this in a follow-up, but I’d like to start documenting a comprehensive guide here on how to run the tests locally using openshift-bot.

Ideally something like:

  • Step 1
  • Step 2
  • Step 3

I haven’t tried it yet, but from what I understand, we need to:

  • Create the OCP instance with the PR changes
  • Set KUBECONFIG to point to that OCP cluster
  • Then run the tests against it

If you know of any tricks or caveats—for example, how to enable OLMv1 with TechPreview features using openshift-bot—I’d really appreciate your guidance.

Once I try it locally, I’ll create a follow-up PR with the detailed steps.

Thanks! 🙏

Copy link
Contributor

@kuiwang02 kuiwang02 Jul 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@camilamacedo86
actually whether the cluster is created by kind, cluster-bot, and installation step of openshift CI, as long as there is KUBECONFIG file, it is same to run case (for kind cluster, need to manually enable TP if needed).

for example,
1, cluster created by kind: you will get KUBECONFIG. and you enable TP manually. and then you set KUBECONFIG and test case.

2, cluster created by cluster-bot (I think openshift-bot you said is cluster-bot): when you create it with launch with variant techpreview , like launch openshift/operator-framework-operator-controller#393,4.20 aws,techpreview, the TP is already enabled if the cluster is created successfully. and then you set KUBECONFIG and test case. no need to enable TP manually before testing case.
3, cluster created by installation step of openshift CI, (like openshift-e2e-aws): after you set env FEATURE_SET: TechPreviewNoUpgrade in configuration, the TP is ready enabled after the cluster is created sucessfully. and then you set KUEBECONFG and test case. no need to enable TP manually before testing casee.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 26, 2025

@camilamacedo86: This pull request references OPRUN-3983 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

This PR adds the setup to initialise the test environment and access the cluster config.
It also includes the first test migrated from OpenShift/Origin (more will be added in follow-up PRs; I want to ensure that we have a good base before migrating all).

We added extlogs to improve test logs. It uses Ginkgo’s writer to show clear info and warnings,
which helps when tests fail. Since we will need to migrate the tests, it is an excellent opportunity to ensure that we have good logs to cover the scenarios.

The setup utilises the controller-runtime client, similar to our E2E tests.
This lets us run the tests locally with KinD (with OLMv1 installed) or in CI on OpenShift.
Only a few parts are OCP-specific; we have only 3/4 of the features specific to OCP, while the rest is the same as used in upstream. Therefore, most tests can run using Kind and locally, which facilitates the development and troubleshooting process. Furthermore, my vision for the future is to explore the possibility of reusing the tests created for our e2e tests within our organisation, rather than duplicating staff efforts. (We will probably need to advocate for the team accept gingo in upstream tests and either check the trade-offs, but it seems so far a possibility)

This makes writing and running tests for OCP much easier and faster.

c/c @tmshort @kuiwang02 @dtfranz @anik120

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 26, 2025

@camilamacedo86: This pull request references OPRUN-3983 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

This PR adds the setup to initialise the test environment and access the cluster config.
It also includes the first test migrated from OpenShift/Origin (more will be added in follow-up PRs; I want to ensure that we have a good base before migrating all).

We added extlogs to improve test logs. It uses Ginkgo’s writer to show clear info and warnings,
which helps when tests fail. Since we will need to migrate the tests, it is an excellent opportunity to ensure that we have good logs to cover the scenarios.

The setup utilises the controller-runtime client, similar to our E2E tests.
This lets us run the tests locally with KinD (with OLMv1 installed) or in CI on OpenShift.
Only a few parts are OCP-specific; we have only 3/4 of the features specific to OCP, while the rest is the same as used in upstream. Therefore, most tests can run using Kind and locally, which facilitates the development and troubleshooting process. Furthermore, my vision for the future is to explore the possibility of reusing the tests created for our e2e tests within our organisation, rather than duplicating staff efforts

This makes writing and running tests for OCP much easier and faster.

c/c @tmshort @kuiwang02 @dtfranz @anik120

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

//
// References:
// - https://github.com/openshift/origin/blob/main/pkg/cmd/openshift-tests/run/flags.go#L53
// - https://github.com/openshift/origin/blob/main/pkg/clioptions/clusterdiscovery/provider.go#L100
Copy link
Contributor Author

@camilamacedo86 camilamacedo86 Jul 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@kuiwang02

We’re initialising the suite directly.
WHY?

  • If a test doesn’t require a cluster (e.g., sanity checks or validations that run outside the cluster), we skip environment initialisation.
  • Most Important If initialising the env globally here for ALL it can cause issues for [parallel] tests. It would result in a shared config across all tests, but we want isolated configs per test. This avoids conflicts when tests are run in parallel across different instances or clusters.

Copy link
Contributor

@kuiwang02 kuiwang02 Jul 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@camilamacedo86
the OTE suggests this way https://github.com/openshift-eng/openshift-tests-extension/blob/main/cmd/example-tests/main.go#L64-L66.

Because your code has

func Init() *TestEnv {
	if testEnv == nil {
		testEnv = initTestEnv()
	}
	return testEnv
}

to avoid to init cluster repeatedly

so, I do not find side effective with initing it in BeforeEach, not taking the way suggested by OTE.
So, I think it is ok. and we can go in this way to monitor if we meet issue late.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done the change since it makes it easier for us to develop.
Less one thing to remmber and will do the same :-)

}
}
})
})
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same test done in : https://github.com/openshift/origin/blob/48d56f2e5bb8bcf6c4560fb077c7b809e18a5c19/test/extended/olm/olmv1.go#L34-L74

(Migration - After we migrate for here and check that all is fine with Sippy we can remove from opc/origin)

if !env.Get().IsOpenShift {
extlogs.Warn("Skipping feature capability check: not OpenShift")
return
}
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note: If OLMv1 is installed locally in KinD, the same test will run just fine.
We’re only checking if the CRDs are present, which works outside OpenShift.
The capability check (e.g. "OperatorLifecycleManagerV1") is specific to OpenShift
and only required in OpenShift CI.

So now, we skip that check when running on non-OCP clusters.
This allows us to run and validate the test locally, making development easier and faster before pushing the PR.

This is the same logic used here:
https://github.com/openshift/origin/blob/48d56f2e5bb8bcf6c4560fb077c7b809e18a5c19/test/extended/olm/olmv1.go#L321-L327

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 26, 2025

@camilamacedo86: This pull request references OPRUN-3983 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

This PR adds the setup to initialise the test environment and access the cluster config.
It also includes the first test migrated from OpenShift/Origin (more will be added in follow-up PRs; I want to ensure that we have a good base before migrating all).

We added extlogs to improve test logs. It uses Ginkgo’s writer to show clear info and warnings,
which helps when tests fail. Since we will need to migrate the tests, it is an excellent opportunity to ensure that we have good logs to cover the scenarios.

The setup utilises the controller-runtime client, similar to our E2E tests.
This lets us run the tests locally with KinD (with OLMv1 installed) or in CI on OpenShift.
Only a few parts are OCP-specific; we have only 3/4 of the features specific to OCP, while the rest is the same as used in upstream. Therefore, most tests can run using Kind and locally, which facilitates the development and troubleshooting process. Furthermore, my vision for the future is to explore the possibility of reusing the tests created for our e2e tests within our organisation, rather than duplicating staff efforts

This makes writing and running tests for OCP much easier and faster.

c/c @tmshort @kuiwang02 @dtfranz @anik120 @grokspawn

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

utilruntime.Must(configv1.AddToScheme(scheme))
}

k8sClient, err := crclient.New(cfg, crclient.Options{Scheme: scheme})
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We don’t need to use RunCommand("oc") as done in https://github.com/openshift/origin/blob/48d56f2e5bb8bcf6c4560fb077c7b809e18a5c19/test/extended/util/client.go#L165-L193 and ALL that complixities.

Instead, we can use the controller-runtime client, just like we do in our e2e tests, to interact with the cluster.

This approach helps:

  • Keep tests compatible with both oc and Kubernetes
  • Simplify the codebase
  • Avoid depending on CLI tooling

For reference, see how we do our e2e tests:
https://github.com/openshift/operator-framework-operator-controller/tree/main/test

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @kuiwang02

It works like our current e2e tests and keeps things simple.
If we find any test that really needs something different, we’ll handle it case by case.
Let’s keep it simple (KISS) and only add what’s truly needed for those special cases.

That way, we avoid unnecessary complexity and keep our setup clean.

If we try to add everything we might need up front, we risk falling into the premature optimization trap.
That often leads to unnecessary complexity and code that's harder to maintain — without actually solving real problems.

Copy link
Contributor

@kuiwang02 kuiwang02 Jul 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@camilamacedo86
1, For all QE cases, we need oc client (it seems olmv1 origin case does not need it), like https://github.com/openshift/origin/blob/48d56f2e5bb8bcf6c4560fb077c7b809e18a5c19/test/extended/util/client.go#L165-L193. currently you could not care it because you are migrate olmv1 origin case. when we migrate QE cases, I will think about it.
2, For all QE cases, it depends on k8s.io/kubernetes/test/e2e/framework (it seems olmv1 origin case does not need it). it also need init when initing cluster. when we migrate QE cases, I will think about it.

So, as summary the current code on it is ok to migrate olmv1 origin cases. you could go on for your work.

I will care about it late for QE cases migration.

Copy link
Contributor Author

@camilamacedo86 camilamacedo86 Jul 28, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1, For all QE cases, we need oc client

Currently, we use oc/kubectl by calling them from the command line and parsing the output manually.
The change here is to use the Kubernetes API directly instead.

If needed, we can still call the CLI like in this example:
https://github.com/operator-framework/operator-controller/blob/main/test/utils/utils.go#L11-L23
—but using the API gives us the same result, in a cleaner way.

k8s.io/kubernetes/test/e2e/framework

We’ll need to review the specific cases as we migrate them, but from what I’ve seen so far across the repos, there’s a lot of complexity that might not be necessary.

It looks like we have multiple ways to achieve the same outcomes. I see this migration as a good opportunity to revisit the tests being added, reduce duplication, avoid unnecessary complexity, and make sure we have solid logging in place to help us debug when errors occur.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@camilamacedo86
Frankly, QE test cases are far more complex than the origin OLMv1 cases. They’re similar to the complex cases in other Origin modules that rely on Origin’s NewCLI, which in turn depends on k8s.io/kubernetes/test/e2e/framework and various other initializations.
Because the Origin OLMv1 cases are relatively simple, the cluster initialization work you’ve done so far is sufficient—but it may only apply to those simple OLMv1 cases. When I eventually migrate the QE cases, I’ll revisit and adapt the initialization logic; that work will take quite some time. Right now I’m tied up with other priorities and don’t have the bandwidth to tackle it.
Fortunately, your current work can proceed as planned.

Therefore:

/lgtm

And maybe I’ll adjust the init logic when migrating the QE cases in the future.
Thanks

// The default limits (QPS=5, Burst=10) are too low for most test workloads.
func configureQPS(cfg *rest.Config) *rest.Config {
cfg.QPS = 10000
cfg.Burst = 10000
Copy link
Contributor Author

@camilamacedo86 camilamacedo86 Jul 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

func configureQPS(cfg *rest.Config) *rest.Config {
cfg.QPS = 10000
cfg.Burst = 10000
cfg.RateLimiter = nil
Copy link
Contributor Author

@camilamacedo86 camilamacedo86 Jul 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@camilamacedo86 camilamacedo86 force-pushed the ote-add-first-test branch 2 times, most recently from 4af8314 to 82ffddd Compare July 26, 2025 07:24
@camilamacedo86
Copy link
Contributor Author

/test okd-scos-e2e-aws-ovn

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 26, 2025

@camilamacedo86: This pull request references OPRUN-3983 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

This PR adds the setup to initialise the test environment and access the cluster config.
It also includes the first test migrated from OpenShift/Origin (more will be added in follow-up PRs; I want to ensure that we have a good base before migrating all).

We added extlogs to improve test logs. It uses Ginkgo’s writer to show clear info and warnings,
which helps when tests fail. Since we will need to migrate the tests, it is an excellent opportunity to ensure that we have good logs to cover the scenarios.

The setup utilises the controller-runtime client, similar to our E2E tests.
This lets us run the tests locally with KinD (with OLMv1 installed) or in CI on OpenShift.
Only a few parts are OCP-specific; we have only 3/4 of the features specific to OCP, while the rest is the same as used in upstream. Therefore, most tests can run using Kind and locally, which facilitates the development and troubleshooting process. Furthermore, my vision for the future is to explore the possibility of reusing the tests created for our e2e tests within our organisation, rather than duplicating staff efforts

This makes writing and running tests for OCP much easier and faster.

Test

We can check that the test worked.

Screenshot 2025-07-26 at 18 51 46

Migration

c/c @tmshort @kuiwang02 @dtfranz @anik120 @grokspawn

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 26, 2025

@camilamacedo86: This pull request references OPRUN-3983 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

This PR adds the setup to initialise the test environment and access the cluster config.
It also includes the first test migrated from OpenShift/Origin (more will be added in follow-up PRs; I want to ensure that we have a good base before migrating all).

We added extlogs to improve test logs. It uses Ginkgo’s writer to show clear info and warnings,
which helps when tests fail. Since we will need to migrate the tests, it is an excellent opportunity to ensure that we have good logs to cover the scenarios.

The setup utilises the controller-runtime client, similar to our E2E tests.
This lets us run the tests locally with KinD (with OLMv1 installed) or in CI on OpenShift.
Only a few parts are OCP-specific; we have only 3/4 of the features specific to OCP, while the rest is the same as used in upstream. Therefore, most tests can run using Kind and locally, which facilitates the development and troubleshooting process. Furthermore, my vision for the future is to explore the possibility of reusing the tests created for our e2e tests within our organisation, rather than duplicating staff efforts

This makes writing and running tests for OCP much easier and faster.

Test

Migration

We need to ensure the same names:

https://sippy.dptools.openshift.org/sippy-ng/tests/4.20?filters=%257B%2522items%2522%253A%255B%257B%2522columnField%2522%253A%2522current_runs%2522%252C%2522operatorValue%2522%253A%2522%253E%253D%2522%252C%2522value%2522%253A%25227%2522%257D%252C%257B%2522columnField%2522%253A%2522variants%2522%252C%2522not%2522%253Atrue%252C%2522operatorValue%2522%253A%2522contains%2522%252C%2522value%2522%253A%2522never-stable%2522%257D%252C%257B%2522columnField%2522%253A%2522variants%2522%252C%2522not%2522%253Atrue%252C%2522operatorValue%2522%253A%2522contains%2522%252C%2522value%2522%253A%2522aggregated%2522%257D%252C%257B%2522columnField%2522%253A%2522current_flake_percentage%2522%252C%2522not%2522%253Atrue%252C%2522operatorValue%2522%253A%2522%253D%2522%252C%2522value%2522%253A%2522100%2522%257D%252C%257B%2522id%2522%253A99%252C%2522columnField%2522%253A%2522name%2522%252C%2522operatorValue%2522%253A%2522contains%2522%252C%2522value%2522%253A%2522%255BOCPFeatureGate%253ANewOLM%255D%2520OLMv1%2520CRDs%2522%257D%255D%252C%2522linkOperator%2522%253A%2522and%2522%257D&sort=asc&sortField=net_improvement

Screenshot 2025-07-26 at 18 56 29

c/c @tmshort @kuiwang02 @dtfranz @anik120 @grokspawn

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 27, 2025

@camilamacedo86: This pull request references OPRUN-3983 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

This PR adds the setup to initialise the test environment and access the cluster config.
It also includes the first test migrated from OpenShift/Origin (more will be added in follow-up PRs; I want to ensure that we have a good base before migrating all).

We added extlogs to improve test logs. It uses Ginkgo’s writer to show clear info and warnings,
which helps when tests fail. Since we will need to migrate the tests, it is an excellent opportunity to ensure that we have good logs to cover the scenarios.

The setup utilises the controller-runtime client, similar to our E2E tests.
This lets us run the tests locally with KinD (with OLMv1 installed) or in CI on OpenShift.
Only a few parts are OCP-specific; we have only 3/4 of the features specific to OCP, while the rest is the same as used in upstream. Therefore, most tests can run using Kind and locally, which facilitates the development and troubleshooting process. Furthermore, my vision for the future is to explore the possibility of reusing the tests created for our e2e tests within our organisation, rather than duplicating staff efforts

This makes writing and running tests for OCP much easier and faster.

c/c @tmshort @kuiwang02 @dtfranz @anik120 @grokspawn

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

},
"source": "openshift:payload:olmv1",
"lifecycle": "blocking",
"environmentSelector": {}
Copy link
Contributor Author

@camilamacedo86
Copy link
Contributor Author

/acknowledge-critical-fixes-only

^ we are not in code freeze yet.
It is only to be more cautious.
It is only 1 test for OTE, which should not impact the release at all
Furthermore, we are testing it out without e2e:

https://prow.ci.openshift.org/view/gs/test-platform-results/pr-logs/pull/openshift_operator-framework-operator-controller/415/pull-ci-openshift-operator-framework-operator-controller-main-e2e-aws-techpreview-olmv1-ext/1949167652710649856

To ensure that all is fine.
We need to merge this one to unblock the next steps.

@camilamacedo86
Copy link
Contributor Author

/acknowledge-critical-fixes-only

@camilamacedo86
Copy link
Contributor Author

/test openshift-e2e-aws-techpreview

@camilamacedo86
Copy link
Contributor Author

/test openshift-e2e-aws

Copy link
Contributor

openshift-ci bot commented Jul 28, 2025

@camilamacedo86: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/okd-scos-e2e-aws-ovn cf04ac0 link false /test okd-scos-e2e-aws-ovn

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@openshift-ci openshift-ci bot added the lgtm Indicates that a PR is ready to be merged. label Jul 29, 2025
@camilamacedo86 camilamacedo86 changed the title OPRUN-3983: OTE add first test from openshift/origin olmv1.go OPRUN-3964: OTE add first test from openshift/origin olmv1.go Jul 31, 2025
@openshift-ci-robot
Copy link

openshift-ci-robot commented Jul 31, 2025

@camilamacedo86: This pull request references OPRUN-3964 which is a valid jira issue.

Warning: The referenced jira issue has an invalid target version for the target branch this PR targets: expected the story to target the "4.20.0" version, but no target version was set.

In response to this:

This PR adds the setup to initialise the test environment and access the cluster config.
It also includes the first test migrated from OpenShift/Origin (more will be added in follow-up PRs; I want to ensure that we have a good base before migrating all).

We added extlogs to improve test logs. It uses Ginkgo’s writer to show clear info and warnings,
which helps when tests fail. Since we will need to migrate the tests, it is an excellent opportunity to ensure that we have good logs to cover the scenarios.

The setup utilises the controller-runtime client, similar to our E2E tests.
This lets us run the tests locally with KinD (with OLMv1 installed) or in CI on OpenShift.
Only a few parts are OCP-specific; we have only 3/4 of the features specific to OCP, while the rest is the same as used in upstream. Therefore, most tests can run using Kind and locally, which facilitates the development and troubleshooting process. Furthermore, my vision for the future is to explore the possibility of reusing the tests created for our e2e tests within our organisation, rather than duplicating staff efforts

This makes writing and running tests for OCP much easier and faster.

c/c @tmshort @kuiwang02 @dtfranz @anik120 @grokspawn

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@tmshort
Copy link
Contributor

tmshort commented Jul 31, 2025

/approve

Copy link
Contributor

openshift-ci bot commented Jul 31, 2025

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: camilamacedo86, tmshort

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 31, 2025
@openshift-merge-bot openshift-merge-bot bot merged commit 1b06fc6 into openshift:main Jul 31, 2025
12 of 13 checks passed
@openshift-bot
Copy link

[ART PR BUILD NOTIFIER]

Distgit: ose-olm-operator-controller
This PR has been included in build ose-olm-operator-controller-container-v4.20.0-202507310343.p0.g1b06fc6.assembly.stream.el9.
All builds following this will include this PR.

@openshift-bot
Copy link

[ART PR BUILD NOTIFIER]

Distgit: ose-olm-catalogd
This PR has been included in build ose-olm-catalogd-container-v4.20.0-202507310343.p0.g1b06fc6.assembly.stream.el9.
All builds following this will include this PR.

@camilamacedo86 camilamacedo86 deleted the ote-add-first-test branch July 31, 2025 05:59
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

approved Indicates a PR has been approved by an approver from all required OWNERS files. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. lgtm Indicates that a PR is ready to be merged.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants