Skip to content

OCPQE-30671:Add [OTP] to migrated tests-private cases Per ERT #547

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Nov 10, 2025
Merged

OCPQE-30671:Add [OTP] to migrated tests-private cases Per ERT #547

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
98 changes: 64 additions & 34 deletions openshift/tests-extension/.openshift-tests-extension/openshift_payload_olmv1.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
[
{
"name": "[sig-olmv1][Jira:OLM] clustercatalog PolarionID:69242-[Skipped:Disconnected]Catalogd deprecated package/bundlemetadata/catalogmetadata from clustercatalog CR",
"name": "[sig-olmv1][Jira:OLM] clustercatalog PolarionID:69242-[OTP][Skipped:Disconnected]Catalogd deprecated package bundlemetadata catalogmetadata from clustercatalog CR",
"originalName": "[sig-olmv1][Jira:OLM] clustercatalog PolarionID:69242-[Skipped:Disconnected]Catalogd deprecated package bundlemetadata catalogmetadata from clustercatalog CR",
"labels": {
"Extended": {},
"NonHyperShiftHOST": {}
"NonHyperShiftHOST": {},
"original-name:[sig-olmv1][Jira:OLM] clustercatalog PolarionID:69242-[Skipped:Disconnected]Catalogd deprecated package bundlemetadata catalogmetadata from clustercatalog CR": {}
},
"resources": {
"isolation": {}
Expand All @@ -15,11 +17,13 @@
}
},
{
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:83069-olmv1 static networkpolicy.",
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:83069-[OTP]olmv1 static networkpolicy.",
"originalName": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:83069-olmv1 static networkpolicy.",
"labels": {
"Extended": {},
"NonHyperShiftHOST": {},
"ReleaseGate": {}
"ReleaseGate": {},
"original-name:[sig-olmv1][Jira:OLM] clusterextension PolarionID:83069-olmv1 static networkpolicy.": {}
},
"resources": {
"isolation": {}
Expand All @@ -31,10 +35,12 @@
}
},
{
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:68936-[Skipped:Disconnected]cluster extension can not be installed with insufficient permission sa for operand",
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:68936-[OTP][Skipped:Disconnected]cluster extension can not be installed with insufficient permission sa for operand",
"originalName": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:68936-[Skipped:Disconnected]cluster extension can not be installed with insufficient permission sa for operand",
"labels": {
"Extended": {},
"NonHyperShiftHOST": {}
"NonHyperShiftHOST": {},
"original-name:[sig-olmv1][Jira:OLM] clusterextension PolarionID:68936-[Skipped:Disconnected]cluster extension can not be installed with insufficient permission sa for operand": {}
},
"resources": {
"isolation": {}
Expand All @@ -46,10 +52,12 @@
}
},
{
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:68937-[Skipped:Disconnected]cluster extension can not be installed with insufficient permission sa for operand rbac object",
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:68937-[OTP][Skipped:Disconnected]cluster extension can not be installed with insufficient permission sa for operand rbac object",
"originalName": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:68937-[Skipped:Disconnected]cluster extension can not be installed with insufficient permission sa for operand rbac object",
"labels": {
"Extended": {},
"NonHyperShiftHOST": {}
"NonHyperShiftHOST": {},
"original-name:[sig-olmv1][Jira:OLM] clusterextension PolarionID:68937-[Skipped:Disconnected]cluster extension can not be installed with insufficient permission sa for operand rbac object": {}
},
"resources": {
"isolation": {}
Expand All @@ -61,11 +69,12 @@
}
},
{
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:75492-[Skipped:Disconnected]cluster extension can not be installed with wrong sa or insufficient permission sa",
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:75492-[OTP][Level0][Skipped:Disconnected]cluster extension can not be installed with wrong sa or insufficient permission sa",
"originalName": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:75492-[Skipped:Disconnected]cluster extension can not be installed with wrong sa or insufficient permission sa",
"labels": {
"Extended": {},
"LEVEL0": {},
"NonHyperShiftHOST": {}
"NonHyperShiftHOST": {},
"original-name:[sig-olmv1][Jira:OLM] clusterextension PolarionID:75492-[Skipped:Disconnected]cluster extension can not be installed with wrong sa or insufficient permission sa": {}
},
"resources": {
"isolation": {}
Expand All @@ -77,11 +86,12 @@
}
},
{
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:75493-[Skipped:Disconnected]cluster extension can be installed with enough permission sa",
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:75493-[OTP][Level0][Skipped:Disconnected]cluster extension can be installed with enough permission sa",
"originalName": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:75493-[Skipped:Disconnected]cluster extension can be installed with enough permission sa",
"labels": {
"Extended": {},
"LEVEL0": {},
"NonHyperShiftHOST": {}
"NonHyperShiftHOST": {},
"original-name:[sig-olmv1][Jira:OLM] clusterextension PolarionID:75493-[Skipped:Disconnected]cluster extension can be installed with enough permission sa": {}
},
"resources": {
"isolation": {}
Expand All @@ -93,10 +103,12 @@
}
},
{
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81538-[Skipped:Disconnected]preflight check on permission on allns mode",
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81538-[OTP][Skipped:Disconnected]preflight check on permission on allns mode",
"originalName": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81538-[Skipped:Disconnected]preflight check on permission on allns mode",
"labels": {
"Extended": {},
"NonHyperShiftHOST": {}
"NonHyperShiftHOST": {},
"original-name:[sig-olmv1][Jira:OLM] clusterextension PolarionID:81538-[Skipped:Disconnected]preflight check on permission on allns mode": {}
},
"resources": {
"isolation": {}
Expand All @@ -108,10 +120,12 @@
}
},
{
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81664-[Skipped:Disconnected]preflight check on permission on own ns mode",
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81664-[OTP][Skipped:Disconnected]preflight check on permission on own ns mode",
"originalName": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81664-[Skipped:Disconnected]preflight check on permission on own ns mode",
"labels": {
"Extended": {},
"NonHyperShiftHOST": {}
"NonHyperShiftHOST": {},
"original-name:[sig-olmv1][Jira:OLM] clusterextension PolarionID:81664-[Skipped:Disconnected]preflight check on permission on own ns mode": {}
},
"resources": {
"isolation": {}
Expand All @@ -123,10 +137,12 @@
}
},
{
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81696-[Skipped:Disconnected]preflight check on permission on single ns mode",
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81696-[OTP][Skipped:Disconnected]preflight check on permission on single ns mode",
"originalName": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:81696-[Skipped:Disconnected]preflight check on permission on single ns mode",
"labels": {
"Extended": {},
"NonHyperShiftHOST": {}
"NonHyperShiftHOST": {},
"original-name:[sig-olmv1][Jira:OLM] clusterextension PolarionID:81696-[Skipped:Disconnected]preflight check on permission on single ns mode": {}
},
"resources": {
"isolation": {}
Expand All @@ -138,10 +154,12 @@
}
},
{
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:74618-[Skipped:Disconnected]ClusterExtension supports simple registry vzero bundles only",
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:74618-[OTP][Skipped:Disconnected]ClusterExtension supports simple registry vzero bundles only",
"originalName": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:74618-[Skipped:Disconnected]ClusterExtension supports simple registry vzero bundles only",
"labels": {
"Extended": {},
"NonHyperShiftHOST": {}
"NonHyperShiftHOST": {},
"original-name:[sig-olmv1][Jira:OLM] clusterextension PolarionID:74618-[Skipped:Disconnected]ClusterExtension supports simple registry vzero bundles only": {}
},
"resources": {
"isolation": {}
Expand All @@ -153,10 +171,12 @@
}
},
{
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76843-[Skipped:Disconnected]support disc with icsp[Timeout:30m] [Serial][Disruptive][Slow]",
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76843-[OTP][Skipped:Disconnected]support disc with icsp[Timeout:30m] [Serial][Disruptive][Slow]",
"originalName": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76843-[Skipped:Disconnected]support disc with icsp[Timeout:30m] [Serial][Disruptive][Slow]",
"labels": {
"Extended": {},
"NonHyperShiftHOST": {}
"NonHyperShiftHOST": {},
"original-name:[sig-olmv1][Jira:OLM] clusterextension PolarionID:76843-[Skipped:Disconnected]support disc with icsp[Timeout:30m] [Serial][Disruptive][Slow]": {}
},
"resources": {
"isolation": {}
Expand All @@ -168,10 +188,12 @@
}
},
{
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76844-[Skipped:Disconnected]support disc with itms and idms[Timeout:30m] [Serial][Disruptive][Slow]",
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76844-[OTP][Skipped:Disconnected]support disc with itms and idms[Timeout:30m] [Serial][Disruptive][Slow]",
"originalName": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76844-[Skipped:Disconnected]support disc with itms and idms[Timeout:30m] [Serial][Disruptive][Slow]",
"labels": {
"Extended": {},
"NonHyperShiftHOST": {}
"NonHyperShiftHOST": {},
"original-name:[sig-olmv1][Jira:OLM] clusterextension PolarionID:76844-[Skipped:Disconnected]support disc with itms and idms[Timeout:30m] [Serial][Disruptive][Slow]": {}
},
"resources": {
"isolation": {}
Expand All @@ -183,10 +205,12 @@
}
},
{
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:78193-[Skipped:Disconnected]Runtime validation of container images using sigstore signatures [Serial][Disruptive][Slow]",
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:78193-[OTP][Skipped:Disconnected]Runtime validation of container images using sigstore signatures [Serial][Disruptive][Slow]",
"originalName": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:78193-[Skipped:Disconnected]Runtime validation of container images using sigstore signatures [Serial][Disruptive][Slow]",
"labels": {
"Extended": {},
"NonHyperShiftHOST": {}
"NonHyperShiftHOST": {},
"original-name:[sig-olmv1][Jira:OLM] clusterextension PolarionID:78193-[Skipped:Disconnected]Runtime validation of container images using sigstore signatures [Serial][Disruptive][Slow]": {}
},
"resources": {
"isolation": {}
Expand All @@ -198,10 +222,12 @@
}
},
{
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:78300-[Skipped:Disconnected]validation of container images using sigstore signatures with different policy [Serial][Disruptive][Slow]",
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:78300-[OTP][Skipped:Disconnected]validation of container images using sigstore signatures with different policy [Serial][Disruptive][Slow]",
"originalName": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:78300-[Skipped:Disconnected]validation of container images using sigstore signatures with different policy [Serial][Disruptive][Slow]",
"labels": {
"Extended": {},
"NonHyperShiftHOST": {}
"NonHyperShiftHOST": {},
"original-name:[sig-olmv1][Jira:OLM] clusterextension PolarionID:78300-[Skipped:Disconnected]validation of container images using sigstore signatures with different policy [Serial][Disruptive][Slow]": {}
},
"resources": {
"isolation": {}
Expand All @@ -213,10 +239,12 @@
}
},
{
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76983-[Skipped:Disconnected]install index and bundle from private image[Slow]",
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76983-[OTP][Skipped:Disconnected]install index and bundle from private image[Slow]",
"originalName": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76983-[Skipped:Disconnected]install index and bundle from private image[Slow]",
"labels": {
"Extended": {},
"NonHyperShiftHOST": {}
"NonHyperShiftHOST": {},
"original-name:[sig-olmv1][Jira:OLM] clusterextension PolarionID:76983-[Skipped:Disconnected]install index and bundle from private image[Slow]": {}
},
"resources": {
"isolation": {}
Expand All @@ -228,10 +256,12 @@
}
},
{
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76985-[Skipped:Disconnected]authfile is updated automatically[Timeout:30m] [Serial][Disruptive][Slow]",
"name": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76985-[OTP][Skipped:Disconnected]authfile is updated automatically[Timeout:30m] [Serial][Disruptive][Slow]",
"originalName": "[sig-olmv1][Jira:OLM] clusterextension PolarionID:76985-[Skipped:Disconnected]authfile is updated automatically[Timeout:30m] [Serial][Disruptive][Slow]",
"labels": {
"Extended": {},
"NonHyperShiftHOST": {}
"NonHyperShiftHOST": {},
"original-name:[sig-olmv1][Jira:OLM] clusterextension PolarionID:76985-[Skipped:Disconnected]authfile is updated automatically[Timeout:30m] [Serial][Disruptive][Slow]": {}
},
"resources": {
"isolation": {}
Expand Down
43 changes: 23 additions & 20 deletions openshift/tests-extension/test/qe/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,7 +36,6 @@ We need to identify all cases from tests-private among all cases, then mark whic
g.It("xxxxxx", g.Label("ReleaseGate"), func() {
```
- This makes the case equivalent to origin cases for openshift-tests
- Test framework automatically sets these cases without `ReleaseGate` as `Informing`
- For the cases with `ReleaseGate` that need `Informing`, add:
```go
import oteg "github.com/openshift-eng/openshift-tests-extension/pkg/ginkgo"
Expand Down Expand Up @@ -161,6 +160,8 @@ We need to identify all cases from tests-private among all cases, then mark whic

## Test Case Migration Guide

**Required For all QE cases**: Do not use `&|!,()/` in case title

### A. Code Changes for Migrated Cases

All migrated test case code needs the following changes to run in the new test framework:
Expand All @@ -180,32 +181,33 @@ All migrated test case code needs the following changes to run in the new test f
2. **Jira Component**: Add `[Jira:OLM]` in case title
3. **OpenShift CI compatibility**: If you believe the case meets OpenShift CI requirements, add `ReleaseGate` label to Ginkgo
- **Note**: Don't add `ReleaseGate` if case title contains `Disruptive` or `Slow`, or labels contain `StressTest`
4. **Required For Migrated case from test-private**: Add `[OTP]` in case title

#### Optional Label for Migration and New
4. **LEVEL0**: Use Ginkgo label `g.Label("LEVEL0")`
5. **Author**: Deprecated
6. **ConnectedOnly**: Add `[Skipped:Disconnected]` in title
7. **DisconnectedOnly**: Add `[Skipped:Connected][Skipped:Proxy]` in title
8. **Case ID**: change to `PolarionID:xxxxxx`
9. **Importance**: Deprecated
10. **NonPrerelease**: Deprecated
1. **LEVEL0**: Use title label `[Level0]`
2. **Author**: Deprecated
3. **ConnectedOnly**: Add `[Skipped:Disconnected]` in title
4. **DisconnectedOnly**: Add `[Skipped:Connected][Skipped:Proxy]` in title
5. **Case ID**: change to `PolarionID:xxxxxx`
6. **Importance**: Deprecated
7. **NonPrerelease**: Deprecated
- **Longduration**: Change to `[Slow]` in case title
- **ChkUpg**: Not supported (openshift-tests upgrade differs from OpenShift QE)
11. **VMonly**: Deprecated
12. **Slow, Serial, Disruptive**: Preserved
13. **DEPRECATED**: Deprecated, corresponding cases deprecated. Use `IgnoreObsoleteTests` for deprecation after addition
14. **CPaasrunOnly, CPaasrunBoth, StagerunOnly, StagerunBoth, ProdrunOnly, ProdrunBoth**: Deprecated
15. **StressTest**: Use Ginkgo label `g.Label("StressTest")`
16. **NonHyperShiftHOST**: Use Ginkgo label `g.Label("NonHyperShiftHOST")` or use `IsHypershiftHostedCluster` judgment, then skip
17. **HyperShiftMGMT**: Deprecated. For cases needing hypershift mgmt execution, use `g.Label("NonHyperShiftHOST")` and `ValidHypershiftAndGetGuestKubeConf` validation (to be provided when OLMv1 supports hypershift)
18. **MicroShiftOnly**: Deprecated. For cases not supporting microshift, use `SkipMicroshift` judgment, then skip
19. **ROSA**: Deprecated. Three ROSA job types:
8. **VMonly**: Deprecated
9. **Slow, Serial, Disruptive**: Preserved
10. **DEPRECATED**: Deprecated, corresponding cases deprecated. Use `IgnoreObsoleteTests` for deprecation after addition
11. **CPaasrunOnly, CPaasrunBoth, StagerunOnly, StagerunBoth, ProdrunOnly, ProdrunBoth**: Deprecated
12. **StressTest**: Use Ginkgo label `g.Label("StressTest")`
13. **NonHyperShiftHOST**: Use Ginkgo label `g.Label("NonHyperShiftHOST")` or use `IsHypershiftHostedCluster` judgment, then skip
14. **HyperShiftMGMT**: Deprecated. For cases needing hypershift mgmt execution, use `g.Label("NonHyperShiftHOST")` and `ValidHypershiftAndGetGuestKubeConf` validation (to be provided when OLMv1 supports hypershift)
15. **MicroShiftOnly**: Deprecated. For cases not supporting microshift, use `SkipMicroshift` judgment, then skip
16. **ROSA**: Deprecated. Three ROSA job types:
- `rosa-sts-ovn`: equivalent to OCP
- `rosa-sts-hypershift-ovn`: equivalent to hypershift hosted
- `rosa-classic-sts`: doesn't use openshift-tests
20. **ARO**: Deprecated. All ARO jobs based on HCP are equivalent to hypershift hosted (don't actually use openshift-test)
21. **OSD_CCS**: Deprecated. Only one job type: `osd-ccs-gcp` equivalent to OCP
22. **Feature Gates**: Handle test cases based on their feature gate requirements:
17. **ARO**: Deprecated. All ARO jobs based on HCP are equivalent to hypershift hosted (don't actually use openshift-test)
18. **OSD_CCS**: Deprecated. Only one job type: `osd-ccs-gcp` equivalent to OCP
19. **Feature Gates**: Handle test cases based on their feature gate requirements:

**Case 1: Test only runs when feature gate is enabled**
- The test should not execute if the feature gate is disabled
Expand All @@ -223,6 +225,7 @@ All migrated test case code needs the following changes to run in the new test f
- The test executes the same way regardless of feature gate status
- Do NOT use `IsFeaturegateEnabled` check
- Do NOT add `[OCPFeatureGate:xxxx]` label
20. **Exclusive**: change to `Serial`

## Test Automation Code Requirements

Expand Down
2 changes: 1 addition & 1 deletion openshift/tests-extension/test/qe/specs/olmv1_cc.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,7 @@ var _ = g.Describe("[sig-olmv1][Jira:OLM] clustercatalog", g.Label("NonHyperShif
exutil.SkipNoOLMv1Core(oc)
})

g.It("PolarionID:69242-[Skipped:Disconnected]Catalogd deprecated package/bundlemetadata/catalogmetadata from clustercatalog CR", func() {
g.It("PolarionID:69242-[OTP][Skipped:Disconnected]Catalogd deprecated package bundlemetadata catalogmetadata from clustercatalog CR", g.Label("original-name:[sig-olmv1][Jira:OLM] clustercatalog PolarionID:69242-[Skipped:Disconnected]Catalogd deprecated package bundlemetadata catalogmetadata from clustercatalog CR"), func() {
g.By("get the old related crd package/bundlemetadata/bundledeployment")
crds, err := oc.WithoutNamespace().AsAdmin().Run("get").Args("crd").Output()
o.Expect(err).NotTo(o.HaveOccurred())
Expand Down
Loading