KOGITO-953: Cucumber tests: Test OLM subscription #332
Conversation
|
|
||
| Scenario Outline: Deploy Kogito Operator from subscription | ||
| When Kogito Operator is deployed from subscription using channel "<channel>" | ||
| Then Kogito Operator should be installed from subscription with dependencies |
There was a problem hiding this comment.
This step will check that the installed version of the operator matches with the CSV version, otherwise it fails.
There was a problem hiding this comment.
which CSV version is that ?
There was a problem hiding this comment.
The CSV version is checked from the subscription depending on the channel. It's using this method: https://github.com/kiegroup/kogito-cloud-operator/blob/c873d19970784825e8547f1367ee2649dfb2ab6f/pkg/infrastructure/operator.go#L61
radtriste
added
bdd-tests 🧪
|
Isn't such test executed on OpenShift side when they accept the operator PR? |
| @@ -0,0 +1,17 @@ | |||
| @disabled | |||
There was a problem hiding this comment.
We should do more something like this for @release tags:
https://github.com/kiegroup/kogito-cloud-operator/blob/d967c4f6331ab79d4ee71906058dd1bad2035bce/test/main_test.go#L95-L103
There was a problem hiding this comment.
I thought the tags were automatically injected. I've made some changes in regards of this, so we can define the custom tags (smoke, performance and now release).
With this change, we can now compose tags by for example, setting @smoke and another tag @performance altogether.
|
|
||
| Scenario Outline: Deploy Kogito Operator from subscription | ||
| When Kogito Operator is deployed from subscription using channel "<channel>" | ||
| Then Kogito Operator should be installed from subscription with dependencies |
There was a problem hiding this comment.
which CSV version is that ?
If this is something that OpenShift does when they accept the operator, then we should not do this. What do you think @radtriste ? |
| type customTag struct { | ||
| Name string | ||
| IsEnabled func() bool | ||
| IgnoreIfDisabled bool |
There was a problem hiding this comment.
This might be tricky as it's only used by the smoke tag because it's not excluded if it's not set. If smoke tag is set, we only run the smoke tests, if not set, we run all the tests (including smokes).
If this is too much, I can restore it as it was before, but I like the idea of having all the custom tag configurations in the same place to see/change behaviour when/if needed.
There was a problem hiding this comment.
IgnoreIfDisabled => DisableIfNotSet
There was a problem hiding this comment.
I think both are tricky and a bit verbose. More ideas? IgnoreByDefault?
|
@spolti Do you know what checks are done when Kogito operator is added to the catalog (scorecard checks?)? Does it check whether the operator is successfully installed using OLM subscription? |
|
AFAIK the scorecard is running checks before the csv is added to the OLM. |
|
@sutaakar @radtriste yeah, we run scorecard before push the new CSV. |
If this is done, should we reject the JIRA and close this PR without merging? |
|
I tried to remember why I opened this issue but cannot ... I think I just wanted to make sure we have the correct version set into the OperatorHub. With a release automation pipeline, we should not need for such a check anyway I think. |
|
No problem. Will close the PR without merging and reject the ticket. |
JIRA Ticket: https://issues.redhat.com/browse/KOGITO-953
Description: Simple test for checking that OLM subscription is working on alpha and dev-preview channels.
This test will be executed as a post-release step. So, it's disabled now and marked with the tag @Release.
Please make sure that your PR meets the following requirements:
[KOGITO-XYZ] Subject